The grey cobbles in the center of the castle courtyard show where the remainder of Daredevil Tower (the original keep) was located.