Several fragments of worked stone have been incorporated in the walls of Upper Huntlywood steading, which lies about 4km west of Gordon village. These must have come from an earlier structure that once stood in the area.
Built into the western gable are the much-eroded remains of an armorial panel. Although the central portion is now missing, part of an inscription 'O F A' can be read.
Local traditions and observations of the late eighteenth century describe the remains of walls, perhaps belonging to a castle, which lay near the hamlet of Huntly. As the Gordon family owned the surrounding land, it has been suggested that these marked the site of a tower-house or other fortified place they had built. These fragments may have came from such a structure.
