A portion of the property was bought in 1825 by John A. G. Davis, who had a new house constructed and completed by 1826. Davis was a professor of Law at the University of Virginia and one of the first to live in Pavillion X before his death in 1840. Following this, Brigadier General George Armstrong Custer used the house for his headquarters for three days during the Civil War.
The house built in 1826 is a 2 story brick structure with a slate roof, two chimneys, and a portico. Few alterations have been made to the building since its construction, but these changes include a gutter system on the roof, adjustments to windows on the west and east sides, and an addition to the northern face of the house. The Farm was added to the National Register of Historic Places in 1982 before being purchased in 1993 by architects Michael Bednar and Elizabeth Lawson, who made some restorations to it, such as the addition of the gutter system. By then, the property had been reduced to only 1 acre.