In 1793, a year after the first lighthouse was built at Cape Henry, the Whitehurst family built their farmhouse. Daniel Whitehurst, appointed as a Commissioner, authorized the building of the Virginia Beach Courthouse on a property near his farm in 1820. The Whitehurst farm was also home to James Howard Whitehurst who joined the 16th Virginia Regiment of the Confederate Army in 1862. He was captured in April 1865. His tombstone remains on the property. The property stayed in the Whitehurst family until 1940 when the farm was sold to the Bratten family. In 1953, the Bratten family sold the farm to the Buffington's, who maintained the property until 1986. Currently, the home is being maintained by the Whitehurst-Buffington House Foundation as a museum.
The original house was white clapboard and included one room downstairs and a sleeping loft upstairs. Overtime the house was bricked and enlarged by the Whitehurst family. The gambrel roof in front and the long sloping roof in the back are indicative of the “saltbox” design that was popular during the early Federal period. The home has two large chimneys on either end of the home and the front windows incude louvered shutters. There is also preserved hardware such as a lock from 1793.