Construction of this historic home began in 1815 for the family of Alexander Watson Batte who inherited the property from his father, William Batte who purchased the land in 1778. James Brown, a significant planter who became wealthy through the labor of enslaved persons acquired the property after Alexander Batte died in 1853. Brown died two years later and his property was transferred to his nine-year old son James David Feilding Brown. The estate was managed by the courts and an overseer until Brown sold it in 1874. By that time, Brown had graduated from the University of Virginia and become a minister.