The house was built by Robert Allison Brown in 1850. The house is located about one mile north of Harrisonville; it is a two-story brick structure that includes several outbuildings. The outbuildings include a slave quarters, an apple house, a smokehouse, shed, a vegetable cellar and a privy.
Robert A Brown was an early settler in Harrisonville and owned around 1000 acres of land in 1842. He built his house in 1850 and the residence still remains within the family. The slave quarters, located behind the house, once held some of Brown’s forty slaves. Despite owning many slaves, he was against Missouri seceding to the South. Since he voted against secession he was able to keep his home outside of town from being destroyed by Order No. 11 by proving his loyalty. Many who would travel through Harrisonville would stay here at “Wayside Rest.” Some of those who made their way through Harrisonville attacked the Brown home, however Robert Brown was able to hold them off, killing three of the attackers. Today, the front door of the home still has a bullet hole from this attack.