The house would reach its present appearance by 1820 with Poindexter adding on the north and south wings. The building is notable for the mix of decorative styles, such as the use of different woodwork in the northern wing. Further renovations would be done on the south wing in 1935.
Following Parke Poindexter's death in 1847, the house would pass through a number of different hands. Of the 300 acres originally in Poindexter's possession, 38 acres (which included the house) was purchased in 1957 by William Gilmer, the owner as of the plantation house's inclusion in the National Register of Historic Places in 1976.