Descendants and in-laws owned the house until 1864. It is not clear who lived or owned the house for the rest of the 1800s or early 1900s, but it was used as a school, apartments, and then as a day care center during and after WWII. It also suffered periods of neglect. The city bought it in 1926 and it the next couple of decades the was repaired and restored on a few occasions. The exterior was repainted in 2012.