Young died in 1906 and the house remained in the family until 1939 when his granddaughter donated it to the Episcopal Church to be used as a school for girls. However, the church ran into financial problems and the school closed. The county acquired the property in the 1960s and the main floor was converted to a French restaurant, the Stone Manor Inn, in the 1970s. The restaurant eventually closed. It is not clear when, but a Chicago investor bought the mansion and converted it to the condominiums.