In February 1922, W.R. Jagoe bought the house for $15,000, or $215,500 today. Jagoe was the President of the Daviess County Planning Mill Company and a former city council member. His son Carlos Jagoe would go on to found the family-owned Jagoe Homes Company, which is still prominent in Owensboro. One of his other sons, Julius, lived in the house after W.R.'s sudden death in 1937 before the Jagoe heirs sold it to Judge T.F. Birkhead in July 1940. In turn, Birkhead granted the house to one of his daughters, Eva Belle Birkhead. She shared the home with her sister, Flora Lee Birkhead. Miss Flora, as she was known by her many students, gave music lessons in the house for years. Because of the association of Miss Flora to the residence, it became known as the Sweeney-Birkhead House.
Eventually, the house passed out of the Birkhead family and became the Old House Restaurant in the 1970s. The restaurant closed in 1978, and has since been a private residence.The house joined the National Register of Historic Places in September 1980.
The house is a prime example of the Queen Anne style because of the polygonal tower at the Southeast corner and the projecting window bays. A necklace design can be seen throughout the stained glass windows, of which all but one are original. This theme has accounted to a love by Mrs. Sweeney for jewelry, though the origin of the story remains unknown. The house also boasts 10 ﬁreplaces and parquetry floors. It still retains much of the original woodwork, and is one of the few houses of architectural signiﬁcance left in the downtown area.