This structure is three-bay, brick detached townhouse with a gabled roof. The home also features a Greek Revival doorway and porch, which is supported on two pairs of Doric order columnns; these architectural elements were added in the mid-1820s. The street name on which this structure is located, Freemason Street, is derived from the Masonic Lodge which occupied this lot prior to the home's construction.
When William Willoughby died in 1800, his estate included 14 slaves, 10 lots, and six residences. Eliza Francis Sharp, the third of his four children, inherited the house. It was owned by the Baylor family until its sale in 1890. The new owners converted the home into a boarding house and the original furnishings were lost during the transition. The historic home ultimately fell into decline and was slated for demolition. However, in the early 1960s, the Norfolk Historical Foundation purchased and restored the property. Since 1969, the home has been administered by the Chrysler Museum of Art. It reopened as the Norfolk History Museum in 2005 and served as a satellite gallery of American Art during 2013 to 2014.
Today, the Norfolk History Museum showcases both historical exhibits and artwork. Visitors to the museum can see artifacts that include such items as a cannon dating back to the War of 1812, as well as special rotating exhibits. The ground floor serves as exhibition space, while the second floor houses the Norfolk Rooms. Highlights of the second-floor gallery include silver pieces and paintings that were created by Tidewater artisans from the colonial period to the 19th century. The garden, designed by Siska Aurand Landscape Architects, was installed in 1991 and is based upon the current understanding of colonial landscaping practices.