In 1799 he commissioned Dr. William Thornton to build him a winter home in Washington, D.C., to serve as his winter residence. The building was completed in 1800, located at the intersection of 18th Street and New York Avenue, between the Potomac river and the President's residence. The three story building is built of red brick with Aquia Creek sandstone used for the window sills and decorative panels. Other notable features include the exterior ionic columnns, the interior Corinthian columnns, the urn-shaped cast-iron stoves, and the lavishly decorated Treaty Room on the second floor. In 1840 the original flat roof was replaced with the hipped roof seen today. The interior has a restrained yet sophisticated Classical style. Upon closer inspection the visitor will notice that the building is not, in fact, an octagon but an irregular hexagon.
John Tayhoe passed away in 1828, and his wife, Ann Ogle, assumed control of the house. She lived there until her own death in 1855, after which it remained under occupancy of the Tayhoe family for a further fifty years. The building then became a school for girls, and subsequently served as the Navy Hydrographic Office. The American Institute of Architects purchased the building in 1902, at which point it had become derelict and in need of repair. After a period of restoration it became the headquarters of the AIA. In 1968 the AIA transferred ownership of the building to the American Architectural Foundation, who turned it into a museum two years later.