Backstory and Context
Soon after the Civil War began in 1861, Smith, who was a southern sympathizer, and his family fled to the Confederate capital city of Richmond. This allowed the Union seize the home (and others in the neighborhood) to use for military purposes. At first, the home became the headquarters of military Governor John Slough, who made a name for himself leading a Union force to victory in a confrontation with a Confederate force in New Mexico (the decision to attack was his, defying orders of his superiors; President Lincoln, who was desperate for some good news, disregarded this defiance and appointed Slough to the position).
As noted above, the home eventually became a hospital with room for 100 soldiers. Apparently, some of these men carved their names into the woodwork. After the war, Smith returned and converted it back to a home. It was from here that Smith tried to advise Lee on how to recover property the Federals seized. This property is what is now Arlington National Cemetery.
"From Opulent Home to Military Hospital and Back Again." Alexandria Times. Published by the Office of Historic Alexandria. June 8, 2014. https://www.alexandriava.gov/uploadedFiles/historic/info/attic/2014/Attic20140508CWHospital510Wolfe.pdf.
JDF. "The Vowell-Smith House, Alexandria, VA." The Picturesque Style: Italianate Architecture. June 2, 2013. http://picturesqueitalianatearchitecture.blogspot.com/2013/06/the-vowell-smith-house-alexandria-va.html.
Santantonio, Alex. "Admiring the Vowell-Smith House, the Largest Home in Old Town Alexandria." Old Town Home. September 5, 2011. https://www.oldtownhome.com/2011/9/15/Admiring-the-Vowell-Smith-House-the-Largest-Home-in-Old-Town-Alexandria.
Old Town Home: https://www.oldtownhome.com/2011/9/15/Admiring-the-Vowell-Smith-House-the-Largest-Home-in-Old-Town-Alexandria.