In 1828 it became the county's almshouse, after William Brown was forced to relinquish the property to his creditors. A few modifications were made to the interior to accommodate its new function, including the installation of heating and plumbing, but the interior remains essentially the same as it looked in the 18th century. It was managed by Anne Arundel County as a residence for the poor and destitute until 1965.
Today the William Brown House is owned by Anne Arundel County and managed by the London Town Foundation as a museum. It is the focal point of the Historic London Town & Gardens, which also features a reconstructed Colonial-period carpenter's shop and Lord Mayor's tenement building with a kitchen garden, and a tobacco barn. The site serves as the venue for weddings and a variety of corporate events, but places a heavy emphasis on education and offers a range of programs for children and adults alike.