The Merchants and Drovers Tavern Museum is located in the oldest remaining tavern in Rahway. The original part of the building was built sometime in the 1770s as a home and general store. At some point it was then used as stage house stop and a place to change horses. George Washington stopped here on April 23, 1789 on his way to New York to be inaugurated as the country's first president. Another Revolutionary hero, Marquis de Lafayette stopped here at some point as well. The building became a tavern when John Anderson opened it as such in 1789. It has been restored to its 1820s appearance and its aim is to interpret early tavern and travel life. Visitors are able to see the bar, tap room and guest rooms. The building was placed on the National Register of Historic Places in 1978. Next to the museum is another historic building, the Terrill Tavern, which was built around 1765 and is a good example of a 18th century home.
The Girl Scouts occupied the building between the 1930s-1960s. Local residents founded the Rahway Historical Society to save the building in 1969.