Commandants who lived there served during the American Civil War, the Spanish-American War and World Wars I and II. The importance of the naval yard began to decline after World War II, during which thousands of workers built ships, including the U.S.S. Arizona which was sunk at Pearl Harbor in 1941. The yard closed in 1964, and the house was sold to a private owner and remains so today.
New York City designated the house as an historic landmark in 1964 and ten years later in 1974 it received National Historic Landmark status.