Even though the Fort was made obsolete after the Civil War, it was still used by the military, and in the 1870s, construction of the Fort Totten Officers' Club began, which occurred in stages. The three-story structure was finished in 1887. It was designed in a Gothic Revival style to resemble a castle of the Gothic age. Although the building was originally one story, when the extra two stories were added, a library, museum, theater, and YMCA were added as well. After its construction, the building was used by the U.S. Army Corps of Engineers until 1902 when their headquarters were moved to Washington D.C. Even after the Army Corps of Engineers left, the building was used by the military, serving in some capacity for both of the World Wars.
By the middle of the Cold War, the Army barely used Fort Totten anymore, and they moved out in 1974, leaving behind the historic structures that made up the property, including the Fort Totten Officers' Club, which was already being used by New York City as a Jobs Corps Center. In 1987, the City of New York received much of the property of Fort Totten from the Department of Defense, and they turned it into a public park. The park was expanded in 2001 with an additional 93 acres.
In the 1980s, the Bayside Historical Society was given possession of the Fort Totten Officers' Club. The Society set out to restore the building and maintain its historical presence. During the project, the exterior of the building was completely restored along with much of the interior. The building was placed on the National Register of Historic Places in 1986. It had already been designated a New York City Landmark in 1974.