The city later wanted a statue built outside the building to enhance the exterior design. The statue was to serve as a Civil War memorial. The city held another competition to find an artist to create the statue. The winner was Philip H. Martiny, who was a student of Augustus Saint Gaudens. Gaudens sat on the panel that was in charge of selecting the artist. Martiny's design for the statue included a seated female who was inspired by the Roman goddess Bellona. The statue stands nine feet in height, and it is made out of bronze. Standing atop a granite pedestal, the female figure holds a sword in her left hand and a laurel wreath in her right hand, has a shield over her knees, and wears a helmet.
To prevent an accident, the brass cupolas and the central tower were removed from the building in 1955. Then, in 1979, a fire burned five copper friezes. For these reasons and others, renovations were necessary, but funding was a problem for many years. In 2008, the city received a $300,000 grant from the Hudson County Open Space & Historic Preservation Trust Fund. This money was used to start the renovations, which ended up costing $1.9 million before it was completed. The renovations included fixing the plaster friezes, stained glass windows and dome, as well as adding modern amenities such as HVAC and electrical wiring to the chandeliers, which were original.