The building was designed by architect Herman Esser in 1910, and an addition was designed by his son Verner Esser in 1927. It is constructed of white glazed brick, giving it a modern, sleek appearance. The addition was made out of a cream-colored glazed terra cotta, with hints of the architectural styles of the Italian Renaissance seen in the decorative elements on the roof. All of the architecture was made to reflect the attention to cleanliness and modern sanitation practices going on inside.
After Blommer sold the company to National Dairy in 1929, the Blommer family moved forward, creating the Blommer Chocolate Company in the 1930s, which still exists today. Their original factory's windows were bricked up and the space was used for storage. After that, the building remained vacant for about 20 years until restoration projects for the surrounding neighborhood began in the 2010s. In 2017 the old factory gained a place on the National Register of historic places. Plans to convert the building into apartments were created in 2014, but it wasn't until 2018 that the project was completed. Today the ice cream factory is now the Legacy Lofts, a collection of 64 apartments in two buildings. The former factory houses 38 apartments, while a new adjacent building houses the rest.