Both the Layton Gallery and Institute grew and promoted and fostered arts in Milwaukee through various programs in the 1920s and 1930s. The Layton Gallery lent some of its art other other institutions, opened a Wisconsin gallery, founded the Layton School of Art, and organized traveling exhibitions. The Institute increased its arts education programs and became the home of the local arts community.
After World War II, business leaders called for a building that would promote art and music dedicated to Milwaukee veterans. The Metropolitan Milwaukee War Memorial Corporation was formed and it raised the funds to hire an architect (Eero Saarinen) and build The War Memorial Center, which opened in 1957. The Layton Art Gallery and the Institute moved into the new building, combing their art collections to form the Milwaukee Art Center.
The center's collections grew exponentially when Mrs. Harry (Peg) Lynde Bradley donated her entire collection of over 600 works and she called on the community to build a new addition to the museum to house them. The addition, which was designed by architect David Kahler, was completed in 1975. Peg Bradley lived in an apartment on the upper floor for several years and it was eventually converted to exhibit space.
The center changed its name to its current one in 1980 and received accreditation from the American Association (now Alliance) of Museums in 1983. The museum received additional collections around this time as well. Looking to expand further, the museum began building the Quadracci Pavilion in 1997 and it was finished in 2001. The additional space added 341,000 square feet of space which includes a traveling exhibit hall, a cafe, a theater, and a gift shop.