In August 1945, the Fayette County Commission was presented with a petition of more than 20 percent of the voting population asking for the construction of a memorial building to honor the men and women of the armed forces. When the building was established, it was determined the building would be two stories with an auditorium and stage area for various performances and sporting events. The basement would be used for armory drills, farm fairs, physical education and for the National Guard. Further, they decided, the memorial would be constructed at or near Fayetteville, the county seat. The structure was designed by Ludwig Theodore Bengtson, known for the Young-Noyes House and St. George Orthodox Church in Charleston and the Raleigh County Courthouse in Beckley.
A “Wall of Honor” plaque is displayed at the north end wall of the Soldiers and Sailors lobby. Erected in the 1990s, it provides citizens a place to honor loved ones who served in the armed services by inscribing their name, rank and years of service. The Fayetteville branch of the Fayette County Public Libraries is also located in the building. The basement originally included an open dining hall, meeting rooms, extra locker rooms and a kitchen. It was later converted into a dispatch room for Fayette County Emergency Operations Center, but has been vacant since the center moved in 2013. Fayette County plans to use this vacant area as records management and preservation storage space. Since 1952, and still to this day, the memorial's gymnasium has been used by the Fayette County Board of Education for indoor athletics.