The location is also the former site of the Gibson House. The Gibson House was a hotel that operated in Jackson during the Civil War and continued to operate until it was abandoned in the 1990s. In the early 2000s, the Gibson House was demolished by the city along with the city's second high school, Kinison. When General John Hunt Morgan stopped in Jackson during his famous raid through Indiana and Ohio, some of Morgan's men stayed at the Gibson House while Morgan himself stayed at the Isham House, located down the street where the Cambrian currently sits.
The Memorial has plaques dedicated to all the men from Jackson County awarded the Medal of Honor, including John Wollam. John Wollam was among the first recipients of the Medal of Honor, along with the rest of the volunteers who took part in Andrew's Raid, otherwise known as the Great Locomotive Chase. The goal of the raid was to damage and disrupt the Confederate infrastructure. The raid ended in failure, but all of the eligible participants were awarded the Medal of Honor.