Listed on the National Register of Historic Places, the Hiram Butler House is a one-story, brick Georgian-plan house dating back to at least 1870. It has a unique Georgian form and floor plan with an original L-shaped rear wing. The structure is an excellent example of a mid-19th century upper middle-class farming home. In terms of architecture, the house is a fine example of the Italianate and Greek Revival styles. The home has undergone some noted modifications over time but still retains much of its original integrity. Today, it is the focal point of a the Smith-Gilbert Gardens, which features several gardens, a trail, and a butterfly house.
The house is named after its builder, Hiram Anderson Butler (1813-1913), who worked his entire life for the Western and Atlantic Railroad, which later became the Nashville, Chattanooga, Butler & St. Louis Railway. In 1880, he bought 160 acres of land (he may have eventually acquired more land) and built the house, living there for many years. In addition to working for the railroad, he grew crops such as corn, oats, peaches, and apples. Butler also served in the Confederate Army during the Civil War, rising to the rank of major. He was one of the men who captured Union raiders who participated in the Great Locomotive Chase, a military raid by a group of Union soldiers and who stole a Confederate locomotive and rode it north, causing as much damage as they could in the process.
The Butler family owned in the house until 1913. After a series of owners, local resident Robert Gilbert bought it in 1970 and added the gardens, which feature sculptures that he collected. In 2005, the city bought the house and and by this time the property decreased to 13 acres.