The Ridgeley family built the site's second home on the other side of the hill. From this spot the town of Mt. Savage is visible. Ridgeley was the owner of 10 slaves. Ridgeley also worked as a county commissioner and helped to design the town of Frostburg's layout. Ridgeley also helped to develop ties between the B&O Railroad and the C&O Canal companies. These connections helped the region to more quickly develop new and more expedient lines of transportation.
The property was split and the house built by Grimes was sold to George Winter in 1822. Winter resided in the Grimes home which he expanded to become a larger, plantation-style manor. When the property was again sold in 1869, its new owners, the Trimbles, expanded the Grimes home again to form a Victorian-style mansion. The Trimble family operated a farm and invested in the regions coal mines.
The Evergreen owners formed the Evergreen Heritage Center Foundation in 2008. The goal of the foundation was to make the private property available to the public for the purpose of education. The Evergreen Mansion has been converted into a museum and the barn features plantation era farming tools. On the property, there is an Evergreen Coal Trail which goes along an old tramway that once hauled coal out of mines in the early 1900s. Hikers on the trail can see three mine openings, waste slate, and the remains of other mine features like the blacksmith's forge and water tower.