For a number of years, the small town thrived. Abundant timber in the area led to the construction of a sawmill nearby on the Palouse River. The Oregon Railway & Navigation Co. built a rail line through the area, and settlers continued to move into Elberton. Apple and plum orchards proliferated and became a vital part of the town's economy.
At its peak, Elberton had a population of about 500, as well as three churches, a post office, two stores, and a flour mill. From 1893 to 1924, the town held an annual three-day event called the Elberton Picnic. By 1896, the event was well-known enough that William Jennings Bryan made a stop during his campaign for president.
Elberton's prosperity lasted only about a decade. In the early 1900s, a series of setbacks and disasters led to the demise of the town. By 1906, the area around Elberton was deforested to such an extent that the sawmill which had contributed to its founding moved to Idaho. The railroad ended its service to Elberton the following year, and in 1908, the flour mill closed. In 1908, a devastating fire burned much of the town and in 1910, a flood destroyed much of what was left. Many of the businesses were deemed to costly to rebuild.
For the already struggling community, the Great Depression was the final blow. Sadly, the final demise of Elberton in the 1920s and 1930s was a scenario repeated throughout numerous agricultural towns. By the mid-twentieth century, only a few residents called Elberton home, and the town was deincorporated in 1966. In the ensuing years, local fire departments burned many of the abandoned buildings as training, leaving only foundations as reminders of the once prosperous community.
The only remaining intact building in Elberton is the United Brethren Church. Though it has been unused for many years, the building is in reasonably good condition. The town's cemetery remains, as do the remains of the railroad. Elberton was listed on the National Register of Historic Places but removed in 1990.