On the return trip eastward, in August 1806 the expedition encountered two fur trappers on North Dakota who were headed in the opposite direction. Lewis and Clark granted Colter to be honorably discharged so he could lead the men. However, the men had falling out and parted ways six weeks later. Colter continued to trap but began another return journey in 1807.
However, he encountered another group of traders, including members of the expedition, who were headed back up the Missouri River. At the confluence of the Bighorn and Yellowstone Rivers, the men built Fort Raymond. In October 1807, he was tasked with establishing a trade relationship with the Crow Indians. It was during this time that he explored the Yellowstone region as well as the Grand Teton Mountains. His stories of geysers, steaming water, and bubbling mud were initially dismissed and the area was dubbed Colter's Hell.
Colter finally headed east for the last time in 1810, deciding that the wilderness was too dangerous after hearing two partners were killed by Blackfeet Indians. He arrived in St. Louis where he met and married his wife, Sallie, and bought a farm. It was during this time that he met with William Clark to recount what he saw on his explorations. Colter served in the War of 1812 as a member of the U.S. Mounted Rangers commanded by Nathaniel Boone, the son of Daniel Boone. Records state that he died in 1812 of illness, possibly jaundice.