James Dinsmore and John Minor bought around 340 acres of land in Louisianna to start a sugar plantation. They gave the plantation the name Bayou Black. James lived on the plantation and ran it while John was just providing half of the necessary funds. By 1831 there were forty to sixty African American slaves working on the plantation.
In the 1820s James met his future wife Martha Macomb. They got married on May 13, 1829, in New Jersey. After their honeymoon to Yellow Springs Martha stayed in Cincinnati. This is where she gave birth to her first daughter, Isabella. She then moved to Louisiana to be with James in 1831. Two years later she gave birth to their second daughter, Julia. Their last daughter Susan, was born at the plantation as well in 1835.
In 1839 James traveled north to look at the land where the Dinsmore House now stands. He bought the land and the construction of his home began. It was finished in 1842 and his family had moved north and settled down in their house. James brought eleven slaves with him to do all the farming. Potatoes, corn, beans, oats, and hay were the main crops that he grew. He also had a vineyard that he used to produce wine as well as sheep for wool. In 1854 the tenants that lived on the farm started a basket-making factory and began growing willow trees.
Their youngest daughter Susan passed away in 1851 in a boating accident on Lake Erie. Martha only lived for 8 more years after Susan passed away. When James passed away in 1872 his daughter inherited the farm since she never married. Julia kept a journal about her time managing the farm which can be read on the Dinsmore House website.