Before Hooper purchased the house, he and his family were driven away from their Finian estate in Wilmington by the British army. When Hooper died in 1790, he passed away in the house and was buried on the property. After his death, heirs of William Hooper retained ownership of the house until 1853. After the American Civil War ended in 1865, a former governor, United States secretary of the navy, and Confederate senator named William Graham purchased the house in 1870. Much like the death of Hooper, Graham died in the house and was buried on the property on the east side of his garden, but his remains were reinterred in 1894 at the Guilford Courthouse National Military Park in North Carolina.