Matthew Harvey Anderson, Merchant and Director of the Bank of Madison, built this house sometime after July of 1904, the date when he purchased Lots 23 and 24. It was occupied by Dr. James Allen Kyser for about 50 years before falling into neglect. Tony and Cindy Sensenberger purchased the ready-to-collapse structure at auction in 1997.
The house is located on original Lots 23 and 24, including possibly part of Lot 22 in the yard. Lot 24 was first purchased from James Clemens by carpenter Edmund “Ned” Martin on 5 March 1860, just about three months before Clemens’ death and about a year before the start of the Civil War and the associated Emancipation Proclamation. It was the last lot sold by Clemens himself. Edmund was a “free man of color” per deed records and the 1860 census. That census showed that Edmund was thirty years old, had a wife named Sarah at age forty-five and a daughter named Lucinda at age nine, all born in Alabama. They were enumerated in a complex of free black households clustered around the house of blacksmith William T. Dunnivant, who lived next door to Theodorick Clay. Clay lived on today's Front Street at the west side of Buttermilk Alley. Additional data in the Madison County deeds show that Edmund Martin was born in Morgan County. He was living in Valhermosa Springs of Morgan County at the time of the 1870 census.
It may be that events related to the Civil War led to Edmund Martin abandoning Lot 24 or otherwise leaving Madison. The fact is that after the Reconstruction-era chancery court’s handling of the Clemens estate in 1868, physician and depot agent William Dunn sold Lot 24 to Bettie Turner. She was noted as his “coloured” servant in Deed Book JJ, pages 455–6.
Matthew Harvey Anderson purchased the lots in 1904. Harvey Anderson was one of the Directors of the Bank of Madison, founded in 1904. He also was a principal partner of Anderson–Bronaugh & Company as well as the Hertzler–Anderson Company. Both businesses were general stores, which sold groceries and other merchandise, according to the 1905 Alabama Merchantile book listings. Harvey and his wife, Annie Hertzler Anderson, were both from Ohio with Pennsylvania roots. In fact, Annie’s father, Dr. John Hertzler, also came to Madison from Ohio after the Civil War. Annie's brother, Frank Hertzler, later built and lived in the house at 25 Front Street.
After constructing a large house on the lot along Front Street, the Andersons sold it by 1926 to Dr. James Allen Kyser. Dr. Kyser was recognized by the U.S. government for helping to bring the 1918–9 flu epidemic under control in north Alabama. The Kyser family lived in the house for nearly fifty years, and then it was purchased by Billy J. and Nancy Jane Jones of Huntsville. In 1997, the collapsing and abandoned old house was purchased at auction by Tony and Cindy Sensenberger. They lovingly restored and expanded it, keeping as much of the original structure as possible. Today the house has surpassed even its former glory.