In 1981 Danny Hansford, Williams' assistant and partner, was killed in the study and Williams was charged with the murder. In the first trial, he was found guilty, but it was speculated this result was only because he was a gay man. The second trial resulted in the initial ruling being overturned due to contradicting police reports. The third trial resulted in an undecided jury. After four trials he was found not guilty, but he died of pneumonia six months later in the home. The incident was depicted in the movie Midnight in Garden of Good & Evil starring John Cusack and Kevin Spacey, who played Williams. The movie was based on a non-fiction book written by John Berendt.
The house is rumored to be haunted, because of the home's dark history. Along with the deaths of Hansford and Williams, an eleven-year-old boy named Tommy Downs died after falling from the roof or second story balcony while chasing birds in 1969. It is thought he may have been pushed to his death. The house is now owned by Jim Williams' sister Dorothy Kingery, and it operates as a museum open for daily public tours.