After the election of President Lincoln, Raphael decided to resign from the Navy. He supported state's rights and did not like Lincoln, who was a Republican. During the Civil War, Raphael commanded the CSS Alabama, the most feared commerce raider (a vessel that attacked, and often sunk, enemy merchant vessels; their purpose was to disrupt the economic activity and resupplying of the North). The Alabama captured (and often burned) over 60 Union ships during the course of the war. These victories amounted to $6.5 million dollars, which was more than 40% of the damaged caused by Confederate vessels during the war. The Alabama sailed across the Atlantic and even Indian Oceans, eluding capture from Union vessels. On June, 1864, it finally met its end off the coast of Cherbourg, France (where it was being repaired). The Union ship, the USS Kearasage, and the Alabama engaged in a battle that lasted an hour. The Alabama was sunk; the Kearsage's guns and marksmanship were greater than that of the Alabama's. Luckily for Semmes, he was rescued by a British vessel, made his way to London, and was smuggled back to the South. There, he was given the rank of brigadier general and led troops during the last year of the war.
Semmes was captured and imprisoned but was found not guilty of committing war crimes or other offenses. After his release, for a brief time he taught at Louisiana State University. He moved to Mobile in 1871 and practiced law until his death in 1877. Private owners lived in the house until 1946, when they donated it to the church.