The First Presbyterian Church of Port Gibson was first established along Bayou Pierre in April of 1807. The Presbyterian Church moved to there current home in Port Gibson in 1827. When the congregation outgrew the small size of the current church, the decision was made to build a new church with its now famous steeple in 1859. The first sermon in the new building would not be held until December 16, 1860 because the northern contractor failed to finish the job and had to be completed by Elder H.N Spencer. The first edition of the Hand pointing to Heaven was made out of wood by Daniel Foley but it did not last long and was replaced by the current hand in 1901.