The first major artist to make a home in Gloucester was arguably Fitz Henry Lane, who became famous for his renderings of the town's harbor. Over the course of the mid-1800s, more artists continued to come to Gloucester, mostly for summer visits. Winslow Homer summered in Gloucester in 1873 and 1880, bringing national attention to the community.
Regarding the founding of the artists' colony, perhaps no other artist had a more influential role than Frank Duveneck. Duveneck visited Rocky Neck numerous times over a quarter-century span, though the exact location of his studio is unknown. It is also possible that Duveneck simply painted on the grounds of the Rockaway House, a hotel which catered to the area's artists. Numerous members of Duveneck's coterie, often referred to as Duveneck Boys spent at least some time in Rocky Neck.
Over the years, numerous artists spent time at Rocky Neck, including Edward Hopper, Stuart Davis, Milton Avery, and Mark Rothko, to name just a few. The community still functions as an artists' colony, with its office located at 6 Wonson Street, in the building that was formerly Giles Chapel.