When Frederick and Louise Vanderbilt rebuilt the estate home on 600 acres, known as Hyde Park, there already was an established village to hire staff for the grounds, gardens, and to keep the mansion running. The earliest development of the estate began in 1764 when Dr. John Bard purchased land; family ownership continued through 1821 with his son, Dr. Samuel Bard (1742–1821), owning the property from 1799 to 1821.
His colleague and fellow-landscape garden hobbyist, David Hosack, president of the New York Horticultural Society, purchased the property from Samuel Bard's heirs in 1928. In 1840, John Jacob Astor purchased the property from Hosack's heirs for his daughter Dorothea and her husband Walter S. Langdon.
The stores, hotel, shops, services, offices, and a pearl button factory on Market Street were the hub of Hyde Park.