In the 1870s, Irvington won a competition to be the site of Northwestern Christian University, which opened in 1875. The university's policies were quite progressive for the time, admitting both women and African Americans. The school's liberal admission policies were a reflection of the Quaker and abolitionist worldview of Irvington's founders. The university remained in Irvington until 1928, when it was moved to the north side of the city.
With the development of a light rail service in 1900, a new public school, and the annexation of Irvington in 1902 increased the number of residents in the community. By the early 1900s, the area was home to some of the city's best artists and writers, including Kin Hubbard, who was the creator of the cartoon Abe Martin. The community was home to an artists' colony as well as a movement, the Irvington Group, which included such well-known Indiana artists as Dorothy Morlan, William Forsyth, and Clifton and Hilah Wheeler.
Numerous architectural styles can be seen in Irvington, including the French Empire home of Silence Benton, who served as president of Butler University for a number of years in the 1800s. The Julian house is a fine example of Italianate architecture and was the home of George Washington Julian, the brother of Jacob Julian, one of Irvington's co-founders. Julian served two terms in Congress and because of his well-known reputation as an abolitionist, hosted both Frederick Douglass and Sojourner Truth in his home at 115 S. Audubon. His daughter, Grace Julian Clarke, shared her father's progressive sensibilities, working as a leader in the suffragette movement and as the first female columnnist for the Indianapolis Star.
The Irvington Historic District also has another claim to fame. A small cottage at 5811 Julian Avenue occupies the spot that was once the home of serial killer H. H. Holmes, famous for having built his murder castle in Chicago at the 1893 World's Fair. The original structure no longer stands; only the basement of the current cottage was there when Holmes lived at the address.Holmes rented the home on Julian Avenue the year after the World's Fair.
Irvington is currently home to numerous residences, a varied business district, the last remaining buildings from Butler University, as well as churches of many faiths.