The Whitfield-Manjiro Friendship House Museum and Cultural Center, located in the restored Captain Whitfield home, celebrates the remarkable story of Japanese native Nakahama Manjiro and his ties to Fairhaven. Manjiro was the first Japanese person to live in America (in Fairhaven; see below for the backstory). He learned English, received an education, learned shipbuilding techniques and how to sail, and eventually studied military science in Europe. He returned to Japan and there he shared his knowledge about western science and culture, worked as a translator, and was instrumental in helping Japan modernize. The museum was founded in 2008. In 1987, Fairhaven and Tosashimizu, where Manjiro was from, agreed to be sister cities.
In 1841, he and four brothers set out on a fishing trip. Their boat crashed on an island in a storm and they managed to live there for several months. They were eventually rescued by the crew of a whaling ship, the John Howland, led Captain William Whitfield of Fairhaven. Manjiro asked to return to America where he lived from 1843-1846, learning English and receiving a good education. He also grew close to Captain Whitfield and the people of Fairhaven, who treated him kindly. He then worked on a whaling ship and then made his way back to Japan. He never forgot about his time in Fairhaven and later in life visited Whitfield and his other friends. He and his friends could not immediately return to Japan since at that time Japan had a "closed door" policy and any native who left could not return; if someone did they would be put to death. However, when Manjiro and two of the brothers returned they were only in custody and questioned for a relatively short time. Manjiro's value and experiences were quickly recognized as very valuable.