Kagawa was born in Kobe, Japan to a philandering businessman and a concubine. However, both parents died while he was young. He was sent away to school, where he learned from two American missionary teachers, Drs. Harry W. Myers and Charles A. Logan, who took him into their homes.