Holmes was born on May 1, 1826 to a noble family in Banbury, where his father, the 4th Earl of Abingdon, held a large estate. Often, he was criticized by his contemporaries for his start as a young noble, because a lot of other powerful steam magnates had started their own companies from scratch. He was raised by private tutors from around the British Empire, and was well-educated by the time he went to Oxford University in 1844. He earned a Bachelor Degree in Mathematics in 1850, and then went on to earn a Master's Degree in Industrialization in 1852. He was at the top of his class for both of his graduations.