Phelps was born on January 13, 1824 in Chardon, Ohio, and was appointed as a Midshipman in October 1841. When the Civil War broke out in 1861, Phelps was a Lieutenant commanding the gunboat USS Conestoga. He gained command of two additional timberclad gunboats, the USS Tyler and USS Lexington, and as such was instrumental in the Union victory at the Battle of Fort Henry on the Tennessee River in 1862, in which he served as part of General Ulysses S. Grant's invasion force. He was promoted to Lieutenant Commander in July 1862, and resigned from the Navy in October 1864.