Baxter was born in North Carolina on January 7, 1855, the son of Judge John Baxter and Orra Alexander Baxter. When he was two years old, Baxter's family moved to Knoxville, Tennessee, where he lived much of his life. Baxter attended university in Knoxville but later continued his education at the United States Military Academy at West Point. He left Knoxville and enrolled at West Point in 1873 and graduated in 1877. After graduating he was commissioned a second lieutenant and he served in the Third United States Cavalry for three years. Baxter arrived in Wyoming in 1881 after leaving the Army.