Rowe was born in McAllen, Texas on February 8, 1938 to Lee Delavan and Florence (Survillo) Rowe. He grew up in McAllen, joining his local DeMolay chapter, and graduated from McAllen High School in 1956 before leaving for the United States Military Academy at West Point, New York. Rowe married Jane Rowe on December 27, 1969.