Howard "Doc" Ayers was born in Toccoa, Georgia. He is the son of Dr. and Mrs. C. L. Ayers. The nickname “Doc” came to him because his father was a medical doctor. He married Miss Connie Terrell who graduated from Shorter College. They have three children together named Bucky, Bunny, and, Buzzy.