William Booth Taliaferro (who pronounced his Italian surname as "tah'-liver") was born in Gloucester County, Virginia, to a prominent family of English-Italian origin who settled in Virginia in the 17th century. He was the nephew of James A. Seddon, who would become Secretary of War for the Confederate States of America under Jefferson Davis. Taliaferro attended Harvard University and William and Mary College, graduating from the latter in 1841.