Born Nell Ruth Hardy to Horace and Edna Mae Hardy in Birmingham, Alabama, Carter was one of nine children. She overcame adversity and personal hardships before finding success as an actress. Her father died in an accident with a power line. A man raped her when she was 16, and she became pregnant from the attack, giving birth to a daughter, Tracy.