Banner was born into an Austrian Jewish family. He went to college planning to become a lawyer, before leaving after two semesters and then started training to become an actor. After the Anschluss of 1938, he was captured by the Nazis and placed in a concentration camp. He was later released, as he was captured before the implementation of the Nazis' extermination policies. After his release, he emigrated to the United States as a war refugee. He died on his 63rd birthday in January 1973 from an abdominal hemorrhage.