Rupp was born in Detroit in 1942. He joined the Eastern Hockey League's Philadelphia Ramblers in 1963–64. He played one game in the NHL, on loan with the Detroit Red Wings in 1963–64 replacing Terry Sawchuk. The Red Wings lost 4–1 to the Toronto Maple Leafs. He was selected as a member of the US team for the 1964 Winter Olympics in Innsbruck, Austria. During the tournament, Rupp shut out East Germany in an 8–0 victory. This feat was not equalled by a US goaltender until Ray LeBlanc held Germany scoreless in the 1992 Winter Olympics. The US team finished in fifth place.