Prior to his professional career, Wager played basketball for Saint Mary's University. During his college career, he was an all-conference center and an individual scoring champion twice, in 1939 and 1940. He also participated in the 1941 National Invitational Tournament. He was also selected in his senior year as captain of the school's basketball team. He also played football for the school. He was elected in the schools sports hall of fame in 1977.