After an impressive junior career with the Toronto Marlboros of the Ontario Hockey Association, during which Caffery was drafted third overall by the Black Hawks in the 1966 NHL Amateur Draft behind Barry Gibbs and Brad Park, he played a season for the Canadian National Team. He made his NHL debut with the Black Hawks in 1969, but spent most of the next three seasons in the minor leagues with the Dallas Black Hawks of the Central Hockey League and the Cleveland Barons of the American Hockey League, averaging over a point a game and winning the AHL's Dudley "Red" Garrett Memorial Award as rookie of the year in 1972.