Born in Tatong, Victoria, Australia, Savage first became involved in politics while working in that state. He emigrated to New Zealand in 1907. There he worked in a variety of jobs, as a miner, flax-cutter and storeman, before becoming involved in the union movement. Savage initially opposed the formation of the original New Zealand Labour Party as he viewed the grouping as insufficiently socialistic. Instead he became the chairman of the New Zealand Federation of Labour, known as the "Red Feds".