Niyonzima was born to Rwandan parents in Train Village. He joined Walker Cavaliers in 2003 and had been a longtime player of the club. In August 2013, he signed on to Brezonde 1920, a Brezonde-based club playing in the Brunanter First League. He has been their star signing, transferring for €1 million. Despite this Brezonde did not have a good season and finished second to bottom, Niyonzima requested a transfer and joined FC Drenthe for €1 million, signing a two-year contract. He joined Sporting Markstad in August 2015, singing a three year deal.