He was the grandson of the last Umayyad governor of Khurasan, Nasr ibn Sayyar, and served as governor of Samarkand under Ali ibn Isa ibn Mahan, the governor of Khurasan. Ali's ruthless exploitation of the province and oppressive fiscal measures caused much resentment among the local elites. Consequently, when in 806 Rafi launched a revolt in Samarkand, it spread quickly across Khurasan, finding support both among the Arabs and the Iranian natives. Rafi also secured the support of the Oghuz and Karluk Turks.