Xu was born in Jiangsu with a scholar family background. He was one of the youngest persons ever to pass the Imperial examinations. In 1905, he was accepted into the Japanese School of Land Army Officials, and returned to China in 1910. From 1911 to 1917, he served in the First Army with various positions in the general staff, such as chief of Logistics Department, deputy chief of land forces, chief of land forces, etc. In 1914 he founded a middle school called Cheng Da Middle School, which is the predecessor of today’s Affiliated High School of the Capital Normal College.