De Lisle was born at Lons-le-Saunier, reputedly on a market day. His parents lived in the neighbouring village of Montaigu. A plaque was placed at the precise spot of his birth and a statue erected in the town's centre in 1882. He was the eldest son of Claude Ignace Rouget (April 5, 1735 - August 6, 1792) at Orgelet and Jeanne Madeleine Gaillande (July 2, 1734 - March 20, 1811).