Lambert was born in Trenton, New Jersey, to Doctor Thomas and Hannah (Lambert) Cadwalader. By 1750, his family had returned to Philadelphia, Pennsylvania, where he attended Dr. Allison's Academy. Then, in 1757, he entered the City College (later the University of Pennsylvania), but did not graduate. Instead, he went into business with his brother John.