Swasey was born in Canton, Maine on September 4, 1839 and attended the Canton public schools, Dearborn Academy, Hebron Academy, Maine State Seminary (Bates College), and Tufts College. During the American Civil War, he served as a first lieutenant in Company K, Seventeenth Regiment, Maine Volunteer Infantry in the Union Army. In 1863 Swasey became an attorney in Canton, Maine where he later served as a county attorney, state legislator, and revenue assessor. Swasey was elected on November 3, 1908, as a Republican to the Sixtieth Congress to fill the vacancy caused by the resignation of Charles E. Littlefield and was elected to the Sixty-first Congress and served from November 3, 1908, to March 3, 1911. Swasey died on May 27, 1928 and was buried in Pine Grove Cemetery.