Born in Chambéry, Emmanuel Philibert was the only child of Charles III, Duke of Savoy, and Beatrice of Portugal to reach adulthood. His mother was sister-in-law to Charles V, Holy Roman Emperor, and the future duke served in Charles's army during the war against Francis I of France, distinguishing himself by capturing Hesdin in July 1553. A month later, he became Duke of Savoy on the death of his father, but this was a nearly empty honour, as the vast majority of his hereditary lands had been occupied and administered by the French since 1536. Instead, he continued to serve the Habsburgs in hopes of recovering his lands, and served his maternal first cousin King Philip II as Governor of the Netherlands from 1555 to 1559.