The castle was founded by Karlon Stark, a younger son of the then-King in the North after he was rewarded with lands by his father for putting down a rebellion by House Bolton of the Dreadfort. The castle was then called "Karl's Hold". Over the centuries Karl's Hold became Karhold, and the Karl's Hold Starks became House Karstark.