The exterior is wood painted to look like stone, which is always a classy look, but the block houses you see on either side are actual stone, meant for defense. When the house was built, the French and Indian War had just ended, so it made sense to have the house ready for an attack. That, or William Johnson made some well-armed enemies in his business dealings.