In the manga He had only been to Soul Society on two occasions: to rescue Rukia and then later for the fight where he defeated Aizen in the real Karakura Town, which had been moved to Soul Society. In the anime he has been many, many times. You would have to be more specific about it.