Nobody is going to go through the trouble of finding you via YouTube this way, too much of a pain. It's actually far easier to figure out where someone lives by the trail they leave online.
If you're worried about being found, you need to check out everywhere you've used the same nickname, email(s), etc and make sure you haven't given it away anywhere. One of the biggest holes tends to be social media, specifically Facebook. If you've used the same nick or given links to your YT anywhere on there and it's not locked down tight, it's easy as pie to find your posts and then it's not much work to find all your info via your page or your friends' pages.