If you're showing your face, then obviously it's going to affect your results somewhat - however the majority of those extra clicks/subs will be the kind of people who click through to a video because they see someone they find attractive, not because of the video content - who likely aren't your target audience anyway to begin with (unless that is something you want to capitalise on) so I don't think the actual difference it makes is all that dramatic.
That said, I think presentation is important. You might not be able to do much about the looks you were born with - but you should make an effort to at least look presentable. It does make a huge difference. I cringe watching my old videos now because I didn't care about presentation what so ever and I looked like a complete mess. There's been a definite difference in how well received my videos are since getting my hair cut etc. (finally, it took a lot of convincing since I was so used to my long hair, and even though it didn't particularly suit me it was hard to get rid of)
So, my suggestion: Don't worry about whether you're a model or average or a bit rough around the edges - but do take care of your clothes, hair, face, etc for best results.