For me an average gamer and someone who really only watches gaming stuff for figuring things out when stuck like walkthroughs I dont mind too much but I do know that a lot of people get bored quickly watching someone else game when they aren't really talking throughout it, I think as long as you avoid too much silence, have good quality footage and clear audio then you should do fine.
So to answer your questions, no they dont need to be loud or energetic but I think enthusiasm is always good, if someone sounds bored for example that could put people off. I think as long as they are sounding into what they're doing and having fun then it is easier for the viewer to watch. If you were like "Right... 10 seconds silence accompanied by a long sigh... im playing Modern Warfare 3 today...10 more seconds...Oh its loaded now... yeah" all in a bored sounding voice I think that will put people off a lot, as long as you avoid stuff like that then you're on the right track.
Thats my few cents on it.