It can be difficult, and sometimes, you have to take a good long look at what you're doing and what your goals are. For me, I know that there's never going to be a huge call for "puppet videos not for kids but not edgy for adults, about sincere, gently-humorous subjects." It's just a really specific thing that I like to do, and the fact that I can do it for a couple thousand people makes me happy.
That's not everyone, though. Some people like to build their audiences in the hopes of making a living at this, and that requires a totally different strategy. If I were looking for high views and subscribers, I'd find what's trending, what seems to be on the rise, and move towards that as aggressively as possible.
Both approaches (and many others) are totally valid. It's just about what you want to do.