This scenario is kind of subjective, without anyone new appreciating the work you're putting into your videos, time will take it's toll on you and you'll begin to lose motivation due to not having anymore people to entertain, it'd get kind of lonely too; that last part is for me anyways. But seriously, not growing at all and freezing in place with your channel to me would feel very...it's hard to describe.
I'll be honest and say, I'm in the middle if I'd continue or not; because like I said above, motivation would start eating away at you without any new viewers to keep you, well, motivated xD