I know sometimes my sub count will go down after I get a sub, sometimes it's someone subscribing then unsubscribing right away, sometimes YouTube will 'flush' a spam account out of your subscriptions when someone subscribes because the list get refreshed by the REAL person subscribing, I've also noticed the same thing if someone unsubscribes, sometimes I'll lose 2-3 subscribers in one go because of the same thing