I dont think its ever bad getting more subscribers

even if they don't watch your videos the number itself convinces more actual viewers to subscribe because it makes your channel look more well liked than others.
But yes what Loki Doki said. I think you should be focusing on getting a fanbase to do the occasional giveaway with. In that respect you are correct in saying there's not much point making giveaways if the only people will be randoms scouting the net for free stuff... but is this really true?
Picture this.
You see a video for a new guitar giveaway and subscribe, like and sign up for the giveaway instantly... Cause you love guitars apparently. It seems strange that the channel only has three videos, but you dont care. You want a guitar. A month passes. You don't end up winning which is a disappointment, but you notice really good videos, that you enjoy and find funny, popping up in your subscription feed and are like, "woh, who this be?" and then you realise it was that small channel you subscribed to like a month ago. Suddenly you're going "mwahahha" cause you've just found your way into the "original" fanbase of a YouTuber you think will be big.
All I'm saying is they could be inactive subscribers, or you could really captivate these people who thought they were just going to scam a giveaway out of. Boom instant fanbase. Maybe.