Might have to post a video for an example but I think I have an idea of what you mean, does it sound like the audio is coming through the speaker on a phone? You might have to look into the audio settings on OBS, I'd assume they are on default but it may have selected settings to match your current Windows audio settings.
You'll have to check things like the audio bitrate for each channel in the Output tab (128 is usually the sweet spot for mp3, go too low and it will compress the audio & make it sound bad, too high & you'll just be adding unnecessary bytes to your footage making the file sizes larger) and check the sample rate in the audio tab (some games need it set to 48khz but that's more a sync related issue).
Other than that there's not much else I can think of that would cause this other than a bad/faulty sound card but if you're hearing the audio fine in game I doubt it'd be that, I just wouldn't rule it out.
