It doesn't need to be royalty free as that does not equate to attribution, royalty free just means you don't pay royalties each time the video using the royalty free content is viewed. What it needs to be is under creative commons, most commonly the safest to use is Creative Commons Attribution 3.0/4.0 (CC By Attr 3.0 or 4.0) as the name suggests; work licensed under this usually requires you to attribute the music/song and artist used, in the video description or the video itself.
Some artists may even have added conditions on the attribution which you need to follow e.g. they may require a link to the original site or even a direct link to the song to be attributed.
But be careful as different CC 3.0/4.0 licenses each have different requirements on their own. So if an artist has their work licensed under certain CCs you'll need to check which 1 applies. Some CC licenses prevent you from modifying the song usually non-derivative (I think), some require you to list modifications you made which is actually standard for CC BY Attribution 3.0/4.0 but honestly this is a grey area as most artists don't mind as long as you don't modify it to a point it's unrecognisable i.e. cutting parts of the song is fine or extending it, some require your content to also be under the CC license in order to use their music usually the CC 3.0 ShareAlike and then some work may be licensed under mixed creative commons licenses so non-derritive sharealike may be used as well, there is also a non-commercial version of the CC By 3.0/4.0. So you need to check with the artists, it's also better to make sure they have a website to reference to in case of false content ID claims.