I currently own four domains myself. If you want to get a domain name, Google Domains is fairly cheap, and they have a great interface. They charge me $12 a year for the domain registration. In the past I've used Yahoo, Network Solutions, and GoDaddy. I've been the happiest with Google Domains.
As for hosting, I actually host out of my basement. I have decent upload bandwidth with my cable Internet. It's just a normal residential internet account. This approach will require some knowledge on how to run your own server, firewall, and router. This gives you complete control with the kind of software you want. You chose your own own *scheduled* maintenance downtime. Plus you don't have a hosting services that's over-committing hardware or screwing you around on an ever-changing, arbitrary bandwidth limit. If you do have some old computer equipment laying around you can put it to use as a simple webserver. You don't need a machine with a lot muscle to host a simple website. You do have to pay for your electricity. My servers pull about 380-450 watts (I have a special meter for them). Since I work from home, my computers are going to be on anyway for most of the day so there isn't much added cost for me. If you do have the know how this is probably the cheapest and the most bang for your buck than any other service.
If you don't want to deal with your own hardware, but you want control of the servers, there is always Amazon AWS. If you are going to have a server up for 24/7, I'd suggest getting a heavy utilization reserve instance. There is an upfront cost, but it is the cheapest on a per hour basis. Almost 40% cheaper than on-demand prices. They about every capability and pricing that you can think of.
Doing this stuff on your own is a good way to learn. If you don't have time for that, you might want to look at some other services.
