You will definitely need control of the domain name. You can get the current company to initiate a transfer to the account/registrar the new company uses, although you may feel more secure setting up your own account with a domain registrar so you have full control. (For example if the new company were to also become unresponsive down the line you could easily set up new hosting and point the domain there instead.)
For the hosting, it depends on what exactly is the current setup, and how you want the new setup to be. For example, if the current company has unique hosting for your site (e.g. a dedicated server) they may just need to hand over the account details and you can continue with that. You or the new dev company would need to add their own payment method and obviously change the username/password for the account.
Most likely they are hosting multiple sites on one server. In this case you'll need to obtain all the code and files that are on the server, plus a backup of the database. If you're using Wordpress this will be MySQL code in a
.sql file (or compressed/zipped version thereof). Then the code will need to be uploaded to a new server, database restored, and the domain name pointed to that server. The new company should be able to handle that for you.
As it's a Wordpress site that should be all you need. Direct server access or control panel access is probably not necessary, assuming the old company cooperates and hands over what you ask for. The only other thing I can think of that could get missed is a "cron job", which is a script or program set to run separately, independently of the website. Often they are used for nightly backups.
In summary, you'll need:
- Domain name
- Website code
- MySQL database backup
- Cron jobs or any other vital software running on the server