Does it matter for SERP, where my site is hosted and whether it's created with the use of a CMS, and if yes, does it matter, what particular CMS was used?
Yes they both matter.
Some CMS are more Search-Engine Optimizable than others, so if we think the amount of work as fixed (as in a fixed-budget project), you can reach higher rank and thus more visibility.
In practice this mean that a less SEO-proficient person can be used to keep a site updated, as he will have to handle less paramaters, or that the assigned person can spend less times on optimizing new content, concentrating on generating the same new content.
Moreover Google use geolocalization of the site and the user, in their search algorithm. So if you are selling apple pies in London, your site will rank higher to the users in the UK if your hosting company will have a server in the UK.
There are also the issues of uptime, as a site which is often down will rank lower, and the server IP reputation: if you share the same IP with spammers, e.g. in a shared hosting environment, you can loose all your rank in no time; but these are more trivial issues that good host should be able to take care of.
The CMS matters only as far as the HTML and sitemap it generates.
You could use lots of different CMS systems to generate exactly the same HTML, so in this sense you need to focus more on the templates or views than on the system used to generate those templates and views.
Also it is important that your CMS not be generating URLs with duplicate content. Some CMS, depending on how they are configured, will allow lots of different URLs to reach exactly the same page. You should solve this either through configuration or with setting up rewrites in your web server.
The host location does matter somewhat if you are targeting specific regions. For example, a site hosted in Germany is more likely to appear on google.de results than the same site hosted in Queens. Whether the inverse is true or not is hard to say. My experience with this has show that physical hosting locations is really not very relevant.
Another hosting issue in terms of SEO is server speed. The faster the better. That can be related to location, but again it is not very relevant.
In most cases it doesn't matter much at all. Unfortunately, it's the kind of thing that people fret needlessly over and "SEO gurus" sell snake oil for often to the "total newbies".
Geo-location matters, a little, at the national level but it doesn't make any difference where, geographically, in that country the server is located. Google's engineers know that co-location, virtual hosting and shared hosting are often used and these facilities are often not in the same city as the company who owns the site.
As for the hosting company, any reputable hosting company is the same. There is a lot of paranoia that circulates concerning having the same IP address as a "spammer". This is way overblown. Once again, Google's engineers know that shared hosting is popular and that there is an IP address shortage. Something that may have been applicable in 1998 is no longer applicable in 2011.
On the CMS, this can make a difference in some cases but almost any decent CMS system that's been optimized for SEO will do the trick. You'll sometimes find a "guru" trying to tell you that WordPress, Joomla, DotNetNuke or whatever they're trying to selling you is better for SEO. Don't believe them.