It depends on the type of cache. Simply put, you've got two types of caching:
- Client side caching.
- Server side caching.
Client side caching, which involves the saving of files onto the user's computer:
- CSS, images and JS files. These will be saved in the browser cache so that the user doesn't have to repeatedly download the same files from your server.
- HTML, which can be achieved through some of the various HTTP cache headers.
Server side caching is different, as the files are stored on your server (obviously). This includes:
- PHP bytecode. Whenever a PHP file is parsed, it is compiled into bytecode. By default, PHP will parse each file as though it is new. Because this is wasteful (PHP files don't change that much), many webmasters and server admins will cache the resulting bytecode and serve it up until the file in question changes (Look up OPcache). Implementing a bytecode cache will typically lower the amount of resources that your server is using, simply because your PHP files are not being parsed on every request.
- Object caching. This is when the results of database queries are stored in the server's memory, using a daemon such as Memcached or Redis, etc.
When optimizing the load times of your website, it is best to use both client-side caching and server-side caching.
Note: Plugins such as W3 total cache will use file-based caching if a bytecode / object cache isn't installed. This means that physical files will be stored on your server.