I've found the quickest way to do it, so I'm sharing it here.
First, upload the custom HTML page somewhere in your server or any other server where it can be downloaded from (it doesn't matter where the file is uploaded as long as it's accessible from your Plesk server using wget).
Then, connect to the server with the Plesk installation as root via SSH and enter the following commands to replace the custom page for all future Plesk sites:
mv maintenance.html maintenance.old
By doing this, the new custom page will be applied to all new domains created in the Plesk panel from now on, but we still have to modify all existing domains as well. To have the new custom page copied to all existing domains in the server you will have to run some extra commands.
Please note that this will overwrite the error pages in ALL of your domains. If all of your domains have the same exact error messages and you think you don't need a backup, you can skip the second line. Otherwise, paste everything into the command line (be careful with the long lines and copy the whole thing).
If you want to run all lines, including the second, which creates backup directories, you can copy and paste all four lines in one go. The backup directories will be called "error_docs_backup" and will be created inside each of the domain directories.
plesk bin subscription -l > subscriptions.txt
while read i; do mkdir -p /var/www/vhosts/$i/error_docs_backup/; chown $(stat -c %U /var/www/vhosts/$i/):psacln /var/www/vhosts/$i/error_docs_backup; cp -a /var/www/vhosts/$i/error_docs/*.html /var/www/vhosts/$i/error_docs_backup/; done < subscriptions.txt
while read i; do \cp /var/www/vhosts/.skel/0/error_docs/*.html /var/www/vhosts/$i/error_docs; chown -R $(stat -c %U /var/www/vhosts/$i/):psacln /var/www/vhosts/$i/error_docs; done < subscriptions.txt
rm -f subscriptions.txt
That's it! No server reboot is needed. The new custom page (or pages, if you modified more than one) will be available in all domains.