When I'm transferring big files between two servers with ncftpget I'm experiencing very high load on the receiving server. What can cause this?
The most common issue here is a lack of RAM on your server. Having used a range of FTP servers in the past most FTP servers store the file in memory during the transfer and once the whole file has been received or sent the file is flushed from memory to a file on the receiving server, and just flushed from memory on the sending server. NCFTP has a general rule of thumb for the maximum number of connections to allow on the server based on the amount of memory of
(M-32) * 2 but this is based on the server not doing anything else other than FTP. This will never be the case as the operating system uses RAM, any other server daemons that are running use RAM as well (Apache, mail server, etc). If you are seeing this problem happen it it is a fairly good sign that you need to increase the amount of RAM on your server.