Would you be able to post part of your httpd.conf (or included configuration if it's split into multiple files)?
Inside of it should be the "per user" configuration section, which defines which directory the servers looks for in a user's home.
For example, here is the default I normally have on my Apache servers:
# Turn on userdirs
<IfModule mod_userdir.c>
UserDir public_html
<Directory /home/*/public_html>
Order allow,deny
Allow from all
</Directory>
</IfModule>
In this case as long as a user has a public_html directory, with permissions of 0711 (or at least 0755) - the read and execute bits of public_html must be set, along with at least the execute of their home directory - Apache will serve files from it with the proper permissions.
One other question, from your URL above of "server/~user/public_html/" - did you upload the files under the directory public_html/public_html? Otherwise, you should just drop the "public_html" part at the end, as Apache will automatically look under there.
That would make the URL http://server/~user/
and should work, unless your server has a non-standard configuration.
If none of the above works, check to see if you have directory indexes enabled, and remove any index files from your public_html; that should give you a list of all the files, to verify the structure!
public_html
folder in the URL. This would imply that the document root of your site was the parent directory that contained thepublic_html
folder, not thepublic_html
folder itself - this itself would probably be a misconfiguration.