Preventing Google from indexing PDF Files

Question

I sell PDF patterns and when someone types my website and file type PDF in Google, all of my patterns for sale are listed to download for free. I use WordPress and Jigoshop. I need to find a way to block the PDF files from coming up in a Google search but still make them available to download when a customer purchases them. Any ideas?

Answer 1

You can block Google indexing certain file extensions by using the robots.txt file on your server.

SOURCE: Block or Remove Pages robots.txt

To block files of a specific file type (for example, .gif), use the following:

User-agent: Googlebot Disallow: /*.gif$

The above will block Google from indexing gif files and obviously you just adjust the code to: User-agent: Googlebot Disallow: /*.pdf$ if you want to block all search engines rather than just Google you can use: Disallow: /*.pdf$.

This can cause the issue of not indexing PDF files that you want indexing and therefor if you do not want to block based on file extension would be to use:

Disallow: /path/to/pdf/thefilename.pdf

Answer 2

You need to outright block your files from anyone who hasn't paid, not just the Google spider! To prevent direct download links, you can either store the files outside your site root or use an .htaccess rule to deny all access to the PDF folder(s).

Once a user has paid, a PHP script can reach inside the inaccessible PDF folder and get the PDF to download.

Answer 3

You could configure your server to use X-Robots-Tag=noindex HTTP header for PDF files.

If you are using apache with mod_headers, you could put this in your httpd.conf or .htaccess file:

<Files ~ "\.pdf$">
    Header append X-Robots-Tag "noindex"
</Files>