4

I have a website which hosts a PDF document (math paper). Main page of the website provides link to the document, which is

http://example.com/download.php?file=Document.pdf  

The purpose of download.php script is to log IP addresses that download the document.

Now, the document can also be viewed/downloaded by following the link

http://example.com/Document.pdf

Does this setup has any disadvantages from the SEO perspective (duplicate content)? And if yes, how can I make it better?

  • 1
    Are you linking to both these URLs? Which one is the canonical? Are these URLs completely separate ways to access the same document? Or do you always rewrite requests through download.php (which I assume you would do)? – MrWhite Nov 20 '16 at 22:27
  • Yes, both URLs are used for linking. The canonical link has not been defined -- I don't know where to do define canonical link for PDF. I do not rewrite requests through download.php – Leo Nov 20 '16 at 22:41
5

Yes, this is duplicate content. The same content is accessible from two different URLs and there is no canonicalisation.

Basically, this means that the search engines will pick one or the other to return in the SERPs. Ranking is essentially split between the two URLs.

both URLs are used for linking.

You need to decide which is the canonical/preferred URL and link only to that one URL.

For simplicity, we'll consider just the two URLs you've listed. The preferred URL would seem to be the one that goes via your download script (ie. download.php), otherwise you aren't going to be tracking the IPs of users downloading the file.

To resolve any already indexed URLs, you can externally redirect the direct link to your script. Assuming Apache, then you can do something like the following in your root .htaccess file:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^([^/]+\.pdf)$ /download.php?file=$1 [R=302,L]

This will redirect a request for /Document.pdf (only if it exists as a physical file on the file system) to /download.php?file=Document.pdf.

$1 is a backreference to the first captured group in the RewriteRule pattern (ie. ([^/]+\.pdf)).

Change the 302 (temporary) redirect to a 301 (permanent) when you are sure it's working OK. 301s are cached by the browser so can make testing problematic.


A more "user friendly" URL (UPDATED)

You could take it one step further and create a more "user friendly" URL like /download/Document.pdf. This would then become the canonical URL - the URL that you link to.

In this case, since you have a file whose basename is also "download" (ie. download.php vs /download), you need to make sure that MultiViews is disabled. Otherwise mod_negotiation is likely to make an internal subrequest for download.php (depending on the request) before we've rewritten the URL. So, at the top of .htaccess:

Options -MultiViews

Any direct requests for /Document.pdf or /download.php?file=Document.pdf should be externally redirected to the canonical URL. For example:

RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^([^/]+\.pdf)$ /download/$1 [R=301,L]

RewriteCond %{THE_REQUEST} GET\ /download\.php\ HTTP
RewriteCond %{QUERY_STRING} ^file=(.+\.pdf)$
RewriteRule ^download\.php$ /download/%1 [R=301,L]

%1 (as opposed $1, mentioned above) is a backreference to the last matched RewriteCond CondPattern (ie. (.+\.pdf)).

The additional RewriteCond (condition) that checks against THE_REQUEST is necessary in order to prevent a redirect loop. (THE_REQUEST contains the original request header and does not change when the URL is rewritten.)

/download/Document.pdf would then be internally rewritten to the "real" URL. ie. /download.php?file=Document.pdf. An internal rewrite, as it suggests is internal to the server. There is no external HTTP request. The URL in the address bar does not change. It is completely hidden from the end user.

RewriteRule ^download/([^/]+\.pdf)$ download.php?file=$1 [L]

Note that there is no R (redirect) flag on this directive that would otherwise trigger an external redirect.

Ideally, you would make the regex as restrictive as possible. For example, in the above regex, .+ matches any characters (1 or more). However, if your filenames consist of only upper and lowercase letters then it would be preferable to change the regex to match only letters. eg. [a-zA-Z]+.

  • Great answer, thank you. Two follow-up questions. (1) When I use your RewriteRule approach, I get www.example.com/download.php?file=Document.pdf instead of canonical example.com/download.php?file=Document.pdf. So another rule in .htaccess that removes www part is not working when I add your RewriteRule. What do I do about it? (2) In the second part of your answer, what do you mean by "internally rewritten" and how can I implement that? – Leo Nov 21 '16 at 3:18
  • (1) There is no obvious reason why that should be happening. The canonical www to non-www redirect should come first. Presumably you are already linking to the non-www version. You can add your current .htaccess file to your question to check. (2) I've updated my answer with code examples. NB: There was a typo in "RewriteEngine", but I guess you caught that! – MrWhite Nov 21 '16 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.