4

Is there a programmatic way to easily deal with the issue of search engines considering pages where the page works with and without the trailing slash duplicates like via a robots.txt or Nginx rewrite rule?

Examples of "duplicate content" according to search engines:

https://www.jumpingthruhoops.com/filepath
https://www.jumpingthruhoops.com/filepath/

Maybe something like this for the rewrite rule, except I'm not sure what to use as far as where to redirect to (hence the question mark below)

rewrite ^/$ ? permanent;

We use Magento 1.13, where some content already has canonical links generated automatically. So if either the robots or rewrite rules won't work, I'll probably have to try to find an extension that performs that task on pages that don't already have canonical links. Or if anyone has a better approach, I'd be very interested in hearing about it.

Improve this question

1 Answer 1

Reset to default
6 
rewrite ^/$ ? permanent;

This would only match the document root (a valid request) - this isn't a URL that ends with a slash, but a URL that starts with a slash.

Try something like the following instead, to match any request URL that ends with a slash:

rewrite ^(.+)/$ $1 redirect;

$1 is a backreference to the original URL-path, less the trailing slash.

Change the redirect flag to permanent once you have tested that it works OK - to avoid any caching issues.

Improve this answer
4
  • Thanks for that code example. I'll test it out on our staging site and see how it works out. Aside from the question of whether or not it can be done, do you think there's any negative aspect involved in resolving the problem like this?
    – Realto619
    May 25, 2018 at 16:41
  • Can someone with the proper access mark this as the accepted answer? I don't have the ability to do that yet since this was my first question posed on this site. Thanks again, @DocRoot !
    – Realto619
    May 25, 2018 at 18:06
  • "any negative aspect" - only if you were incorrectly linking to the URL with a slash elsewhere on your site, then you should obviously fix your URLs. But otherwise, I think this is a good solution to resolve a potential duplicate content issue.
    – DocRoot
    May 26, 2018 at 7:51
  • Only you (the account that asked the question) can mark the answer as "accepted". (Not even the moderators.) AFAIK there is just a 15 minute delay enforced from asking a question to being able to accept an answer (regardless of it being your first question or a low rep user) - so you should certainly be able to accept the answer now. "To mark an answer as accepted, click on the check mark beside the answer to toggle it from hollow to green" (reference). Thanks, much appreciated, glad it worked for you. :)
    – DocRoot
    May 26, 2018 at 8:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.