3

In your experience, have you ever had problems with duplicate content being caused by the following URLs:

  1. www.mysite.net
  2. www.mysite.net/
  3. www.mysite.net/index.html

Do you think that this could result in a duplicate content issue if my canonical link is constantly pointing to only this: www.mysite.net

Improve this question
1
  • Note: The domain used by the OP was mysite.net as referenced in the first answer. Subsequent edits to example.com would break that reference, however example.com can be considered the same.
    – dan ♦
    May 19, 2014 at 23:34

2 Answers 2

Reset to default
3

No, not if their content is identical (or very nearly so), and they all have a canonical link element referencing www.mysite.net.

However, that sort of issue happening site wide can be considered a crawl efficiency issue and the canonical link element doesn't really solve that. In your example, a search bot would crawl three pages when it really only needed to crawl one. Scale that up, and add other duplication issues, and it gets to be a problem.

So for that sort of issue (we can include things like with/without the www subdomain, with/without HTTPS, etc.), it's usually better to use URL rewrites to 301 redirect back to the canonical form.

The canonical link element is the weapon of choice when there's a lot of variation in the duplications you're managing, or you don't know in advance what they will be. Think of a shopping site where tens of product category refinement parameters can be added to a base URL in any order, for example. That doesn't mean don't add it unless you have that sort of problem, just don't rely on it to solve problems there are better solutions for.

Improve this answer
5
  • Thanks for you help, all HTTPS requests are 301nd to HTTP, all mysite.net requests are also 301nd to www subdomain sitewide, I have now also managed to redirect "/" in root to www.mysite.net, but now have to work with index.html in root and directories to one consistent version.
    – AlexB
    May 17, 2014 at 9:29
  • This may help you stackoverflow.com/questions/4646241/…
    – GDVS
    May 17, 2014 at 9:32
  • To add more, I have also got redirect on every directory with lower/upper-case, so if my directory is /MyDirectory/ and request is /mydirectory/ the index.php will 301 it to the right case and corrects URL in the address bar, so hopefully I am on the right track.
    – AlexB
    May 17, 2014 at 9:33
  • THe rule in .htaccess is working fine with static version of the url, but it did not work for me, since all of the content is written on index.php in my case and it will not work along with some other rules in my .htaccess.
    – AlexB
    May 17, 2014 at 9:36
  • 1
    Normalising case is another good one to do, yes. If you've no option, you could normalise to /index.php rather than /: it doesn't really matter which, as long as only one variant resolves a page.
    – GDVS
    May 17, 2014 at 9:49
1

From a technical standpoint, only two of those are actually different URLs. The following are always the same:

  • http://www.example.com
  • http://www.example.com/

The reason is that it is impossible for your site to serve different content at those two URLs because of the way that the HTTP protocol is defined. When a request to your server is made, the first line looks like:

GET / HTTP/1.1

Where the slash is the the part after your domain name. The HTTP spec does not allow this to be empty, so a browser must send a slash when fetching http://www.example.com.

This is in contrast with

  • http://www.example.com/path
  • http://www.example.com/path/

which could have different content because they look different when your server receives them.

Because clients always send a slash, I prefer http://www.example.com/ as the canonical URL because it best matches what actually happens under the hood.

Your canonical element should prevent search engines from indexing duplicate content, or penalizing for it.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.