0

My website is composed of thousands of products, dynamically driven.

Ex.: mysite.com/brand/model/

When looking at my Webmaster Tools, under the Smartphone Crawl Errors, I see a lot of "Invented URLs" that are not part of the architecture of my website or my sitemap, in any way..

Ex1: mysite.com/mobile/brand/model/
Ex2: mysite.com/m/brand/model/

Anybody knows why Google is crawling these URLs out of the blue? What is the approach recommended to deal with the situation? Is it harmful in the SEO point of view?

  • did you check your sitemap? maybe it's been generated before you set your final url structure. – lucgenti Jun 4 '15 at 19:02
  • I verified and my sitemap doesn't contain any unexpected mobile structure – Jivago Jun 4 '15 at 19:17
  • You could try adding the directories, such as /m or /mobile to your robots.txt as Disallow... – DeveloperACE Jun 4 '15 at 23:23
1

Double check your various forms of links JIC. If you have done your work right, then there is nothing for you to do.

There are a ton of scrapers and sites that will make bad links and perhaps that is what is happening. It is not uncommon that site automation of this type to go wrong creating links that Google will see as new and follow.

Just make sure your work is right and be as thorough as you can. It is very easy to miss something- we have all done it- it can be embarrassing, but important to find. It might be worth while to use both a desktop and mobile device and walk around your site. Just turn off ads for a while as not to get into trouble.

If it helps, you can use ScreamingFrog which is free up to 500 pages or SEO PowerSuite which may do the same thing on a larger scale in trial mode. In that, you can use these tools to find broken links which is something you should do from time to time anyway. These tools will find mistakes that are easily overlooked. Again we all do it. I found some mistakes I thought I took care of a long time ago on my last audit.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.