1

I had submitted my site's URL to Google Webmaster Tools. I am using OpenCart. I also have robots.txt file which is accessible to Googlebot.

Now issue I am facing: Googlebot is blocked from accessing 450 URLs by robots.txt but these URLs are present in the sitemap which I submitted. Now I am seeing almost 150 warnings on the "Google Index" page of Google Webmasters Tools.

Will this effect the website indexing? I do not know how to control the sitemap. am using the default sitemap for OpenCart.

| improve this question | |
  • If your robots.txt is blocking these URL's them presumably you do not want them crawlable/indexable in Google? If that's the case, there isn't a problem. And you do have control over the default sitemap in OpenCart, the platform is open source so you can modify anything you like. – zigojacko May 7 '14 at 7:36
  • @zigojacko: You are right. I do not want them to be indexed. Can you provide any pointer how can i control sitemap for opencart? so that i could research for it. – DavKan May 7 '14 at 7:49
  • Sure, this place will be a good starter point that shows you where you can change the URL's included in OC sitemap. Difficult to elaborate without knowing what URL's you are wanting to remove from the sitemap. – zigojacko May 7 '14 at 8:33
  • @zigojacko: I will be removing the URL based on my robots.txt to minimise the warnings in google webmasters. Thanks for the help – DavKan May 7 '14 at 8:49
  • I got this from Wikipedia: 405 Method Not Allowed - A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource. – closetnoc May 7 '14 at 16:10
1

This is not an issue if you don't want Google to index these 450 pages. But you may get these warnings forever, or until you get control on the sitemap.

| improve this answer | |
  • At least one SEO claims that having a page in your sitemap, but blocked by robots.txt sends Google mixed messages, and may reduce page rank. The same author recommended pruning your sitemap to eliminate 'utility pages' E.g. pages that are placeholders, pages that serve as resources for other pages. So don't index Downloads in your sitemap. Google will find it from the pages that reference a particular download. – Sherwood Botsford Nov 24 at 21:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.