Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a WPMU 3.6.1 with Domain Mapping (0.5.4.3) with W3TC (0.9.3) and Google XML Sitemaps (4.0 BETA).

I have 4 different sitemaps.

  • sub-1.com/sitemap.xml
  • sub-2.com/sitemap.xml
  • sub-3.com/sitemap.xml
  • sub-4.com/sitemap.xml

on Google Webmaster Tools, I got 59 errors and 14 warnings.

Sitemap errors > Errors:

We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.

General HTTP error: 404 not found

Sitemap: sub-2.com/sitemap-pt-post-2011-02.xml

etc.

But when I click on my sitemap links they work fine.

Sitemap errors > Warnings:

URLs not accessible When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted.

Sitemap: sub-2.com/sitemap-misc.xml

HTTP Error: 404

URL: /sitemap.html

But when I click on my sitemap links they work fine.

Sitemap errors > Index errors:

URLs not accessible When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted.

HTTP Error: 404

URL: /sitemap-pt-post-2010-09.xml

But when I click on my sitemap links they work fine.

Web pages

  • 3,276 Submitted
  • 3,247 Indexed

What do I have to put on network admin > performance(w3tc) > page cache > cache preload > Sitemap URL ?

I have added "/sitemap.xml"

My robots.txt: http://pastebin.com/3K2U0mQa

My .htaccess: http://pastebin.com/efJJ6zwy

How can I make it work?

share|improve this question
add comment

1 Answer

I have two suggestions:

  1. Check your server logs. Make sure that everybody that is able to access these files gets the expected status. See if these pages returned 404 errors to Googlebot.
  2. Use the "Fetch as Google" feature in webmaster tools in the "Crawl" menue to try to access these resources.

Your problem is likely to be one of the following:

  • You have invalid links in your sitemap (but you say everything works when you click on them)
  • Your site is not up all the time and sometimes Google crawls it when it is serving 404 pages
  • Your site has some sort of rule that prevents Googlebot from crawling certain pages and serves 404 errors to it by user agent or IP address.
share|improve this answer
    
my robots.txt: pastebin.com/3K2U0mQa my .htaccess: pastebin.com/efJJ6zwy –  Alex Sep 20 '13 at 23:25
    
also i can't find the "download as Googlebot" –  Alex Sep 20 '13 at 23:35
    
Its now called "Fetch as Google", I updated the answer accordingly. –  Stephen Ostermiller Sep 20 '13 at 23:58
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.