Hot answers tagged

29 votes
Accepted

Google Search Console is making up URLs which don't exist in my Sitemap and then complains that these pages have error

You have a misconception of what a sitemap is. The sitemap is used to audit the site crawl by the search engine bot. The sitemap and crawling a site are two different and independent things. Google ...
user avatar
  • 32.5k
19 votes
Accepted

Prevent XML sitemaps from showing up in Google search results

Google does index XML sitemaps (like any XML file). If Google is aware of a URL and it returns a valid response then it's going to pass Google's inclusion rules and could get indexed. Personally, I ...
user avatar
  • 40.9k
18 votes

Google Search Console is making up URLs which don't exist in my Sitemap and then complains that these pages have error

Closetnoc is correct about sitemaps. Don't expect them to limit what URLs Google will crawl and index. In fact sitemaps have little to no influence over SEO. See The Sitemap Paradox Google won't ...
user avatar
14 votes
Accepted

Can a sitemap index contain other sitemap indexes?

I'm not sure if this changed over the years since this was asked; while in theory you can (as the first answer states), in practice Google at least will give you an error (as seen in their Webmaster ...
user avatar
  • 256
14 votes
Accepted

Google is changing my urls in search results - Joomla3 bug

In your header you have a canonical link (on line 11, just under <title>). It looks like this on your page: <link href="http://escene.ir/component/products/?task=view.12" rel="canonical" /&...
user avatar
  • 291
14 votes

Prevent XML sitemaps from showing up in Google search results

MrWhite's answer about using X-Robots-Tag appears to be the correct way to do this. Here is code that can be used in .htaccess or Apache configuration files to do so. (Reference: WebmasterWorld - ...
user avatar
12 votes
Accepted

Is a different sitemap per language OK? How do I tell Google about them?

You can have multiple sitemaps per website, and this is a great example of when that makes sense. You should make sure you have a Sitemap Index listing each of your sitemaps. It will probably look ...
user avatar
  • 5,794
12 votes
Accepted

Can the files robots.txt and sitemap.xml be dynamic through an .htaccess redirect?

You can make any file dynamic. The best way to do so is not through redirects, but through rewrite rules. RewriteRule ^robots\.txt$ /robots.php [L] That way, you power it with a dynamic script, ...
user avatar
12 votes
Accepted

sitemap.xml not rendered as XML document in browser

This suggests your HTTP server is delivering the sitemap with a text/html content type rather than an application/xml one. You need to configure your server correctly.
user avatar
  • 1,581
11 votes

Sitemaps - one per subdomain or one for the base domain?

Generally, you have to use a separate sitemap for each host (i.e., different protocol, domain, or subdomain): From the FAQ "Where do I place my Sitemap?": All URLs listed in the Sitemap ...
user avatar
  • 21.4k
9 votes
Accepted

Sitemaps - one per subdomain or one for the base domain?

As per Matt Cutts blog post, he said: A subdomain can be useful to separate out content that is completely different. Google uses subdomains for distinct products such news.google.com or maps....
user avatar
8 votes

Is it necessary to submit sitemap to www and non www?

No, you don't need to submit a sitemap for the version (here with www) you don't want to use. Remember submitting a sitemap to your Google Webmaster Tools account helps indexing of the site. ...
user avatar
  • 10k
8 votes

Is it possible to make an XML sitemap pretty enough to show to users?

Yes. Everything you ask for is possible. And here's an example XML sitemap file generated by the Drupal XML sitemap module with a little bit of configuration. Everything is done for you out of the box....
user avatar
7 votes

Is submitting a sitemap an outdated technique for SEO?

We have another question here that ask why items in the sitemap are not ranked better: The Sitemap Paradox. Google's John Mueller has this to say about common SEO misconceptions regarding sitemaps: ...
user avatar
7 votes

Which is the best choice for SEO sitemap for multilingual website?

You're better off creating a multilingual sitemap, just to avoid any source of confusion. The format you have shown is correct. In fact, you could even drop the hreflang declarations in your page ...
user avatar
  • 676
7 votes

Are alternate links included in the 50,000 URL limit for sitemaps?

As closetnoc suggests in comments, the 50,000 URL limit for sitemaps refers to the number of URLs in the sitemap file itself. ie. the number of <loc> elements. This is an individual sitemap ...
user avatar
  • 40.9k
7 votes

HTTP to HTTPS: Wait for new sitemap to be indexed?

Google will never index HTTPS while the canonicals point to HTTP. I switched my largest site over to HTTPS using the following protocol: Enabled HTTPS for the site without switching canonicals for ...
user avatar
7 votes
Accepted

Ampersand (&) in actual URL and sitemap

if this difference of ampersand in URL and sitemap will cause any issue. tl;dr No issue, because the URLs are the same. Since in sitemap & has to be escaped I replaced & with &amp; ... ...
user avatar
  • 40.9k
7 votes
Accepted

How to tell bots to forget a website and reindex it from scratch

That isn't possible. You need to map your old URLs to the new with redirects for SEO and user experience. Google never forgets about old URLs, even after a decade. When you migrate to a new CMS, ...
user avatar
7 votes

Why did Yoast put a no-index tag in my XML sitemap?

Because it is an XML file, the sitemap doesn't support meta tags. Instead it is technically an HTTP header: X-Robots-Tag: noindex You don't want to remove it that header. It prevents the XML ...
user avatar
6 votes
Accepted

How long does Bingbot typically take to process a sitemap submission?

This post is 4 years old, hope the status in not Pending anymore. But this helped me. After 14 days of pending status, I found this link, to PING bing with your sitemap. It's was a kind of awake call ...
user avatar
6 votes

How to submit sitemap to Baidu?

http://zhanzhang.baidu.com/sitemap/index is the (sparse) documentation about sitemaps for Baidu. It only describes that it may help to submit your sitemap, and it shows a screenshot of their webmaster ...
user avatar
  • 21.4k
6 votes

How can I fix Google indexing URLs that are not part of my sitemap?

Most likely some part of your web site generated links like that, and that is how Google started to crawl the URLs. You should check the links in your web pages to see where these incorrect URLs are, ...
user avatar
6 votes

How important is it to include <lastmod> in a sitemap?

Most crawlers tend to ignore the lastmod tag as many webmasters do a poor job of keeping it up-to-date and forget to update it when they update content on the site. A sitemap is still important to ...
user avatar
6 votes
Accepted

Yoast SEO plugin creates XML Sitemap with http. Is this correct for a https site?

You need to disable and reenable the sitemap, going to SEO > Sitemap XML. It probably will solve your problem (Yoast bug).
user avatar
5 votes

Does Google penalize daily updated <lastmod> tags in sitemaps if the data is not daily updated?

I've never liked the idea of updating <lastmod> every day as itt's not just wrong, it's misleading search engines. In a post over on SO, Google's Gary Illyes wrote: The lastmod tag is ...
user avatar
  • 5,794
5 votes
Accepted

Should paginated sites be included in sitemap?

You should include to your sitemap all the URLs you want to see in a search engine's index.
user avatar
  • 270
5 votes
Accepted

Google sitemap HrefLang tag without the main site url

Ok. Found the answer at: Help Google serve the correct language to your visitors We need to have a url tag for each of the url and specify the others as alternate urls.
user avatar
  • 211
5 votes

How can I fix Google indexing URLs that are not part of my sitemap?

Sitemaps serve to include, not limit the content Google indexes. If you want to exclude some files, use a robots.txt file as mentioned, or setup redirects. The reason this URL is included is likely ...
user avatar
  • 151
5 votes
Accepted

Yahoo sitemap ping URL?

Here is Yahoo's documentation for their sitemap ping. At the top there is a prominent notification that Yahoo webmaster tools have moved to Bing webmaster tools. That is why the ping times out. ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible