We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [xml-sitemap]

XML Sitemaps are XML files created to make it easier for Search Engine crawlers and other bots to efficiently and fully crawl a website.

Filter by
Sorted by
Tagged with
251
votes
19answers
37k views

The Sitemap Paradox

We use a sitemap on Stack Overflow, but I have mixed feelings about it. Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
73
votes
5answers
30k views

Are there any clear indicators that my sitemap file is beneficial?

I have recently created a sitemap.xml file and uploaded it to my Google Webmasters Tools account. Google didn't report any issues or errors with the uploaded sitemap of my site. Now my question is: ...
23
votes
4answers
10k views

Prevent XML sitemaps from showing up in Google search results

How do I prevent my XML sitemap files from showing up in Google search results like this result of a site: search query: I don't understand why Google would choose to show sitemap files in search ...
13
votes
2answers
1k views

Should I Include RSS Feed URLs in an XML Sitemap?

Simple question - Should I Include the RSS Feed URLs in my site's XML Sitemap? I am NOT asking whether I should USE an RSS Feed AS an XML sitemap, but rather, should I include the URLs to the ...
11
votes
1answer
470 views

Proper sitemap.xml setup

I have a dynamic site which has many (well, less than 50) users. Each user is allowed to create as many pages as they want. I know that there is a limit to how many pages you can be listed in ...
10
votes
7answers
7k views

How to hide my XML Sitemap from competitors but not from search engines

I want to hide my sitemap XML file from all but allow access from search engines. What is the way to do it? I want to hide the depth of site's content from competitors.
10
votes
2answers
2k views

Is incorrect to have the HTTPS version of the sitemaps.org URL in the xmlns sitemap schema?

I have the schema with this: <sitemapindex xmlns="https://www.sitemaps.org/schemas/sitemap/0.9"> Is it correct or should be: <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0....
9
votes
1answer
618 views

Internal links to pages vs. sitemap.xml links to pages

I have a page that is not linked to from anywhere, but it is listed in sitemap.xml. It will be crawled and display in the SERP. Would it be preferable, from a SEO perspective, to have the page ...
8
votes
4answers
3k views

Robots.txt vs Sitemap — Who wins in a Conflict

If I block off the directory /foo in robots.txt, but my xml sitemap contains URLs with /foo, will the URLs in the sitemap get picked up by Google and other search engines? In other words, does the ...
8
votes
1answer
936 views

Should I ping to Google sitemap-index.xml or only the sitemap file that has been modified?

I have a sitemap-index.xml with 7 files. After updating one of the 7 with new content should I ping Google with the index file or with the updated file? In my index I also use lastmod so Google can ...
7
votes
2answers
1k views

Is it possible to make an XML sitemap pretty enough to show to users?

I wouldn't mind showing my XML sitemap to users if there were a way to make it human usable as opposed to just machine readable. Is it possible to: Choose the colors and layout Make the locations ...
7
votes
2answers
1k views

Are there specific advantages or disadvantages of an XML sitemap over a TXT based sitemap?

Are there any specific advantages or disadvantages in generating an XML based sitemap in place of a simple txt based sitemap (list of URLs)? I realize that in the XML format I can set priority and ...
7
votes
3answers
4k views

Can a sitemap index file reference sitemap files under different domains?

My company owns several domain names. We also host websites for our users at their custom domains. Our domains: company.com product1.com product2.com Users's domains: user1.com user2.com ...
6
votes
3answers
5k views

Sitemap: Hierarchical layout possible?

Is it possible to design my sitemap hierarchically? My site is laid out hierarchically into World=>Continent=>Country=>State=>City=>Articles Can I design my sitemap xml like this?...
6
votes
2answers
367 views

Google sitemap priorities, am I doing the right thing?

I've built a website which has maritime weather and tide data for UK locations, and each location has 7 day forecast (where data is available). My URL structures are like this: http://www.example.com/...
6
votes
1answer
337 views

In Google Webmaster Tools we have 3 sitemaps attributed to 1 domain

I have a website that has been on the internet for almost 10 years created in "Microsoft FrontPage" with over 900 pages. Currently in Google Webmaster tools it shows up as 2 domains and 3 sitemaps: ...
6
votes
1answer
561 views

If I use the same image on multiple pages, should it be on my sitemap multiple times?

Following Google's guide to Image sitemaps <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:image="http://www.google.com/schemas/...
6
votes
2answers
390 views

Why is Google reporting that hreflang URLs in my sitemap do not have “return tags”?

Google Search Console (Webmaster Tools) keeps listing the following errors for my sitemap: Sitemap provided URLs and alternate URLs in 'de' that do not have return tags. +---+-----------------+----...
5
votes
6answers
4k views

How to tell Google that I have changed my website URLs?

I have done major updates to my website, and renamed all my URLs. How can I tell Google that I have renamed these URLs and let Google refresh its index? My website URL: http://www.pndmasr.com/ My ...
5
votes
5answers
3k views

Is there any benefit to having a link to your website's XML sitemap?

In Google's WebmasterTools, I submitted my site's XML sitemap, which is accessible through www.mysite.com/sitemap. However, no where on the site do I have an actual link that points to this. Is ...
5
votes
3answers
18k views

Sitemap submission : Do sitemaps need to be resubmitted when they are updated?

In short: Should I resubmit my sitemap through Google Webmasters Tools every time I replace a sitemap.xml with an existing sitemap.xml (i.e., www.domain.com/sitemap.xml)? Or does it check it ...
5
votes
3answers
2k views

Should I include forum posts in the sitemap?

I am launching a medical discussion forum for sharing study materials and discussing other topics. The forum will probably grow fast and reach at least a few hundred topics. Should every discussion ...
5
votes
2answers
2k views

How often are sitemap.xml checked for updates by crawlers?

How often do crawlers check for updates in submitted sitemap.xml? Every hour, day, week, month, more?
5
votes
4answers
4k views

Do you really need robots.txt and sitemap.xml if I want bot to follow every link? [duplicate]

I wonder if I really need robots.txt/*sitemap.xml* files if I want bots to go through every link on my website without any restrictions? And if I do, what should I write in there?
5
votes
3answers
8k views

Sitemap to display hash tag links or not?

I am just creating sitemap for my site and wanted to find out if I should include # links for SEO? For Example: www.mydomain.com/#comment1 www.mydomain.com/#comment2 Does this help with SEO or ...
5
votes
1answer
416 views

Are alternate links included in the 50,000 URL limit for sitemaps?

I see the number 50,000 referenced by Google and Schema.org as the limit for the number of URLs that a sitemap can contain, however, it is unclear to me if that means <loc> elements specifically,...
5
votes
2answers
573 views

sitemap.xml for huge online store

I'm running online store with 30,000+ products (having unique urls) which are updated every day. I want google to index my products faster. Now crawler index ~500 pages/day (reports from google ...
5
votes
4answers
9k views

What URLs should I put into XML Sitemap?

We will launch a project with over 1.000.000 products and those are in approx 150 categories. Later on you will find reviews and technical detail pages to those products. So there will be more than 2....
5
votes
1answer
1k views

Sitemap file getting double compressed

I have a script that generates my XML sitemap and writes it to the file sitemap.xml.gz - i.e. an XML file, compressed with gzip. This file is definitely written correctly as when I download it via FTP ...
5
votes
3answers
752 views

Does an XML sitemap need to have a .xml extension?

Im using a CMS that generates a sitemap automatically but without ".xml".IS this ok? Can we put this is our robots.txt: Sitemap: https://www.example.com/sitemap or must it be: Sitemap: https://www....
5
votes
3answers
5k views

Google SEO - remove site map from search result

I am an amateur to SEO. I have a new site recently and submitted the site map to GWT. But after several weeks. I found when I search my domain in google. The site map also appears to be one of the ...
4
votes
1answer
2k views

Why important/big sites don't include a sitemap? [closed]

I've been looking at most important sites and none of them have a sitemap.xml defined. My list includes: Amazon Ebay Yahoo Microsoft ... and others But none of them include a sitemap.xml file. Why ...
4
votes
2answers
3k views

Can a sitemap index file list sitemaps in a different directory?

If I have a sitemap index, can the sitemaps listed in the index be located in a subdirectory, but have links that isn't specific to the subdirectory? I'm trying to avoid having multiple sitemaps in ...
4
votes
3answers
376 views

Delay before getting URLs into Google's in web index?

4 days ago, I have submitted a sitemap.xml into Google's WebMaster Tool for a new website. It contains 3 entries. I have tested the site map and Google found not errors. In about how much time can I ...
4
votes
2answers
15k views

Why does my sitemap have a “.gz” extension, and how can I edit it?

An XML sitemap generator plugin for WordPress puts the following strings in my robots.txt file: User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ But other WordPress blogs have lots of ...
4
votes
1answer
4k views

Is it okay to have multiple sitemaps?

Recently, I moved a site from http to https and the sitemaps got updated. I submitted the sitemaps to Google. There were around 9 sitemaps for the site, each of them containing pages/images the site. ...
4
votes
1answer
66 views

Can a very large not-so-relevant sitemap hurt my site to Google’s eyes?

I have a Wiki site (running MediaWiki) which I am trying to optimize in terms of page speed and SEO. One of the many things in my to-do list is submit a sitemap on Google Webmaster tools, now here is ...
4
votes
2answers
357 views

Is it OK to “pause” an advert in terms of SEO?

I have a website along the lines of Craigslist, where users can post classified ads. Some users have requested a feature where they're able to pause/resume their ads. From an seo point of view, the ...
4
votes
1answer
140 views

Do I need to update the sitemap.xml file after removing some URLs?

Do I need to update the sitemap.xml file after removing some URLs from the website and merge three different URLs to a single one. For example, three different URLs: www.example.com/one.html (main ...
4
votes
1answer
188 views

Can XML Sitemap include itself as one of the URLs?

A little while ago I discovered an XML sitemap where the sitemap itself was included. <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> ...
4
votes
1answer
137 views

sitemap.xml : shall I put all my URLs including the ones that have “empty result” for a dataset?

Here's my problem: on my website, you can click on all the regions, and the new web page is the result of the search of the region. Thus the result may be empty. Shall I include all those "empty ...
4
votes
1answer
50 views

Do the order of “Disallow” and “Sitemap” lines in robots.txt matter?

One can sort robots.txt this way: User-agent: DESIRED_INPUT Sitemap: https://example.com/sitemap-index.xml Disallow: / instead: User-agent: DESIRED_INPUT Disallow: / Sitemap: https://example.com/...
4
votes
1answer
92 views

Can I set a priority for images in a sitemap?

I have two diagrams on a page. One is a newer version of the other (the older image is still useful for some people, but most would refer to the newer image). The problem is that the older image is ...
4
votes
2answers
389 views

Impact of blocking site search from googlebot, but listing content pages in sitemap.xml

We have a fairly large site 600k+ unique pages with various unique URLs along with a custom, local search interface for the site. The search interface has sorting and faceting capabilities to allow ...
4
votes
1answer
3k views

Where can I put my sitemap in my directory structure under Laravel? It is 404 in the directory root

I'm trying to submit my sitemap on my Google console but every time I add it, it keeps saying it couldn't fetch, I have uploaded my sitemap.xml into my domain root directory but still Google is not ...
4
votes
2answers
216 views

Can an XML sitemap have Drupal permalink URLs instead of alias URLs?

While the following question is totally general and theoretical, it is based on a more particular case regarding the current development state of the simple_sitmeap module developed for Drupal 8 and ...
4
votes
1answer
85 views

Understanding product search links in sitemaps

I am looking at sitemaps of a few large websites like ebay.com and I noticed that the URLs in the sitemap point to search results rather than the actual product page and I have confirmed that all the ...
4
votes
1answer
8k views

Is there an XML sitemap generator with command line interface for nginx on Linux?

I'm looking for an XML sitemap generator that can be triggered from the command line, supports nginx, and works on Linux (Debian). What can you recommend?
4
votes
1answer
323 views

Receiving “no return tags (sitemaps)” error in Google Search Console should I add hreflang=“en” to sitemap XML file?

Under my Google Search Console I'm receiving a 'es' - no return tags (sitemaps) error for: Originating URL: /en/about-us/testimonials Alternate URL: /es/about-us/testimonials And I have my sitemap ...
4
votes
1answer
60 views

Is it harmful to have irrelevant xmlns attributes in a sitemap urlset element?

For example, if a site has absolutely no videos and is not a news site, is there any negative (or positive) impact to having a sitemap urlset element like this: <urlset xmlns="http://www.sitemaps....