Questions tagged [xml-sitemap]

XML Sitemaps are XML files created to make it easier for Search Engine crawlers and other bots to efficiently and fully crawl a website.

Filter by
Sorted by
Tagged with
259
votes
19answers
40k views

The Sitemap Paradox

We use a sitemap on Stack Overflow, but I have mixed feelings about it. Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
72
votes
5answers
30k views

Are there any clear indicators that my sitemap file is beneficial?

I have recently created a sitemap.xml file and uploaded it to my Google Webmasters Tools account. Google didn't report any issues or errors with the uploaded sitemap of my site. Now my question is: ...
28
votes
4answers
12k views

Prevent XML sitemaps from showing up in Google search results

How do I prevent my XML sitemap files from showing up in Google search results like this result of a site: search query: I don't understand why Google would choose to show sitemap files in search ...
15
votes
2answers
2k views

Should I Include RSS Feed URLs in an XML Sitemap?

Simple question - Should I Include the RSS Feed URLs in my site's XML Sitemap? I am NOT asking whether I should USE an RSS Feed AS an XML sitemap, but rather, should I include the URLs to the ...
12
votes
7answers
10k views

How to hide my XML Sitemap from competitors but not from search engines

I want to hide my sitemap XML file from all but allow access from search engines. What is the way to do it? I want to hide the depth of site's content from competitors.
12
votes
2answers
3k views

Is incorrect to have the HTTPS version of the sitemaps.org URL in the xmlns sitemap schema?

I have the schema with this: <sitemapindex xmlns="https://www.sitemaps.org/schemas/sitemap/0.9"> Is it correct or should be: <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0....
11
votes
1answer
496 views

Proper sitemap.xml setup

I have a dynamic site which has many (well, less than 50) users. Each user is allowed to create as many pages as they want. I know that there is a limit to how many pages you can be listed in ...
10
votes
1answer
1k views

Internal links to pages vs. sitemap.xml links to pages

I have a page that is not linked to from anywhere, but it is listed in sitemap.xml. It will be crawled and display in the SERP. Would it be preferable, from a SEO perspective, to have the page ...
9
votes
4answers
4k views

Robots.txt vs Sitemap — Who wins in a Conflict

If I block off the directory /foo in robots.txt, but my xml sitemap contains URLs with /foo, will the URLs in the sitemap get picked up by Google and other search engines? In other words, does the ...
8
votes
3answers
2k views

Are there specific advantages or disadvantages of an XML sitemap over a TXT based sitemap?

Are there any specific advantages or disadvantages in generating an XML based sitemap in place of a simple txt based sitemap (list of URLs)? I realize that in the XML format I can set priority and ...
8
votes
1answer
998 views

Should I ping to Google sitemap-index.xml or only the sitemap file that has been modified?

I have a sitemap-index.xml with 7 files. After updating one of the 7 with new content should I ping Google with the index file or with the updated file? In my index I also use lastmod so Google can ...
7
votes
1answer
3k views

Why important/big sites don't include a sitemap? [closed]

I've been looking at most important sites and none of them have a sitemap.xml defined. My list includes: Amazon Ebay Yahoo Microsoft ... and others But none of them include a sitemap.xml file. Why ...
7
votes
2answers
2k views

Is it possible to make an XML sitemap pretty enough to show to users?

I wouldn't mind showing my XML sitemap to users if there were a way to make it human usable as opposed to just machine readable. Is it possible to: Choose the colors and layout Make the locations ...
7
votes
1answer
632 views

Are alternate links included in the 50,000 URL limit for sitemaps?

I see the number 50,000 referenced by Google and Schema.org as the limit for the number of URLs that a sitemap can contain, however, it is unclear to me if that means <loc> elements specifically,...
7
votes
3answers
5k views

Can a sitemap index file reference sitemap files under different domains?

My company owns several domain names. We also host websites for our users at their custom domains. Our domains: company.com product1.com product2.com Users's domains: user1.com user2.com ...
6
votes
3answers
6k views

Sitemap: Hierarchical layout possible?

Is it possible to design my sitemap hierarchically? My site is laid out hierarchically into World=>Continent=>Country=>State=>City=>Articles Can I design my sitemap xml like this?...
6
votes
2answers
21k views

Why does my sitemap have a “.gz” extension, and how can I edit it?

An XML sitemap generator plugin for WordPress puts the following strings in my robots.txt file: User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ But other WordPress blogs have lots of ...
6
votes
3answers
9k views

Sitemap to display hash tag links or not?

I am just creating sitemap for my site and wanted to find out if I should include # links for SEO? For Example: www.mydomain.com/#comment1 www.mydomain.com/#comment2 Does this help with SEO or ...
6
votes
2answers
373 views

Google sitemap priorities, am I doing the right thing?

I've built a website which has maritime weather and tide data for UK locations, and each location has 7 day forecast (where data is available). My URL structures are like this: http://www.example.com/...
6
votes
1answer
359 views

In Google Webmaster Tools we have 3 sitemaps attributed to 1 domain

I have a website that has been on the internet for almost 10 years created in "Microsoft FrontPage" with over 900 pages. Currently in Google Webmaster tools it shows up as 2 domains and 3 sitemaps: ...
6
votes
3answers
1k views

Does an XML sitemap need to have a .xml extension?

Im using a CMS that generates a sitemap automatically but without ".xml".IS this ok? Can we put this is our robots.txt: Sitemap: https://www.example.com/sitemap or must it be: Sitemap: https://www....
6
votes
1answer
743 views

If I use the same image on multiple pages, should it be on my sitemap multiple times?

Following Google's guide to Image sitemaps <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:image="http://www.google.com/schemas/...
6
votes
2answers
537 views

Why is Google reporting that hreflang URLs in my sitemap do not have “return tags”?

Google Search Console (Webmaster Tools) keeps listing the following errors for my sitemap: Sitemap provided URLs and alternate URLs in 'de' that do not have return tags. +---+-----------------+----...
5
votes
6answers
6k views

How to tell Google that I have changed my website URLs?

I have done major updates to my website, and renamed all my URLs. How can I tell Google that I have renamed these URLs and let Google refresh its index? My website URL: http://www.pndmasr.com/ My ...
5
votes
5answers
3k views

Is there any benefit to having a link to your website's XML sitemap?

In Google's WebmasterTools, I submitted my site's XML sitemap, which is accessible through www.mysite.com/sitemap. However, no where on the site do I have an actual link that points to this. Is ...
5
votes
3answers
19k views

Sitemap submission : Do sitemaps need to be resubmitted when they are updated?

In short: Should I resubmit my sitemap through Google Webmasters Tools every time I replace a sitemap.xml with an existing sitemap.xml (i.e., www.domain.com/sitemap.xml)? Or does it check it ...
5
votes
3answers
2k views

Should I include forum posts in the sitemap?

I am launching a medical discussion forum for sharing study materials and discussing other topics. The forum will probably grow fast and reach at least a few hundred topics. Should every discussion ...
5
votes
2answers
2k views

How often are sitemap.xml checked for updates by crawlers?

How often do crawlers check for updates in submitted sitemap.xml? Every hour, day, week, month, more?
5
votes
1answer
6k views

Is it okay to have multiple sitemaps?

Recently, I moved a site from http to https and the sitemaps got updated. I submitted the sitemaps to Google. There were around 9 sitemaps for the site, each of them containing pages/images the site. ...
5
votes
1answer
4k views

How important is it to include <lastmod> in a sitemap?

In a sitemap, is it important to include information about when the url was last changed in a <lastmod> tag? How does not including a <lastmod> tag in a sitemap affect how my site is ...
5
votes
2answers
580 views

sitemap.xml for huge online store

I'm running online store with 30,000+ products (having unique urls) which are updated every day. I want google to index my products faster. Now crawler index ~500 pages/day (reports from google ...
5
votes
4answers
9k views

What URLs should I put into XML Sitemap?

We will launch a project with over 1.000.000 products and those are in approx 150 categories. Later on you will find reviews and technical detail pages to those products. So there will be more than 2....
5
votes
1answer
1k views

Sitemap file getting double compressed

I have a script that generates my XML sitemap and writes it to the file sitemap.xml.gz - i.e. an XML file, compressed with gzip. This file is definitely written correctly as when I download it via FTP ...
5
votes
3answers
5k views

Google SEO - remove site map from search result

I am an amateur to SEO. I have a new site recently and submitted the site map to GWT. But after several weeks. I found when I search my domain in google. The site map also appears to be one of the ...
4
votes
3answers
960 views

Do I need an XML sitemap for just a few images and videos?

If a website has few images and one or two videos, do I need to include them in the sitemap or have a sitemap separately for them?
4
votes
4answers
4k views

Yoast SEO plugin creates XML Sitemap with http. Is this correct for a https site?

I have recently upgraded my Wordpress site to https and i have noticed that the Yoast SEO plugin generates sitemaps that have URLS defined as http. Is this incorrect for a https site and if so is ...
4
votes
2answers
4k views

Can a sitemap index file list sitemaps in a different directory?

If I have a sitemap index, can the sitemaps listed in the index be located in a subdirectory, but have links that isn't specific to the subdirectory? I'm trying to avoid having multiple sitemaps in ...
4
votes
3answers
378 views

Delay before getting URLs into Google's in web index?

4 days ago, I have submitted a sitemap.xml into Google's WebMaster Tool for a new website. It contains 3 entries. I have tested the site map and Google found not errors. In about how much time can I ...
4
votes
3answers
1k views

Why is the sitemap validator returning an error for the “lastmod” tag?

The sitemap of my company's brand's website seems to be fine when I look at it at first glance. I don't particularly see any errors as such. However, SEMrush, the SEO tool, reports that our sitemap is ...
4
votes
4answers
4k views

Do you really need robots.txt and sitemap.xml if I want bot to follow every link? [duplicate]

I wonder if I really need robots.txt/*sitemap.xml* files if I want bots to go through every link on my website without any restrictions? And if I do, what should I write in there?
4
votes
1answer
69 views

Can a very large not-so-relevant sitemap hurt my site to Google’s eyes?

I have a Wiki site (running MediaWiki) which I am trying to optimize in terms of page speed and SEO. One of the many things in my to-do list is submit a sitemap on Google Webmaster tools, now here is ...
4
votes
1answer
14k views

What does url > lastmod in a sitemap really mean?

XML sitemap files are specified at sitemaps.org. The description for urlset > url > lastmod says The date of last modification of the file. This date should be in W3C Datetime format. This ...
4
votes
2answers
357 views

Is it OK to “pause” an advert in terms of SEO?

I have a website along the lines of Craigslist, where users can post classified ads. Some users have requested a feature where they're able to pause/resume their ads. From an seo point of view, the ...
4
votes
1answer
154 views

Do I need to update the sitemap.xml file after removing some URLs?

Do I need to update the sitemap.xml file after removing some URLs from the website and merge three different URLs to a single one. For example, three different URLs: www.example.com/one.html (main ...
4
votes
1answer
222 views

Can XML Sitemap include itself as one of the URLs?

A little while ago I discovered an XML sitemap where the sitemap itself was included. <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> ...
4
votes
1answer
137 views

sitemap.xml : shall I put all my URLs including the ones that have “empty result” for a dataset?

Here's my problem: on my website, you can click on all the regions, and the new web page is the result of the search of the region. Thus the result may be empty. Shall I include all those "empty ...
4
votes
1answer
103 views

Can I set a priority for images in a sitemap?

I have two diagrams on a page. One is a newer version of the other (the older image is still useful for some people, but most would refer to the newer image). The problem is that the older image is ...
4
votes
2answers
397 views

Impact of blocking site search from googlebot, but listing content pages in sitemap.xml

We have a fairly large site 600k+ unique pages with various unique URLs along with a custom, local search interface for the site. The search interface has sorting and faceting capabilities to allow ...
4
votes
1answer
246 views

Do the order of “Disallow” and “Sitemap” lines in robots.txt matter?

One can sort robots.txt this way: User-agent: DESIRED_INPUT Sitemap: https://example.com/sitemap-index.xml Disallow: / instead: User-agent: DESIRED_INPUT Disallow: / Sitemap: https://example.com/...
4
votes
1answer
3k views

Where can I put my sitemap in my directory structure under Laravel? It is 404 in the directory root

I'm trying to submit my sitemap on my Google console but every time I add it, it keeps saying it couldn't fetch, I have uploaded my sitemap.xml into my domain root directory but still Google is not ...

1
2 3 4 5
7