Questions tagged [sitemap]
A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.
812
questions
269
votes
20
answers
44k
views
The Sitemap Paradox
We use a sitemap on Stack Overflow, but I have mixed feelings about it.
Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
- 13.9k
73
votes
5
answers
30k
views
Are there any clear indicators that my sitemap file is beneficial?
I have recently created a sitemap.xml file and uploaded it to my Google Webmasters Tools account. Google didn't report any issues or errors with the uploaded sitemap of my site.
Now my question is:
...
user1420373
30
votes
4
answers
14k
views
Prevent XML sitemaps from showing up in Google search results
How do I prevent my XML sitemap files from showing up in Google search results like this result of a site: search query:
I don't understand why Google would choose to show sitemap files in search ...
- 97.2k
19
votes
6
answers
5k
views
Why did Google stop indexing pages from our sitemap.xml?
We're seeing some pages that exist in our sitemap.xml but are inexplicably missing from Google's public search index.
You can't download https://superuser.com/sitemap.xml -- we protect this file ...
- 935
18
votes
5
answers
14k
views
& in sitemap links, are they correct?
Simple question, I'm asking just to make sure.
A Google sitemap generator generated a sitemap.txt fle with links written like this:
http://www.domain.com/category.htm?name=some-name&cat_id=8
...
- 6,450
17
votes
1
answer
26k
views
Does sitemap.xml have to be in the root directory?
This may be a simple and absurd question, but is it completely necessary for sitemap.xml file to be in the root directory of the wordpress install directory of the site, or is it ok to create a ...
- 303
16
votes
3
answers
7k
views
Google Search Console is making up URLs which don't exist in my Sitemap and then complains that these pages have error
I am working on an e-commerce website. I used to have a sitemap with category Ids, something like:
<url>
<loc>https://my-domain.com/home/browse/2/45/139</loc>
<changefreq&...
- 538
16
votes
3
answers
16k
views
Can a sitemap index contain other sitemap indexes?
I have a bi-lingual website with a sitemap index for each language that links to several different sitemaps (one for videos, one for static content, and one for articles). I would like to introduce ...
- 369
16
votes
4
answers
5k
views
Can the files robots.txt and sitemap.xml be dynamic through an .htaccess redirect?
I have a multilanguage and multidomain site. It runs through a unique CMS installation (Drupal), so I have a single root directory. So if I have a static robots.txt, there I only can show the files ...
- 405
15
votes
1
answer
4k
views
Should I include everything in the sitemap or only new content?
For a website with dynamic content (new content is constantly being added), should I only include the newest content in the sitemap or should I include everything (with a sitemap index)? What are the ...
- 533
15
votes
1
answer
6k
views
Can I host a sitemap on another domain?
Just out of curiosity, I have developed a small app analysing and generating sitemaps for a given domain.
Can I tell search engines to look for the sitemap of a given domain on my app using a ...
- 434
15
votes
4
answers
9k
views
How long does Bingbot typically take to process a sitemap submission?
I submitted a domain's sitemap.xml to Bingbot (in Webmaster Tools).
It's been a couple of days but the status is still pending. Does anyone know how much time does Bing Webmaster Tools takes to ...
- 253
14
votes
4
answers
507
views
easy tool for creating XML sitemaps? (constantly expanding site)
Does anyone recommend either
a php script that I can run as a CRON
or
somekind of thirdparty tool i can use
to create sitemaps?
My website is events bases and so the pages are pretty temporal with ...
- 293
13
votes
4
answers
26k
views
Sitemaps - one per subdomain or one for the base domain?
I have more than 5 subdomains on a project, say developer, www, staff and some more, what is the best practice in creating sitemaps?
All subdomains that share a same sitemap at base domain (example....
- 245
13
votes
3
answers
22k
views
Is a different sitemap per language OK? How do I tell Google about them?
The website is very, very large and I have created:
sitemap_fr.xml
sitemap_en.xml
sitemap_es.xml
Is this solution OK? If so, do I have to insert a tag to inform Google about the different sitemaps?...
- 135
13
votes
2
answers
7k
views
How to properly use Google Webmaster Tools (for "non-www" instead of "www")
I'm trying to use Google Webmaster Tools correctly: I don't use the www in my website's URL, but just noticed that I must have entered the www when I originally added my site to Webmaster Tools a few ...
- 233
11
votes
7
answers
10k
views
Does Google penalize daily updated <lastmod> tags in sitemaps if the data is not daily updated?
I've got a sitemap that is generated daily with a lot of links to product pages. These products are imported daily from another data source. Because the update consists of throwing away all current ...
- 173
11
votes
6
answers
3k
views
Is it necessary/useful to build a sitemap for a small website?
I've created several small scale websites, usually for small businesses. They usually contain some information about the company, a contact form, and sometimes a few dynamic parts : news, pictures, ...
- 1,597
11
votes
2
answers
657
views
Have search engines recognise multi-lingual sites
I have organised my site by language using the following structure:
www.domain.com #spanish language version homepage
www.domain.com/en #english language version homepage
The site is of Spanish ...
- 325
11
votes
3
answers
5k
views
How to create a good sitemap for dynamic website
I have a website with dynamic content and different kind of pages. I have some pages that rarely change, and I have pages like blogs that change often. The blog pages also have links for sorting, for ...
- 1,580
11
votes
1
answer
2k
views
Internal links to pages vs. sitemap.xml links to pages
I have a page that is not linked to from anywhere, but it is listed in sitemap.xml.
It will be crawled and display in the SERP.
Would it be preferable, from a SEO perspective, to have the page ...
- 211
10
votes
2
answers
1k
views
Can an XML sitemap file by generated dynamically, or does it have to be static?
Does it matter if I serve sitemap.xml as a dynamic page? e.g., use .htaccess to rewrite sitemap.xml (or whatever static URL I submitted at the Webmaster Tools) to a PHP file. The PHP file will call ...
- 673
10
votes
2
answers
898
views
Google is changing my urls in search results - Joomla3 bug
My urls is something like
escene.ir/component/products/تلفن-های-تحت-شبکه-مدیریتی/Escene-ES620-PEN-IP-Phone?task=view.12
but when I try to search site:escene.ir to see the whole links of my site, ...
- 273
10
votes
2
answers
12k
views
Is it necessary to submit sitemap to www and non www?
I have submitted my website to Google Webmaster Tools with both www and with no-www. I had made the domain without www to my preffered domain and have submitted a sitemap.
My question is should I ...
- 201
10
votes
1
answer
403
views
using multiple sitemaps in a domain
I have a domain http://sub.example.com/.
In it there is a wordpress blog at http://sub.example.com/blog and other more static content at http://sub.example.com/other, http://sub.example.com/stuff, ...
- 205
9
votes
2
answers
4k
views
Ask.com sitemap crawler down (for good)?
I have been using Ask.com as one of the search engines to ping my sitemap. When I went to ping my sitemap today to have my sitemap re-indexed, the website (http://submissions.ask.com/ping) says it no ...
- 554
9
votes
4
answers
10k
views
Google doesn't seem to update the description or title of my homepage
Before we launched our website, we had set up a "coming soon" page and google picked up the title and description from its contents. So the description in the search results said:
Coming soon! ...
- 191
8
votes
5
answers
2k
views
Is it possible to 'trigger' indexing on a site?
We're launching a redeveloped site soon. Google has already indexed the site that we're going to replace, but the links that google indexed are no longer applicable, because the new site has a totally ...
- 213
8
votes
3
answers
2k
views
Are there specific advantages or disadvantages of an XML sitemap over a TXT based sitemap?
Are there any specific advantages or disadvantages in generating an XML based sitemap in place of a simple txt based sitemap (list of URLs)?
I realize that in the XML format I can set priority and ...
- 325
8
votes
3
answers
24k
views
Sitemap for a one-page site
I'm wondering whether it's worth the effort to create a sitemap for a single-page site.
By single page I'm not talking about a webapp made with Angular or Backbone, just a presentation site/page ...
- 215
8
votes
3
answers
10k
views
How to submit sitemap to Baidu?
I've some pages in my site in Chinese but I don't understand Chinese (it's some GIS data).
Is there any way to submit sitemap to Baidu?
- 1,448
8
votes
1
answer
13k
views
Locate sitemap at HTTP or HTTPS?
Our homepage has a 301 redirect from HTTP to HTTPS but all other pages on the site are on HTTP.
(1) Should the sitemap be located at HTTPS and is there any problem if we:
(2) include the HTTPS ...
- 113
8
votes
1
answer
5k
views
Will an incorrect change frequency in my XML sitemap harm my website?
What happens if I set the change frequency of a page to daily and don't update it daily? Will this make search engine blacklist my site? What is the accepted changefreq to balance search indexing ?
user581544
8
votes
1
answer
1k
views
Which pages from the sitemap are not indexed by Google?
I have a sitemap file that has been submitted to Google, and has been processed, according to Google Webmaster Tools.
As per GWT for 2013-05-31, 8255 web-pages are submitted, but only 8209 are ...
- 972
7
votes
3
answers
543
views
Shall I submit sitemap containing all the bad (now 404) URLs after the spam URL hack?
Earlier this month, one of my sites got hacked and an attacker created near 1 million spam URLs. The site has been fixed relatively fast, however, after one month Google search results still shows ...
- 419
7
votes
1
answer
3k
views
Why important/big sites don't include a sitemap? [closed]
I've been looking at most important sites and none of them have a sitemap.xml defined. My list includes:
Amazon
Ebay
Yahoo
Microsoft
... and others
But none of them include a sitemap.xml file. Why ...
- 203
7
votes
3
answers
6k
views
Sitemap: Hierarchical layout possible?
Is it possible to design my sitemap hierarchically?
My site is laid out hierarchically into
World=>Continent=>Country=>State=>City=>Articles
Can I design my sitemap xml like this?...
- 512
7
votes
2
answers
179
views
Can a sitemap change result in a drop of visits?
I recently restructured and resubmitted the sitemap for a website of mine in GWT. The old one was built by a script I acquired, which had its flaws but dit the job. After having implemented URL ...
- 483
7
votes
2
answers
2k
views
Is it possible to make an XML sitemap pretty enough to show to users?
I wouldn't mind showing my XML sitemap to users if there were a way to make it human usable as opposed to just machine readable. Is it possible to:
Choose the colors and layout
Make the locations ...
- 97.2k
7
votes
4
answers
230
views
Sitemaps HTML and others [closed]
I am looking for the best resources on how to build sitemaps.
I am currently aware of the obvious HTML sitemaps and the (google) sitemap.xml and (yahoo) urllist.txt
Anyone know of resources for ...
- 281
7
votes
1
answer
841
views
Should URLs in the sitemap contain the slug?
I'm coding the sitemap logic of my side-project blogging platform and I would like to know if omitting the slug in the <loc> nodes of the sitemap. Could be a problem for the crawlers.
Having ...
- 299
7
votes
1
answer
858
views
Are alternate links included in the 50,000 URL limit for sitemaps?
I see the number 50,000 referenced by Google and Schema.org as the limit for the number of URLs that a sitemap can contain, however, it is unclear to me if that means <loc> elements specifically,...
- 171
7
votes
1
answer
294
views
If a website is accessed on HTTPS instead of HTTP, should the URLs on the sitemap be HTTPS or HTTP?
I have changed my website from HTTP to HTTPS.
At this moment, I have the next rewrite rule working to redirect calls for HTTP to HTTPS:
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(...
- 107
7
votes
1
answer
2k
views
RSS vs Sitemap for SEO
What are the differences of the two and how can they help in SEO? I'm creating a custom-scripted CMS and I'm thinking if should I either add RSS/Sitemap/Both?
- 375
7
votes
3
answers
1k
views
How soon Google is supposed to include new pages to search
I'm using Google CSE on my site. As soon as new entity is published, it's immediately added to sitemap.xml rendering script too. How soon after Googlebot fetches new sitemap new entities should be ...
- 193
7
votes
6
answers
304
views
In regards to SEO, is a site map important for a small 3 page website?
In regards to SEO, is a site map important for a small 3 page website? I understand that it is not needed for the convenience of a user visiting the site, but will it help the search engines find it?
- 71
7
votes
3
answers
3k
views
Google says my sitemap index was "processed successfully", yet "total discovered URLs" is 0
I submitted a new version of the sitemap index for my site in Google Search Console two days ago. Soon after submission, GSC started saying that the index was "processed successfully", and yet has ...
- 274
7
votes
1
answer
128
views
How to tell which URLs have fallen out of search engine index
I've had a few pages trimmed from google's indexed number on my sitemap in webmaster tools. How can I tell which pages have been dropped?
- 3,270
7
votes
3
answers
6k
views
Can a sitemap index file reference sitemap files under different domains?
My company owns several domain names. We also host websites for our users at their custom domains.
Our domains:
company.com
product1.com
product2.com
Users's domains:
user1.com
user2.com
...
- 171
7
votes
3
answers
6k
views
Should I submit my RSS feed as Google Sitemap?
I currently have no sitemap for a website I'm creating.
I do have an RSS feed which includes the N latest updated posts on the site. It doesn't include everything on the site though, just blog posts. ...
- 543