Questions tagged [sitemap]
A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.
102
questions
271
votes
20
answers
45k
views
The Sitemap Paradox
We use a sitemap on Stack Overflow, but I have mixed feelings about it.
Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
- 13.9k
3
votes
3
answers
2k
views
HTTP to HTTPS: Wait for new sitemap to be indexed?
Running HTTP and HTTPS versions of the site side-by-side.
HTTP is canonical right now. Goal is to change that to HTTPS.
Created a new site in Search Console and submitted a new sitemap 4 days ago.
...
- 31
30
votes
4
answers
14k
views
Prevent XML sitemaps from showing up in Google search results
How do I prevent my XML sitemap files from showing up in Google search results like this result of a site: search query:
I don't understand why Google would choose to show sitemap files in search ...
- 98.3k
14
votes
4
answers
27k
views
Sitemaps - one per subdomain or one for the base domain?
I have more than 5 subdomains on a project, say developer, www, staff and some more, what is the best practice in creating sitemaps?
All subdomains that share a same sitemap at base domain (example....
- 255
9
votes
4
answers
11k
views
Google doesn't seem to update the description or title of my homepage
Before we launched our website, we had set up a "coming soon" page and google picked up the title and description from its contents. So the description in the search results said:
Coming soon! ...
- 191
4
votes
1
answer
5k
views
How to fix "404 Not Found" crawl errors reported by Google in Search Console
I was checking on Google webmaster tools for my blog site to see if there's any indication on why my blog traffic decreased to half in one day and i saw 43 Not Found crawl errors and 5 in Sitemap Not ...
- 41
7
votes
3
answers
3k
views
Google says my sitemap index was "processed successfully", yet "total discovered URLs" is 0
I submitted a new version of the sitemap index for my site in Google Search Console two days ago. Soon after submission, GSC started saying that the index was "processed successfully", and yet has ...
- 274
5
votes
6
answers
1k
views
Looking for a sitemap generator
I am looking for a good way to generate a sitemap for my current site. I inherited a huge site at work and we are going to start the process of IA analysis and redesign. It would be awesome if there ...
- 270
8
votes
3
answers
782
views
Shall I submit sitemap containing all the bad (now 404) URLs after the spam URL hack?
Earlier this month, one of my sites got hacked and an attacker created near 1 million spam URLs. The site has been fixed relatively fast, however, after one month Google search results still shows ...
- 439
5
votes
6
answers
6k
views
How to tell Google that I have changed my website URLs?
I have done major updates to my website, and renamed all my URLs.
How can I tell Google that I have renamed these URLs and let Google refresh its index?
My website URL: http://www.pndmasr.com/
My ...
- 153
2
votes
2
answers
2k
views
Why the new Google Search Console does not read my sitemap?
I have the following problem:
I cannot submit any sitemap recently to Google Search Console.
Let me give you an example:
https://ru.azrhymes.com/static/sitemaps/ru/sitemap-main-1.xml.
(validator says ...
- 145
12
votes
7
answers
10k
views
Does Google penalize daily updated <lastmod> tags in sitemaps if the data is not daily updated?
I've got a sitemap that is generated daily with a lot of links to product pages. These products are imported daily from another data source. Because the update consists of throwing away all current ...
- 183
5
votes
1
answer
82
views
SEO consequences for category pages 404-ing
Question, is it ok for one of the pages levels to 404? (SEO-wise)
Let's say I have a sitemap like this:
example.com
--/services
--/foo
--/bar
--/sectors
--/my-sector
--/another-sector
--/...
- 53
3
votes
1
answer
128
views
Warning after submitting sitemap to Google: "We found that some URLs redirect to other locations"
I noticed that I have three warnings when I submitted my Sitemap. They all say this:
When we tested a sample of URLs from your Sitemap, we found that some
URLs redirect to other locations. We ...
- 31
73
votes
5
answers
30k
views
Are there any clear indicators that my sitemap file is beneficial?
I have recently created a sitemap.xml file and uploaded it to my Google Webmasters Tools account. Google didn't report any issues or errors with the uploaded sitemap of my site.
Now my question is:
...
user1420373
15
votes
1
answer
6k
views
Can I host a sitemap on another domain?
Just out of curiosity, I have developed a small app analysing and generating sitemaps for a given domain.
Can I tell search engines to look for the sitemap of a given domain on my app using a ...
- 434
4
votes
1
answer
905
views
Should paginated sites be included in sitemap?
My website is ad listing for boats, boats' engines, boat rentals, etc (www.mynautics.com).
There are ads listed on multiple pages. I have finished (not jet seen on link above) pagination:
added prev
...
- 177
2
votes
1
answer
818
views
Sitemaps & <link rel=... for quad-directional websites
I realize there are many similar questions, but I think this consolidates multiple points of confusion. This question follows from the great advice given here. I have a site that has a separate mobile ...
- 105
1
vote
4
answers
1k
views
How to remove indexed pages from google effectively
My site was getting traffic and it was on the rise, then I added https and somewhere in that time I also created tons of extra link for google to crawl, thinking this will increase traffic in those ...
- 552
8
votes
3
answers
11k
views
How to submit sitemap to Baidu?
I've some pages in my site in Chinese but I don't understand Chinese (it's some GIS data).
Is there any way to submit sitemap to Baidu?
- 1,438
7
votes
1
answer
296
views
If a website is accessed on HTTPS instead of HTTP, should the URLs on the sitemap be HTTPS or HTTP?
I have changed my website from HTTP to HTTPS.
At this moment, I have the next rewrite rule working to redirect calls for HTTP to HTTPS:
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(...
- 107
6
votes
4
answers
6k
views
Google Webmaster Tools says my XML sitemap "appears to be an HTML page"
We're running a lot of sites and we've started to get a lot of these errors in Webmaster Tools:
Sitemap is HTML
Your Sitemap appears to be an HTML page. Please use a supported sitemap format instead.
...
- 411
5
votes
2
answers
1k
views
Why do some websites hide their sitemaps and exclude mentioning it from the robots.txt file? Does this affect SEO at all?
A lot of websites hide their sitemap.xml file.
What I mean by this is that they make the URL something that a person cannot guess and then do not mention them in the robots.txt file.
So my question is ...
- 689
4
votes
1
answer
3k
views
Submitting a URL to Google using PHP/cUrl
I've been successfully submitting xml sitemaps to Google using cURL.
The URL is http://www.google.com/webmasters/sitemaps/ping?sitemap=
The problem is that XML sitemaps take several days to be ...
- 43
3
votes
3
answers
6k
views
When should I update lastmod value in the sitemap?
I have a website like Stackoverflow. I'm creating a sitemap for it. I want to know, what timestamp should be as the value of last-modified for a page? (a question and all its answers)
All I'm trying ...
- 415
3
votes
2
answers
11k
views
Free unlimited-page sitemap generator? [duplicate]
Possible Duplicate:
Looking for a sitemap generator
I have been using www.xml-sitemaps.com/ but their limit is 500 pages. My site has grown beyond that and I am wondering whether there is ...
- 1,264
3
votes
1
answer
2k
views
Should I change the URLs in the XML sitemap after 301 redirecting them?
I am using 301 redirects for all URLs. Should I change sitemap URLs now?
The sitemap currently contains the following URLs:
example.com/cat/1.html
example.com/cat/2.html
example.com/cat/3.html
...
- 143
3
votes
1
answer
1k
views
Does name of sitemap file that Wordpress generates matter?
The Yoast SEO plugin for wordpress creates an XML sitemap with the file name sitemap_index.xml. There is another sitemap plugin (Google XML Sitemaps) in the Wordpress site I'm working on that creates ...
- 225
3
votes
1
answer
127
views
How should I choose which URLs I want indexed to include in my sitemap?
I am working on a sitemap. I have read it should contain the URLs that you want indexed by Google. But which pages do I want indexed?
It seems to me that makes sense to index pages such as the root ...
2
votes
1
answer
251
views
Why my sitemap.xml isn't recognized by the Google Search Console?
I have generated this sitemap.xml file :
<?xml version="1.0" encoding="UTF-8"?>
<urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xsi=...
- 71
1
vote
1
answer
7k
views
How to control what appears on google search results?
So I have just built a site and all is well... but I want to control what appears on google search results.
Lets say I google for "thefwa", first what comes is the link to www.thefwa.com, then ...
fjckls
1
vote
1
answer
226
views
Google Search Console Adding Sitemap "Couldn't fetch" and "Sitemap could not be read" txt file
I made a fairly simple .txt file
I am getting the following errors
I find it very odd. The 2.txt is just the .txt with fewer than 50,000 results. Somehow it seems like Google can read the longer .txt ...
- 111
1
vote
2
answers
926
views
Do Google sitemap URLs need to be double urlencoded?
Do Google sitemap URLs need to be double urlencoded?
I am submitting my sitemaps to Google and Google is having difficulty navigating to URLs that contain urlencoded characters. An example of a ...
- 459
1
vote
1
answer
1k
views
Sitemap update lastmod and ping. what is the advantage?
My website has 6milions pages of companies.
ex: www.mywebsite.com/company-a
I will generate one sitemap index file and put the url in robots.txt
When the company A update one information in your ...
- 13
0
votes
2
answers
548
views
Question about paginated based URLs and category URLs with XML sitemaps
I have a blog site that has this URL structure:
http://website.com/category/
http://website.com/category/page1/
http://website.com/category/subcat/
http://website.com/category/article_title
A flow ...
- 363
16
votes
4
answers
5k
views
Can the files robots.txt and sitemap.xml be dynamic through an .htaccess redirect?
I have a multilanguage and multidomain site. It runs through a unique CMS installation (Drupal), so I have a single root directory. So if I have a static robots.txt, there I only can show the files ...
- 405
15
votes
4
answers
9k
views
How long does Bingbot typically take to process a sitemap submission?
I submitted a domain's sitemap.xml to Bingbot (in Webmaster Tools).
It's been a couple of days but the status is still pending. Does anyone know how much time does Bing Webmaster Tools takes to ...
- 253
11
votes
1
answer
2k
views
Internal links to pages vs. sitemap.xml links to pages
I have a page that is not linked to from anywhere, but it is listed in sitemap.xml.
It will be crawled and display in the SERP.
Would it be preferable, from a SEO perspective, to have the page ...
- 211
10
votes
2
answers
1k
views
Can an XML sitemap file by generated dynamically, or does it have to be static?
Does it matter if I serve sitemap.xml as a dynamic page? e.g., use .htaccess to rewrite sitemap.xml (or whatever static URL I submitted at the Webmaster Tools) to a PHP file. The PHP file will call ...
- 683
9
votes
2
answers
4k
views
Ask.com sitemap crawler down (for good)?
I have been using Ask.com as one of the search engines to ping my sitemap. When I went to ping my sitemap today to have my sitemap re-indexed, the website (http://submissions.ask.com/ping) says it no ...
- 554
8
votes
1
answer
1k
views
Which pages from the sitemap are not indexed by Google?
I have a sitemap file that has been submitted to Google, and has been processed, according to Google Webmaster Tools.
As per GWT for 2013-05-31, 8255 web-pages are submitted, but only 8209 are ...
- 972
8
votes
3
answers
2k
views
Are there specific advantages or disadvantages of an XML sitemap over a TXT based sitemap?
Are there any specific advantages or disadvantages in generating an XML based sitemap in place of a simple txt based sitemap (list of URLs)?
I realize that in the XML format I can set priority and ...
- 325
7
votes
2
answers
3k
views
Is it possible to make an XML sitemap pretty enough to show to users?
I wouldn't mind showing my XML sitemap to users if there were a way to make it human usable as opposed to just machine readable. Is it possible to:
Choose the colors and layout
Make the locations ...
- 98.3k
7
votes
1
answer
5k
views
How important is it to include <lastmod> in a sitemap?
In a sitemap, is it important to include information about when the url was last changed in a <lastmod> tag?
How does not including a <lastmod> tag in a sitemap affect how my site is ...
- 649
7
votes
3
answers
7k
views
Sitemap: Hierarchical layout possible?
Is it possible to design my sitemap hierarchically?
My site is laid out hierarchically into
World=>Continent=>Country=>State=>City=>Articles
Can I design my sitemap xml like this?...
- 512
7
votes
3
answers
1k
views
How soon Google is supposed to include new pages to search
I'm using Google CSE on my site. As soon as new entity is published, it's immediately added to sitemap.xml rendering script too. How soon after Googlebot fetches new sitemap new entities should be ...
- 193
7
votes
4
answers
232
views
Sitemaps HTML and others [closed]
I am looking for the best resources on how to build sitemaps.
I am currently aware of the obvious HTML sitemaps and the (google) sitemap.xml and (yahoo) urllist.txt
Anyone know of resources for ...
- 281
6
votes
3
answers
10k
views
Sitemap to display hash tag links or not?
I am just creating sitemap for my site and wanted to find out if I should include # links for SEO?
For Example:
www.mydomain.com/#comment1
www.mydomain.com/#comment2
Does this help with SEO or ...
- 63
6
votes
3
answers
3k
views
How long does it take for Google Webmasters to index site after submiting sitemap? [duplicate]
Possible Duplicate:
Why isn't my website in Google search results?
I have submitted my website's sitemap to webmaster's tools.
The status on the sitemap says OK and it shows that 12 URLs ...
Venkatesh Hodavdekar
6
votes
1
answer
2k
views
Flat vs. Hierarchical Tree Structure
I am a developer in the middle of the development of a medium-sized website. A requirement that we just received from the resident SEO person on the project is for all the URLs on the site to be just ...
- 161