A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

168
votes
19answers
24k views

The Sitemap Paradox

We use a sitemap on Stack Overflow, but I have mixed feelings about it. Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
5
votes
2answers
2k views

Google doesn't seem to update the description or title of my homepage

Before we launched our website, we had set up a "coming soon" page and google picked up the title and description from its contents. So the description in the search results said, "Coming soon! Visit ...
4
votes
6answers
2k views

How to tell Google that I have changed my website URLs?

I have done major updates to my website, and renamed all my URLs. How can I tell Google that I have renamed these URLs and let Google refresh its index? My website URL: http://www.pndmasr.com/ My ...
5
votes
6answers
387 views

Looking for a sitemap generator

I am looking for a good way to generate a sitemap for my current site. I inherited a huge site at work and we are going to start the process of IA analysis and redesign. It would be awesome if there ...
57
votes
4answers
28k views

Are there any clear indicators that my sitemap file is beneficial?

I have recently created a sitemap.xml file and uploaded it to my Google Webmasters Tools account. Google didn't report any issues or errors with the uploaded sitemap of my site. Now my question is: ...
8
votes
2answers
374 views

Dynamic sitemap file

Does it matter if I serve sitemap.xml as a dynamic page? e.g., use .htaccess to rewrite sitemap.xml (or whatever static URL I submitted at the Webmaster Tools) to a PHP file. The PHP file will call ...
6
votes
4answers
3k views

Google Webmaster Tools says my XML sitemap “appears to be an HTML page”

We're running a lot of sites and we've started to get a lot of these errors in Webmaster Tools: Sitemap is HTML Your Sitemap appears to be an HTML page. Please use a supported sitemap format ...
5
votes
2answers
222 views

Which pages from the sitemap are not indexed by Google?

I have a sitemap file that has been submitted to Google, and has been processed, according to Google Webmaster Tools. As per GWT for 2013-05-31, 8255 web-pages are submitted, but only 8209 are ...
2
votes
2answers
1k views

Free unlimited-page sitemap generator? [duplicate]

Possible Duplicate: Looking for a sitemap generator I have been using www.xml-sitemaps.com/ but their limit is 500 pages. My site has grown beyond that and I am wondering whether there is ...
0
votes
2answers
200 views

Question about paginated based URLs and category URLs with XML sitemaps

I have a blog site that has this URL structure: http://website.com/category/ http://website.com/category/page1/ http://website.com/category/subcat/ http://website.com/category/article_title A flow ...
7
votes
4answers
178 views

Sitemaps HTML and others

I am looking for the best resources on how to build sitemaps. I am currently aware of the obvious HTML sitemaps and the (google) sitemap.xml and (yahoo) urllist.txt Anyone know of resources for ...
4
votes
1answer
1k views

Flat vs. Hierarchical Tree Structure

I'm a developer in the middle of the development of a medium-sized website. A requirement that we just received from the resident SEO person on the project is for all the URLs on the site be just off ...
4
votes
1answer
362 views

Determining which sitemap entries not indexed by google

We submit a sitemap to google and can see this being indexed in webmaster tools. We have 5140 entries, and have broken this up to 10 child site maps of 500 each. This all seems to be working well. ...
4
votes
1answer
1k views

Is it necessary to re-submit modified sitemaps?

Assuming a sitemap.xml has been registered properly to major players, do these players come and revisit those sitemap.xml from time to time to check for updates and modifications (say new entries are ...
4
votes
2answers
323 views

Frequency to submit sitemap to search engines

I have went live with my site and being new to search engines and SEO fields not sure what should be the best way to handle sitemap.xml. I have created sitemap.xml and submitted it to: Google using ...
4
votes
2answers
428 views

sitemap that contains millions of links/URLs

I am currently working on a shopping / auction website. The involving party who's doing SEO consultancy is insisting to generate URLs for every single combination of products listed on the website, ...
1
vote
3answers
901 views

Where can I find a list of search engine crawler user agents and their domain names?

I'm trying to follow stackoverflow's approach of using a white list for the sitemap. I have tried to find an extensive list of search engine crawlers (and other crawlers) to use in my white list but ...
8
votes
2answers
2k views

Ask.com sitemap crawler down (for good)?

I have been using Ask.com as one of the search engines to ping my sitemap. When I went to ping my sitemap today to have my sitemap re-indexed, the website (http://submissions.ask.com/ping) says it no ...
6
votes
1answer
616 views

Are there specific advantages or disadvantages of an XML sitemap over a TXT based sitemap?

Are there any specific advantages or disadvantages in generating an XML based sitemap in place of a simple txt based sitemap (list of URLs)? I realize that in the XML format I can set priority and ...
6
votes
1answer
1k views

How long does Bingbot typically take to process a sitemap submission?

I submitted a domain's sitemap.xml to Bingbot (in Webmaster Tools). It's been a couple of days but the status is still pending. Does anyone know how much time does Bing Webmaster Tools takes to ...
5
votes
2answers
249 views

How to submit sitemap to Baidu?

I've some pages in my site in Chinese but I don't understand Chinese (it's some GIS data). Is there any way to submit sitemap to Baidu?
4
votes
1answer
150 views

Need I submit sitemap in Webmaster Tools?

I have uploaded my sitemap on server root folder. Do I need it submit it in Google Webmaster Tools?
4
votes
2answers
268 views

Is it possible to find out whether one given page has been indexed?

When one submits sitemaps in Google Webmaster, Google may not index all pages of your site/blog. Is there a way to find out whether one specific page belonging to the sitemap has been indexed or not? ...
4
votes
3answers
210 views

Will ranking be affected with a mobile XML sitemap for a mobile site with the same URLs as the desktop site?

We have a site with both a desktop version and a mobile version. Most of the content are the same and both versions have the same URL, but the HTML generated is device specific. Looking at Google's ...
4
votes
3answers
2k views

Sitemap to display hash tag links or not?

I am just creating sitemap for my site and wanted to find out if I should include # links for SEO? For Example: www.mydomain.com/#comment1 www.mydomain.com/#comment2 Does this help with SEO or ...
4
votes
2answers
160 views

What document types can be included in an XML sitemap?

AFAIK, Google accepts sitemaps which are about ordinary URLs (text/plain), images (image/png, image/gif, image/jpg, etc.), and videos (video/avi, video/mpeg, etc.). My question is, how many Internet ...
3
votes
2answers
1k views

Can a sitemap index file list sitemaps in a different directory?

If I have a sitemap index, can the sitemaps listed in the index be located in a subdirectory, but have links that isn't specific to the subdirectory? I'm trying to avoid having multiple sitemaps in ...
3
votes
1answer
1k views

Should I include PDF files in my sitemap.xml

Should I add a direct link to PDF files in my (xml) sitemap?
2
votes
1answer
79 views

Should paginated sites be included in sitemap?

My website is ad listing for boats, boats' engines, boat rentals, etc (www.mynautics.com). There are ads listed on multiple pages. I have finished (not jet seen on link above) pagination: added ...
2
votes
6answers
559 views

Sitemap tool fails to find all URLs in my site

I added my url to Google http://www.google.com/addurl.html I added a meta tag with description and keywords I added it in Google web master tools and submit the site map There is no any robots in my ...
2
votes
1answer
441 views

Unlimited free sitemap generator for Nginx? [duplicate]

Possible Duplicate: Looking for a sitemap generator Google has listed a couple of SMGs but I don't want to try every single one to find the best. Google's own Sitemap Generator is released ...
2
votes
3answers
148 views

XML Sitemaps, what should they link to?

If I want to make an XML sitemap for my site, do I need to map every forum topic in my forum as a separate URL? If this would help, how do you deal with the hugeness of the sitemap for forums with ...
1
vote
2answers
88 views

Do Google sitemap URLs need to be double urlencoded?

Do Google sitemap URLs need to be double urlencoded? I am submitting my sitemaps to Google and Google is having difficulty navigating to URLs that contain urlencoded characters. An example of a ...
1
vote
3answers
57 views

Right canonical pages for encylopedia-style site

I have a website that contains a lot of short (150 word) descriptions of key concepts. The concepts are organised hierarchically so a description of Cheese will be a subsection under the heading "Milk ...
1
vote
1answer
387 views

How often does GWT check dynamic sitemaps?

I'm working on a fairly large site, that generates a dynamic sitemap hourly. Now in Google Webmaster Tools the sitemap isn't submitted yet and I'm shying away because I'm afraid that the new content ...
1
vote
1answer
316 views

Google Sitemap Limits?

I've read in multiple places that Google's sitemap limits sits at 50,000 URLs per sitemap - though it's my understanding that you can submit multiple sitemaps to overcome this problem. I've also ...
1
vote
1answer
64 views

Sitemap creators not visiting most pages

I am trying to generate a sitemap for a customers website, http://www.ruralmarketing.co.nz . When I use the online generators it only ever creates a sitemap with two urls, the index and contact page. ...
1
vote
2answers
320 views

When does Google Webmaster Tools re-evaluate re-submitted sitemaps?

When I alter and re-submit a sitemap does GWTs immediately re-evaluate it or do I have to wait 'til GoogleBot crawls our site again? I just re-submitted a problematic sitemap and it now displays ...