A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

3
votes
3answers
598 views

What are some client-side sitemap generators?

Most of the sitemap generators I've found all scan my internal files and base it on that. However using apache and htaccess I have many aliases for things for example: /brand/model/photos/ is empty ...
1
vote
3answers
706 views

Does anyone have statistics on percentage of people that use search feature on a website as opposed to the navigation bar

I was interested in knowing percentage of people that use search feature on a website as opposed to the navigation bar. Has search bar become more powerful than the traditional navigation bar for ...
1
vote
1answer
1k views

is it ok to have 2 sitemaps on 1 website?

Do I have to have a sitemap page on my index page for bots to read it or can I just have it anywhere on my server? I have a phpbb/wordpress integration and I need 2 sitemaps mods for each one (or I ...
0
votes
1answer
349 views

webmaster tools - Network Unreachable

webmaster tools for my site displays that robots.txt unreachable and for all links in sitemap it says network unreachable. sitemap.xml unreachable. These appear in crawl stats page. I discussed ...
1
vote
0answers
130 views

How does Stack Overflow have questions appear in Google so quickly? [duplicate]

Possible Duplicate: StackOverflow and Google search I have noticed while asking questions on Stack Overflow that they appear in Google almost instantly. How does SO do this? Is it a feed ...
7
votes
2answers
2k views

Google doesn't seem to update the description or title of my homepage

Before we launched our website, we had set up a "coming soon" page and google picked up the title and description from its contents. So the description in the search results said, "Coming soon! Visit ...
0
votes
1answer
191 views

Sitemap.xml.gz. can I have a compressed file with more than one sitemap in it?

I've got quite a few sitemaps as my site is quite large. Is there any way that I can have a compressed with with multiple sitemap files in it ? I've googled but can't find the answer to this
2
votes
3answers
241 views

Where should sitemap be? (not separate sitemap file)

Most of the big companies got included sitemap at the bottom of thier site. e.g. Piriform / PCTools. I would like to know if there's reason why to put the sitemap above the copyright or they just put ...
1
vote
3answers
565 views

YAHOO and BING support for Index, Image and Mobile sitemaps

I know Google webmaster supports submitting Image, mobile, video and other types of sitemaps. YAHOO also mentions about mobile site map here. But does it support Image and video sitemaps. I could not ...
0
votes
0answers
104 views

Is there a way that is more efficient than sitemap to add/force recrawl/remove your website's index entries in google? [duplicate]

Possible Duplicate: StackOverflow and Google search I'll make it more simple: Right now I am posting this question, after a minute or two... may be it will appear on google search query: ...
8
votes
2answers
567 views

Have search engines recognise multi-lingual sites

I have organised my site by language using the following structure: www.domain.com #spanish language version homepage www.domain.com/en #english language version homepage The site is of Spanish ...
1
vote
1answer
253 views

How Does One Make a Sitemap for a Flex Website?

I've got a Flex website and I'm not sure how to make a sitemap for it. For a standard HTML page, sitemaps are straight forward. As Flex sites use a "pageless" architecture, I'm not sure what I should ...
1
vote
2answers
655 views

Yahoo sitemap validation

I am trying to submit sitemap.xml (Index) to Yahoo Site Explorer but with no luck. I tried using website feed option in the site explorer to submit the sitemap, but I got validation errors. However, ...
0
votes
1answer
86 views

is there anything wrong with my sitemap.xml

I had written an article over Highly searched keywords some days ago. When i search it in google using top searched keywords article-stack, it returns many results from my site. But none of them was ...
3
votes
1answer
1k views

Should I include PDF files in my sitemap.xml

Should I add a direct link to PDF files in my (xml) sitemap?
2
votes
1answer
182 views

Sitemap contains a lot of products and web index contains a few according to Google webmaster tools

Our sitemap has something like 10 million pages: This is the data in the webmaster tools: Submitted URLs 10,873,781 2 URLs in web index. This is a big difference and an incorrect statement, since ...
0
votes
3answers
662 views

Tumblr and Sitemaps

I was wondering if anyone knew how to add a sitemap to a tumblr blog. It was what i use to share stuff online. Since tumblr doesn't let you upload to example.com/sitemap So there lies my ...
2
votes
1answer
159 views

Validating sitemap with Yahoo Explorer

I have a sitemap index on my website, which I successfully validated with "Google webmasters tools". The declarations at the top are: <sitemapindex ...
4
votes
2answers
445 views

sitemap that contains millions of links/URLs

I am currently working on a shopping / auction website. The involving party who's doing SEO consultancy is insisting to generate URLs for every single combination of products listed on the website, ...
0
votes
1answer
107 views

SItemap for site with translations

I have a domain.com which I translated all of the pages in my domain to several languages, all on their own subdomain (e.g. es.domain.com, cn.domain.com, ru.domain.com ...). The top level domain ...
1
vote
2answers
303 views

Validating sitemaps with images tag

I am trying to add the <image> tag to my sitemap, but even trying to look at the XSD , as mentioned by Google at http://www.google.com/support/webmasters/bin/answer.py?answer=183668 , I get 404 ...
8
votes
2answers
382 views

Dynamic sitemap file

Does it matter if I serve sitemap.xml as a dynamic page? e.g., use .htaccess to rewrite sitemap.xml (or whatever static URL I submitted at the Webmaster Tools) to a PHP file. The PHP file will call ...
2
votes
2answers
175 views

Sitemaps index submission

I have two questions regarding site maps: 1) Currently my site has around 5,000 pages. However, in the future it will contain more than 50,000 pages. Should I start using sitemap-index with only one ...
5
votes
1answer
514 views

Sitemap XML version vs simple text (urls list) version, do you know what are the pros?

I usualy submit simple text urls list as sitemap on Google webmaster tools. It's just a matter of simplicity, a simple text file is lightweight, easier to read also for humans (actually for me). :) ...
1
vote
1answer
86 views

What are Sitemap content tags?

i have recently generated sitemap for google. I included following loc, lastmod, changefreq. My site is in two languages. Do i need to add some other tags for specifying languages too
4
votes
3answers
166 views

How do you figure out a large site link/structure?

Suppose you're re-designing an existing large site. The first thing to do is to figure out how the site is structured, how many links, pages, sub domains, etc. Is there a tool you can use to crawl a ...
174
votes
19answers
25k views

The Sitemap Paradox

We use a sitemap on Stack Overflow, but I have mixed feelings about it. Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
4
votes
3answers
134 views

My archives are beating my current content, how do I stop from competing against myself?

I've inherited a site with about 15,000 pages going back 16 years. Often current events are outranked by previous events. What are the best techniques to make sure that I'm not fighting against ...
17
votes
6answers
4k views

Why did Google stop indexing pages from our sitemap.xml?

We're seeing some pages that exist in our sitemap.xml but are inexplicably missing from Google's public search index. You can't download http://superuser.com/sitemap.xml -- we protect this file ...
5
votes
6answers
399 views

Looking for a sitemap generator

I am looking for a good way to generate a sitemap for my current site. I inherited a huge site at work and we are going to start the process of IA analysis and redesign. It would be awesome if there ...
10
votes
5answers
3k views

&amp; in sitemap links, are they correct?

Simple question, I'm asking just to make sure. A Google sitemap generator generated a sitemap.txt fle with links written like this: http://www.domain.com/category.htm?name=some-name&amp;cat_id=8 ...
10
votes
1answer
301 views

using multiple sitemaps in a domain

I have a domain http://sub.example.com/. In it there is a wordpress blog at http://sub.example.com/blog and other more static content at http://sub.example.com/other, http://sub.example.com/stuff, ...
5
votes
2answers
883 views

Does sitemap.xml contain links to the entire site?

I read up on the limitations and how you can link multiple sitemaps. But my question is, are they to contain links to EVERYTHING or just the most recent X thousand changes?
1
vote
3answers
982 views

Where can I find a list of search engine crawler user agents and their domain names?

I'm trying to follow stackoverflow's approach of using a white list for the sitemap. I have tried to find an extensive list of search engine crawlers (and other crawlers) to use in my white list but ...
13
votes
4answers
367 views

easy tool for creating XML sitemaps? (constantly expanding site)

Does anyone recommend either a php script that I can run as a CRON or somekind of thirdparty tool i can use to create sitemaps? My website is events bases and so the pages are pretty ...
4
votes
2answers
319 views

Looking for ideas for a site map page on the website

We have a site with a lot of pages. We are in some of the final stages of redesigning the site and am looking for some examples of good site map designs we can be inspired by. Have you seen any good ...
2
votes
3answers
380 views

Priority of tags vs posts in a sitemap.xml file

I have a content based website and I'm in the process of creating a sitemap.xml file. I was wondering if it would be best to give high priority to tag urls or to post urls. Has anyone experimented ...
1
vote
1answer
597 views

Using Dynamic URLs with more than one Variable in a Sitemap?

I have a site map with generated dynamic urls that I need included in the sitemap. However, the sitemap seems to be generating an error on the & character, so that pages with more than one ...
13
votes
1answer
1k views

Should I include everything in the sitemap or only new content?

For a website with dynamic content (new content is constantly being added), should I only include the newest content in the sitemap or should I include everything (with a sitemap index)? What are the ...
3
votes
3answers
647 views

How do I specify the Google sitemap generator destination directory

Is there a way to specify the location of generated sitemap files using the lastest version of the Google Sitemap Generator? Version is: 1.0.0.0? (a very un-google version number, lol): > sudo ...
1
vote
2answers
317 views

How can I allow robots access to my sitemap, but prevent casual users from accessing it?

I am storing my sitemaps in my web folder. I want web crawlers (Googlebot etc) to be able to access the file, but I don't necessarily want all and sundry to have access to it. For example, this site ...
11
votes
3answers
2k views

How to create a good sitemap for dynamic website

I have a website with dynamic content and different kind of pages. I have some pages that rarely change, and I have pages like blogs that change often. The blog pages also have links for sorting, for ...
11
votes
6answers
1k views

Is it necessary/useful to build a sitemap for a small website?

I've created several small scale websites, usually for small businesses. They usually contain some information about the company, a contact form, and sometimes a few dynamic parts : news, pictures, ...
3
votes
2answers
353 views

Is there a way to add embedded youtube videos to my website sitemap?

The goal is to make embedded youtube videos appear in video search results as part of my website. I searched very extensively how to do this; I found through google some articles that claim that it's ...
7
votes
4answers
180 views

Sitemaps HTML and others [closed]

I am looking for the best resources on how to build sitemaps. I am currently aware of the obvious HTML sitemaps and the (google) sitemap.xml and (yahoo) urllist.txt Anyone know of resources for ...
3
votes
4answers
163 views

How are the sitemaps of big sites generated?

The question is related to my questions about things the filesystem of Wikipedia and generating sitemap for Wikipedia. It is more general. Problem: Suppose a site on MediaWiki with about 10,000 ...
5
votes
3answers
2k views

How long does it take for Google Webmasters to index site after submitting sitemap? [duplicate]

Possible Duplicate: Why isn't my website in Google search results? I have submitted my website today into Google search using Google Webmasters using sitemaps. The status on the ...