A site map (or sitemap) is a list of pages of a web site accessible to crawlers or users.

learn more… | top users | synonyms

2
votes
1answer
100 views

Do or don't submit the comment-feed of your blog?

A commentsfeed (of a blog) might serve perfectly as a commentssitemap. I'm wondering if it's a good idea to submit this feed to Google's webmaster tools. What are the pros & cons?
4
votes
1answer
2k views

Sitemap change frequency

I'm sitemapping all my forum topics. It works great: <url> <loc>http://127.0.0.1/scirranew/Forum/Breaks-and-slowdowns-Vram-200_topic32854.html</loc> ...
1
vote
3answers
312 views

xml sitemap structure

I have been arguing a point and would like a second opinion. The basic point of contention is as follows: We have a site X with a sitemap index file located at: http://www.x.co.uk/sitemap.xml ...
2
votes
2answers
2k views

Free unlimited-page sitemap generator? [duplicate]

Possible Duplicate: Looking for a sitemap generator I have been using www.xml-sitemaps.com/ but their limit is 500 pages. My site has grown beyond that and I am wondering whether there is ...
2
votes
1answer
318 views

Difference between robots.txt and sitemaps index file

Do I need a sitemap index file if I have multiple sitemaps? At the moment I just specify multiple sitemaps in my robots.txt, is the equal to having the index file?
2
votes
3answers
157 views

XML Sitemaps, what should they link to?

If I want to make an XML sitemap for my site, do I need to map every forum topic in my forum as a separate URL? If this would help, how do you deal with the hugeness of the sitemap for forums with ...
3
votes
3answers
602 views

What are some client-side sitemap generators?

Most of the sitemap generators I've found all scan my internal files and base it on that. However using apache and htaccess I have many aliases for things for example: /brand/model/photos/ is empty ...
1
vote
3answers
722 views

Does anyone have statistics on percentage of people that use search feature on a website as opposed to the navigation bar

I was interested in knowing percentage of people that use search feature on a website as opposed to the navigation bar. Has search bar become more powerful than the traditional navigation bar for ...
1
vote
1answer
1k views

is it ok to have 2 sitemaps on 1 website?

Do I have to have a sitemap page on my index page for bots to read it or can I just have it anywhere on my server? I have a phpbb/wordpress integration and I need 2 sitemaps mods for each one (or I ...
0
votes
1answer
354 views

webmaster tools - Network Unreachable

webmaster tools for my site displays that robots.txt unreachable and for all links in sitemap it says network unreachable. sitemap.xml unreachable. These appear in crawl stats page. I discussed ...
1
vote
0answers
130 views

How does Stack Overflow have questions appear in Google so quickly? [duplicate]

Possible Duplicate: StackOverflow and Google search I have noticed while asking questions on Stack Overflow that they appear in Google almost instantly. How does SO do this? Is it a feed ...
7
votes
2answers
2k views

Google doesn't seem to update the description or title of my homepage

Before we launched our website, we had set up a "coming soon" page and google picked up the title and description from its contents. So the description in the search results said, "Coming soon! Visit ...
0
votes
1answer
193 views

Sitemap.xml.gz. can I have a compressed file with more than one sitemap in it?

I've got quite a few sitemaps as my site is quite large. Is there any way that I can have a compressed with with multiple sitemap files in it ? I've googled but can't find the answer to this
2
votes
3answers
241 views

Where should sitemap be? (not separate sitemap file)

Most of the big companies got included sitemap at the bottom of thier site. e.g. Piriform / PCTools. I would like to know if there's reason why to put the sitemap above the copyright or they just put ...
1
vote
3answers
569 views

YAHOO and BING support for Index, Image and Mobile sitemaps

I know Google webmaster supports submitting Image, mobile, video and other types of sitemaps. YAHOO also mentions about mobile site map here. But does it support Image and video sitemaps. I could not ...
0
votes
0answers
104 views

Is there a way that is more efficient than sitemap to add/force recrawl/remove your website's index entries in google? [duplicate]

Possible Duplicate: StackOverflow and Google search I'll make it more simple: Right now I am posting this question, after a minute or two... may be it will appear on google search query: ...
8
votes
2answers
567 views

Have search engines recognise multi-lingual sites

I have organised my site by language using the following structure: www.domain.com #spanish language version homepage www.domain.com/en #english language version homepage The site is of Spanish ...
1
vote
1answer
253 views

How Does One Make a Sitemap for a Flex Website?

I've got a Flex website and I'm not sure how to make a sitemap for it. For a standard HTML page, sitemaps are straight forward. As Flex sites use a "pageless" architecture, I'm not sure what I should ...
1
vote
2answers
660 views

Yahoo sitemap validation

I am trying to submit sitemap.xml (Index) to Yahoo Site Explorer but with no luck. I tried using website feed option in the site explorer to submit the sitemap, but I got validation errors. However, ...
0
votes
1answer
86 views

is there anything wrong with my sitemap.xml

I had written an article over Highly searched keywords some days ago. When i search it in google using top searched keywords article-stack, it returns many results from my site. But none of them was ...
3
votes
1answer
1k views

Should I include PDF files in my sitemap.xml

Should I add a direct link to PDF files in my (xml) sitemap?
2
votes
1answer
184 views

Sitemap contains a lot of products and web index contains a few according to Google webmaster tools

Our sitemap has something like 10 million pages: This is the data in the webmaster tools: Submitted URLs 10,873,781 2 URLs in web index. This is a big difference and an incorrect statement, since ...
0
votes
3answers
663 views

Tumblr and Sitemaps

I was wondering if anyone knew how to add a sitemap to a tumblr blog. It was what i use to share stuff online. Since tumblr doesn't let you upload to example.com/sitemap So there lies my ...
2
votes
1answer
162 views

Validating sitemap with Yahoo Explorer

I have a sitemap index on my website, which I successfully validated with "Google webmasters tools". The declarations at the top are: <sitemapindex ...
4
votes
2answers
453 views

sitemap that contains millions of links/URLs

I am currently working on a shopping / auction website. The involving party who's doing SEO consultancy is insisting to generate URLs for every single combination of products listed on the website, ...
0
votes
1answer
107 views

SItemap for site with translations

I have a domain.com which I translated all of the pages in my domain to several languages, all on their own subdomain (e.g. es.domain.com, cn.domain.com, ru.domain.com ...). The top level domain ...
1
vote
2answers
306 views

Validating sitemaps with images tag

I am trying to add the <image> tag to my sitemap, but even trying to look at the XSD , as mentioned by Google at http://www.google.com/support/webmasters/bin/answer.py?answer=183668 , I get 404 ...
8
votes
2answers
384 views

Dynamic sitemap file

Does it matter if I serve sitemap.xml as a dynamic page? e.g., use .htaccess to rewrite sitemap.xml (or whatever static URL I submitted at the Webmaster Tools) to a PHP file. The PHP file will call ...
2
votes
2answers
175 views

Sitemaps index submission

I have two questions regarding site maps: 1) Currently my site has around 5,000 pages. However, in the future it will contain more than 50,000 pages. Should I start using sitemap-index with only one ...
5
votes
1answer
516 views

Sitemap XML version vs simple text (urls list) version, do you know what are the pros?

I usualy submit simple text urls list as sitemap on Google webmaster tools. It's just a matter of simplicity, a simple text file is lightweight, easier to read also for humans (actually for me). :) ...
1
vote
1answer
86 views

What are Sitemap content tags?

i have recently generated sitemap for google. I included following loc, lastmod, changefreq. My site is in two languages. Do i need to add some other tags for specifying languages too
4
votes
3answers
166 views

How do you figure out a large site link/structure?

Suppose you're re-designing an existing large site. The first thing to do is to figure out how the site is structured, how many links, pages, sub domains, etc. Is there a tool you can use to crawl a ...
177
votes
19answers
26k views

The Sitemap Paradox

We use a sitemap on Stack Overflow, but I have mixed feelings about it. Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow ...
4
votes
3answers
134 views

My archives are beating my current content, how do I stop from competing against myself?

I've inherited a site with about 15,000 pages going back 16 years. Often current events are outranked by previous events. What are the best techniques to make sure that I'm not fighting against ...
17
votes
6answers
4k views

Why did Google stop indexing pages from our sitemap.xml?

We're seeing some pages that exist in our sitemap.xml but are inexplicably missing from Google's public search index. You can't download http://superuser.com/sitemap.xml -- we protect this file ...
5
votes
6answers
403 views

Looking for a sitemap generator

I am looking for a good way to generate a sitemap for my current site. I inherited a huge site at work and we are going to start the process of IA analysis and redesign. It would be awesome if there ...
10
votes
5answers
3k views

&amp; in sitemap links, are they correct?

Simple question, I'm asking just to make sure. A Google sitemap generator generated a sitemap.txt fle with links written like this: http://www.domain.com/category.htm?name=some-name&amp;cat_id=8 ...
10
votes
1answer
303 views

using multiple sitemaps in a domain

I have a domain http://sub.example.com/. In it there is a wordpress blog at http://sub.example.com/blog and other more static content at http://sub.example.com/other, http://sub.example.com/stuff, ...
5
votes
2answers
887 views

Does sitemap.xml contain links to the entire site?

I read up on the limitations and how you can link multiple sitemaps. But my question is, are they to contain links to EVERYTHING or just the most recent X thousand changes?
1
vote
3answers
993 views

Where can I find a list of search engine crawler user agents and their domain names?

I'm trying to follow stackoverflow's approach of using a white list for the sitemap. I have tried to find an extensive list of search engine crawlers (and other crawlers) to use in my white list but ...
13
votes
4answers
367 views

easy tool for creating XML sitemaps? (constantly expanding site)

Does anyone recommend either a php script that I can run as a CRON or somekind of thirdparty tool i can use to create sitemaps? My website is events bases and so the pages are pretty ...
4
votes
2answers
328 views

Looking for ideas for a site map page on the website

We have a site with a lot of pages. We are in some of the final stages of redesigning the site and am looking for some examples of good site map designs we can be inspired by. Have you seen any good ...
2
votes
3answers
386 views

Priority of tags vs posts in a sitemap.xml file

I have a content based website and I'm in the process of creating a sitemap.xml file. I was wondering if it would be best to give high priority to tag urls or to post urls. Has anyone experimented ...
1
vote
1answer
598 views

Using Dynamic URLs with more than one Variable in a Sitemap?

I have a site map with generated dynamic urls that I need included in the sitemap. However, the sitemap seems to be generating an error on the & character, so that pages with more than one ...
13
votes
1answer
1k views

Should I include everything in the sitemap or only new content?

For a website with dynamic content (new content is constantly being added), should I only include the newest content in the sitemap or should I include everything (with a sitemap index)? What are the ...
3
votes
3answers
653 views

How do I specify the Google sitemap generator destination directory

Is there a way to specify the location of generated sitemap files using the lastest version of the Google Sitemap Generator? Version is: 1.0.0.0? (a very un-google version number, lol): > sudo ...
1
vote
2answers
320 views

How can I allow robots access to my sitemap, but prevent casual users from accessing it?

I am storing my sitemaps in my web folder. I want web crawlers (Googlebot etc) to be able to access the file, but I don't necessarily want all and sundry to have access to it. For example, this site ...
11
votes
3answers
2k views

How to create a good sitemap for dynamic website

I have a website with dynamic content and different kind of pages. I have some pages that rarely change, and I have pages like blogs that change often. The blog pages also have links for sorting, for ...
11
votes
6answers
1k views

Is it necessary/useful to build a sitemap for a small website?

I've created several small scale websites, usually for small businesses. They usually contain some information about the company, a contact form, and sometimes a few dynamic parts : news, pictures, ...
3
votes
2answers
353 views

Is there a way to add embedded youtube videos to my website sitemap?

The goal is to make embedded youtube videos appear in video search results as part of my website. I searched very extensively how to do this; I found through google some articles that claim that it's ...