Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 2 down vote favorite

If I want to make an XML sitemap for my site, do I need to map every forum topic in my forum as a separate URL?

If this would help, how do you deal with the hugeness of the sitemap for forums with over 100,000 urls?

Or is it just meant to map entry pages to the rest of your content? And can you semantically nest URLS?

share|improve this question

3 Answers

up vote 6 down vote accepted

If you want to make a proper sitemap that gets automatically updated please use http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156184&from=40318&rd=1 as your starting point.

Google offers the possibility to automatically create a sitemap and update this based on the page visits. You will have to read the instructies carefully.

If this is not what you are looking for you might wat to consider creating a XML sitemap using an online tool, such as: xml-sitemaps.com

The idea of a sitemap is to have this point to all pages. You can, however point out the importance of each page using the priority tag.

Please note that a XML sitemap is for you to submit to the search engines and and HTML sitemap is to be viewed by your website visitors. Its recommended to have not more than 100 links per page.

share|improve this answer
Thanks for answer, but what if my site has say 100,000+ pages on it, do I need multiple sitemap files? Or one huge one? – Tom Gullen Mar 7 '11 at 11:52
Check: google.com/support/webmasters/bin/answer.py?answer=183668#1 So do not exceed 50K pages. If your website has 100K+ pages you should break it up in multiple sitemaps. – Goboozo Mar 7 '11 at 11:56
up vote 1 down vote

@Tom For a forum I would make sure to filter off (both in analysis and output filters) e.g. user profile pages and similar. Only include real discussion topics/pages.

@Gaurav The max URLs per XML sitemap file is 50,000 URLs / 10 MB (but some tools may default to less per sitemap file) You can configure tools like A1 Sitemap Generator to parse through forms and most Javascript. You can also add multiple start search paths and combine it with a variety of filters. So I disagree 3rd party apps are bound to miss URLs :)

share|improve this answer
up vote 0 down vote

Third party apps basically rely on page crawling for generating sitemaps. They are bound to miss out a good chunk of URL's. I'll suggest that you code up a small script which generates an XML sitemap by iterating over your CMS's database. If you have more than 10,000 URL's, you'll need to generate partial sitemaps and link them up via an index sitemap.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.