Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a website with dynamic content and different kind of pages. I have some pages that rarely change, and I have pages like blogs that change often. The blog pages also have links for sorting, for example sorting on date, asc, desc.

On some of the pages I also have links to different tabbed content, and links that are just anchor links.

Now when I use a xml sitemap generator then all the links are thrown into the site, and so I don't think all the links are really relevant.

The blogposts up until now are also taken into the sitemap. Is this really necessary? I think the links to the blogposts can be indexed just fine.

Is the best way to make a sitemap just to manually assign the main menu links to the sitemap, or is indexing everything really recommended?

share|improve this question
add comment

3 Answers 3

up vote 5 down vote accepted

You should use your sitemap to index everything. There's no downside to including every url, and there is a danger that at a large scale some urls will get missed.

In your sitemap, every url can have different set of data associated with it. Have a look at the Element Definitions and adjust the settings for the more important urls and less important

<url>
  <loc>http://www.example.com/about</loc>
  <lastmod> 2010-06-30</lastmod>
  <changefreq>monthly</changefreq>
  <priority>0.8</priority>
</url>

The values for <changefreq> are about how frequently a page may change. Adjust the value as needed,alwaysandhourly` show you think the page updates frequently.

  1. always
  2. hourly
  3. daily
  4. weekly
  5. monthly
  6. yearly
  7. never

And you can set the value for <priority>:

  • <priority>1.0</priority>
  • <priority>0.9</priority>
  • <priority>0.8</priority>
  • <priority>0.7</priority>
  • <priority>0.6</priority>
  • <priority>0.5</priority>
  • <priority>0.4</priority>
  • <priority>0.3</priority>
  • <priority>0.2</priority>
  • <priority>0.1</priority>
  • <priority>0.0</priority>

So for the pages you find more important, set items in order of priority. Very most important set to 1.0, important 0.9, regular importance 0.5 and not important 0.1 or 0.0. That should give you the amount of control you're looking for.

In short, there's no downside to keeping everything, assuming you have less than 50,000 urls.

share|improve this answer
    
Thank you for the answer and fixing the spelling in the question. One thing, If I have a website with about 3000 links, should I manually go through all the links and put in the priority, that will take me ages. –  Saif Bechan Jul 19 '10 at 16:33
    
If you're doing it by hand, I would put everything at 0.5, then put the very most important at 1.0. Then if you have time you can de-prioritize the least important. –  artlung Jul 19 '10 at 16:47
1  
Ok that makes sense. Thanks for the tips! –  Saif Bechan Jul 19 '10 at 16:48
add comment

You could consider A1 Sitemap Generator. It will calculate priority values based on internal linking. (Worth a try maybe. Otherwise just ignore the priority value.)

share|improve this answer
add comment

If you are using any decent blogging software then you should have an RSS feed, which is usable as a Sitemap and probably a better option. You can definitely submit RSS feeds as sitemaps to Google and Yahoo webmaster tools. Depending on your other content you may need a sitemap as well.

The blog pages also have links for sorting, for example sorting on date, asc, desc.

I would try to avoid the pages with different sort orders - they are essentially the same content with different views. Just make sure you have the main pages indexed - e.g. the list of blog posts, paginated.

The blogposts uptil now are also taken into the sitemap. Is this really necessary? I think the links to the blogposts can be indexed just fine.

You are right, search engines should be able to find your content just fine. XML sitemaps aren't always necessary. But it's a good way to get your content indexed a little more quickly, particularly if some pages are nested deep within your site, or it's a particularly new site.

share|improve this answer
1  
Some of the blog packages even have decent sitemap generation tools, like this WordPress plugin: wordpress.org/extend/plugins/google-sitemap-generator –  JasonBirch Jul 19 '10 at 8:05
    
Oh unfortunately I have a website that is build from scratch. But this looks like a nice feature to add, I did not know this was even possible with sitemaps. This way the sitemap is always up to date, that makes more sence than creating a new sitemap every time. –  Saif Bechan Jul 19 '10 at 16:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.