1

We have a website dedicated to our ActiveX and .NET components that has been working for more than 12 years. The site consists of 2 parts: the static part (product descriptions and articles) and the dynamic part (forum dedicated to our tools).

Google indexes the static part without any problems - all the pages are found due to a good textual hyperlink menu system. These pages also have good positions in SERPs.

However, many SEO pro's insist that we need to have a sitemap.xml file to enhance the positions of our website in the "eyes" of Google or other search engines like Bing. My 2 questions related to this are the followings:

1) Do we really need sitemap.xml for our website?

2) If so, how to code the dynamic forum pages in it?

  • Do you have submitted your site to google webmaster? – Helping Hands Dec 26 '14 at 11:25
  • @HelpingHands, sure - many years ago. – TecMan Dec 26 '14 at 13:30
  • 1
    What many SEO Pro's are you talking about? Sitemaps have nothing to do with search engine result page (SERP) placement- no boost at all. I repeat. None. They are only used for sites that are difficult to index. If search engines can follow your links okay and your site is not huge, then there is likely no need for a sitemap at all. In fact, for most medium and smaller sites, the sitemap is totally ignored by search engines. – closetnoc Dec 26 '14 at 16:22
  • @closetnoc, why don't write this comment as an answer? – TecMan Dec 26 '14 at 17:11
1

It might surprise people exactly how old-school search engines are. Of course they have adapted well to all the new technologies such as AJAX and what not. However, much of what has always existed in search engines still exists and is the preferred method.

As far as sitemaps, they are intended only to inform search engines of what pages your site may have. Generally speaking, sitemaps benefit those sites that are very large, cannot or do not link all pages directly, have a barrier such as a pay-wall or login, or where not all pages can be found easily via the linking method used.

Search engines generally prefer to spider any site the old fashioned way. However, if a sitemap is supplied, search engines will compare the sitemap to the pages they can already capture without a sitemap. If the two lists are the same or end up being the same, search engines will prefer to spider and index any site by following links. Part of the reason for this is rather simple. Links signal importance and sitemaps remove this signal. It is quite common that other than reading the sitemap for comparison, search engines will ignore a sitemap. One common exception is for very large sites.

If your dynamic content is available via normal linking, then a sitemap may not be necessary. There does not seem to be a consensus as to how large a site has to be for a site map to be convenient. It never hurts to supply a sitemap. But for a site (and I am assuming something here) about 100k or 200k pages or so, the sitemap may not be necessary. I did not supply a sitemap for a site until it reached about 300+k pages and even then it was not necessary until I stopped linking to all the pages directly.

I wrote my own code for this since I wrote my own CMS. Many CMS and Blogging software will provide options for creating sitemaps. I am not familiar with these options. Sorry. There are also other external spiders that will generate a sitemap for your site. The only one I am familiar with is GSiteCrawler. I used it only once or twice years ago and it worked very well.

If you are familiar with writing your own code, you can follow the protocol found here: http://www.sitemaps.org/protocol.html. This is a very simple thing to do. I used this site originally to create my own sitemap.

There is no search engine metric for a sitemap that gives an advantage in the search engine result page(s) SERPs. There may be a residual advantage if your site is difficult to crawl.

Again, it never hurts to supply a sitemap. Especially if you are not sure there is an advantage for the search engines. There is always a gray area if your site is somewhat large or you are not sure that search engines can follow your links okay. I say go for it just to be safe. Webmasters will worry about providing a sitemap that often is not necessary. Especially if the site is moderate in size or smaller and can be crawled easily. Again, if it helps you sleep at night, I say go for it. Otherwise, my general advice is not to worry too much unless your site is very large or cannot be crawled easily.

| improve this answer | |
  • Awesome answer and you have removed wrong impression of sitemap which was in my mind. Thanks. Great answer. – Helping Hands Dec 27 '14 at 5:15
  • Excellent answer!! The only thing that bothers me after reading it is the following: "Links signal importance and sitemaps remove this signal." Can it screw up my site position in SERP if I provide sitemap.xml only for the static part of my site? FYI: our static part consists of about 70 pages, and the dynamic part (i.e. forum) has 800-900 pages, and this number is constantly increasing - but not very fast. – TecMan Dec 30 '14 at 17:00
  • Not really. I have about 600,000 pages that I cannot link easily and exist only in the sitemap. They perform rather well considering the tough search market I am in. If you can link a page, then you should. For example, important pages or new pages. You can link these from the home page for a while if that works. This will signal to the search engines that these pages are important. But if you cannot link all of the pages, then that is well understood by search engines. It is fairly common that pages exist in sitemaps only. – closetnoc Dec 30 '14 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.