5

Possible Duplicate:
StackOverflow and Google search

this is a question purely for learning purpose.

I have noticed that this site has an incredible power of appearing in search engines.
Questions as new as 2 days old appear on search results.

Can someone please advise me or point me towards resources on how to make a website this versatile towards SEO?

I did a brief search and came up with these:

1) Submit xml site map to search engines - this is really going to be a log, as there are tonnes of questions generated every day. If this is infact the way to go, how often should I submit the sitemap and where?

2) have each page contain it's own unique title and meta tags (desc, keywords) - How will I know what goes into each page to make key words out of it?

The main thing that escapes my brain is, how would each question (the title is different, so as far as the search engine is concerned, it's a new page) - how would I tell the engines that I have a new page?

Thanks.

Improve this question
3
  • 5
    I think the most important thing is being popular already, so google gives you high priority. Of course the site-map helps too. – CodesInChaos Aug 10 '11 at 8:49
  • 4
    I've seen several questions I've asked on SO appear in google after a few minutes. – mark Aug 10 '11 at 8:50
  • 1
    Search engines like Google will try to learn how often your site is updated to optimize its own crawling (e.g. to save bandwith/processing power, provide up-to-date results, etc.). Site maps are a general hint for the search engine, but it will still determine the number of updates etc. on its own. The sitemap xml file should be a generic starting point/hint for search engines; no complete index (depending on page complexity). – Mario Aug 10 '11 at 9:05
8

I think you're misunderstanding, to a certain extent, why Google indexes a site like StackOverflow so aggressively: It's not due so much to things the site is doing, but rather it's in Google's interest to spend time/money/resources as it's such a popular site!

There are things you can do to improve the searchability of your site, but google will only "use" those things, and expend precious crawling resources, if your site is deemed "important" based on the number of links to it, freshness of content, etc.

That said, those things that you can do include:

  • Having other (reliable/reputable) sites link to you, obviously - that's probably the most important thing that determines how much effort the google crawler will spend on you
  • Having a lot of content, and ensuring that Google always knows not to index duplicate content (use Google Webmaster Tools to tell it about duplicate versions of the same content, otherwise you get penalized)
  • Having "homepages" that change / illustrate new content as it comes up, and making sure that those are referenced in your site map: this is almost certainly how new questions appear so quickly in google results for stackoveflow - google knows where to look for them, and has an incentive to do so due to the popularity of the site.
Improve this answer
2

Although your website might have alot of aspx pages or web pages in general, as you said search engines today do few things to keep track of a web site.

First the user statistics learning which google/bing does. And then meta tags learning. (I know these 2 only, lemme know if you know more). So do give common tags for all your webpages plus few extra ones to different ones.

I also think having googleadservices linked to your web sites, it might help a bit.

Improve this answer
2

You're overcomplicating it.

Just add indexing pages and static pages to your site map. So, for a forum, instead of submitting each thread, just submit the first page with the most recent threads and set it's priority and frequency to high (i.e. make it scanned more often), then (if you want) add older pages with lower priority (the older the less checks). Just think about it that way: The more entries in your sitemap, the more entries to pick for the engine, the lower the frequency these are checked. Instead set priorities (archived indexes/threads/questions don't have to be updated that often leaving spider visits for the new content).

Meta tags are rather outdated these days as it's too easy to use meta tags to cheat to get people opening empty or the wrong pages (which is sometimes also part of SEO, but nothing you should try to do as it WILL damage your page reputation in popular search engines). Skip the keywords meta field and add a generic page description describing your page (as most search engines will use it to describe the search result of your page if there aren't any direct text matches in the page content).

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.