I developed a site nearly 2 years (PHP, JAVA), but I am a new SEOer.

My database has almost 1M articles. I have some questions I need help with:

  1. Is it as a good idea to create static pages from all the articles?
  2. Are more pages in the one site better for SEO?
  3. If I created each article into a static page, that could be many GBs. How do I let the search engine crawler collect my database?

It doesn't matter where your pages come from (e.g. database / static).

You just have to make sure that all your pages can be accessed by following links on your site (this is how search engines crawl your site).

Either by adding the url's to a menu (not really the way to go with 1M pages :-) ) or by links at some other place on your site.

To 'help' search engines index your pages you could add an HTML sitemap (basically a page with your most important links on it) and/or an XML sitemap.

An XML sitemap will contain the links to the pages in a specific format.

You should save the file sitemap.xml in the root of your website.

Example of contents of sitemap.xml:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  • as you say, it is impossible write 1M pages into menus, and hard to write them all into sitemap, however if there have few new articles insert into database, I shall rewrite my sitemap each day with a huge cronjob... this make me really headache. – cj333 Nov 30 '11 at 22:38
  • If you all your articles are accessible by just navigating through your website search engines will find them. – PeeHaa Nov 30 '11 at 22:44
  • however, now I just query some articles on my front page for a review, and I will changed the hot words for the page every day. Then when the people like the article, each click make a sql query for a full article read. – cj333 Nov 30 '11 at 22:51
  • Do you use permalinks? I.e. the articles will be accessible by the same URL 'forever'. – PeeHaa Nov 30 '11 at 22:52
  • I am not sure, In my full article read page, the url is contains the article id, and I also make a mysql query, put the article's title in to the page's head meta part. but if no query event, this page is nothing. – cj333 Nov 30 '11 at 23:25

Search engines don't know if articles comes from a database or not. They only see the HTML a URL produces. So if you have one PHP/Java file that gets each article from the database using a unique URL (i.e. http://example.com?id=12345 with 12345 being the ID of the article) then you will have a website that has 1 million pages. Each of those pages will be ranked on its own merit (quality of content, semantic markup, link popularity, etc).

  • so do you think, it is not necessary to created them all in the static pages, just make a URL rules. but where can I stored these url rules? all in the XML sitemap? In the menu links? Or other wise way? Thanks. – cj333 Nov 30 '11 at 22:34
  • If you make an XML sitemap you'll need to create a couple of them and then use an index file to submit to the search engines. You should also have some way for those pages to be found through HTML or else users will never be able to find them. A dynamic sitemap seems like a good way to go with this as 1 million pages is way to much to list on one page. – John Conde Nov 30 '11 at 23:26
  • thanks for suggestion. as I know, one sub-sitemap only support few thousands url links, so 1M links should divide into thousands sub-sitemap, so do you have any good idea how to generate sitemap quickly and easily? Another way, how to search an index file for big site in English from google? I need more articles for reference, Thanks again. – cj333 Dec 1 '11 at 9:57
  • 2
    You can put 50000 URLs in one site map. So you would have one index file with 200 XML sitemaps included it in. – John Conde Dec 2 '11 at 12:32

I will address your points separately.

Is it as a good idea to create static pages from all the articles?

No, it is better to create a single endpoint (e.g. a PHP script) which retrieves the article from the database and displays it as HTML. Of course, it is always a good idea to implement some caching mechanism to reduce hitting the database over and over for popular articles.

However, creating a static page for all articles is unecessary as they will be stored in two places this way - in the database and in static HTML files.

Are more pages in the one site better for SEO?

I am guessing by this you mean: is it better to have 1 million articles in a single site or split them in multiple sites?

The simplest answer is - it depends. If your articles are with mixed topics or you can differentiate them in categories (e.g. 10000 articles per category), it might be a wise idea to split each category in its own website and try to rank the sites on their own, to gain better SE rankings for different long-tail keywords/niches. However, if your goal is to build a general article directory, it might be better to include all 1 million articles in one site. Naturally, you would still want to separate them in topics or separate pages, so that search engines and real human visitors may better navigate and find relevant content.

There is no single solution that fits it all.

How do I let the search engine crawler collect my database?

You can't just upload your database to the search engines, but you may make it easier for them to crawl the pages on your site. By checking your site's content the crawlers determinate if that page is worth including in the SERPs (search engine result pages) for a given keyword.

You should know, that SEs don't just use quantitive factors when ranking pages (e.g. if your page will be the first result or 1000th), but qualititive as well - how old your site is, its PageRank, authority (how much incoming links you have from other websites and their quality).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.