Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

The company I work for has a content management system that builds pages automatically for certain content.

E.g., There is a page in our system that has the ability to show you information about any composer in our database, depending on the input parameters and what information we have on them. However, every now and then, there is a composer that hasn't got any information, thus the page renders a blank page. How can we tell Google not to index the page at that moment, but maybe sometime in the future?

Because the content is auto-generated, we can't just block pages from being visible, as it information about that composer may pop up at any moment.

Edit

Please note that our CMS system that does the fetching is seriously complicated and deep. The content part of the page WILL return some sort of data, so running an empty($content) ? xx : yy as has been suggested is not a quick fix. If a composer does not exist, the system won't have a page for that the composer, if the composer does exist, the system will. When that composer's page is rendered, the system searches every type referring to composers, such as works, to create a dynamic page.

share|improve this question
2  
What exactly do you mean by "blank page"? Do you mean literally no HTML is sent? Or you have the basic 'shell' design but with no content? –  DisgruntledGoat Jun 26 at 12:12

4 Answers 4

up vote 13 down vote accepted

If there is no content for a particular composer, use the meta tag:

<meta name="robots" content="noindex, follow">

As soon as content is added and the meta tag is removed, Google will index it.

Example:

<?php if(empty($composerInfoArray)): ?>
    <meta name="robots" content="noindex, follow">
<?php endif; ?>
share|improve this answer
    
FWIW "follow" is the default behavior, you can drop it if you want (it doesn't cause any problems, and maybe it's meant more for humans reading it, so you can leave it too). –  John Mueller Jul 1 at 20:47

If the "missing" pages indeed contain no actual information, but are simply placeholders for "there may be a page here in the future", then I would recommend configuring your web server to return the 404 Not Found status code for them.

Browsers will still show such pages to the user, just as they show normal 404 error pages (at least as long as they're long enough), but search engines will simply treat the page as if it didn't exist at all.

This is the method used e.g. by Wikipedia for non-existent pages like this one. One of its advantages (besides it being, arguably, semantically correct) is that it guarantees that search engines will treat such pages exactly the same way as they normally treat missing pages that return a 404 status code.

One potential disadvantage is that, if you have links to pages that return a 404 status code, they'll show up as "404 errors" e.g. in Google's Webmaster Tools. However, this is perfectly normal, and not something to worry about.


To further help search engines discover your new pages, when they do get some actual content and stop returning 404s, you should also maintain a regularly updated XML sitemap listing all the pages on your site that do have content. This allows Google and other search engines to discover new pages directly via the site map, instead of having to randomly come across them while recrawling your site.

Ps. See also: Not index page that doesn't have relevant content?

share|improve this answer
    
Karenon, returning a 404 is equal to doing nothing, that's regular web server's behaviour; Wikipedia's example is just a custom 404 page. –  Binarysurf Jun 26 at 13:55
1  
@Binarysurf: According to the OP, they're using a custom script to serve dynamic content for "virtual pages" constructed out of information stored in a database (just like Wikipedia, SE and most major websites do nowadays). This does mean that "regular web server behavior" doesn't apply -- as far as the web server (Apache, IIS, nginx, etc.) is concerned, the content-generating script exists and runs, so it defaults to a "200 OK" response, unless the script tells it otherwise. I'm just suggesting that the OP should make their script send a 404 response, if there's no actual content to show. –  Ilmari Karonen Jun 26 at 14:04
    
there is a custom script retrieving composer data from database, what if somebody's name is not in there? using logic, the script will get no records from database, what should it do? maybe it will then send 404 not found. If the script found something, maybe just metadata (he's name) and no other data, it will send 200 and a page with empty content. Even if he has a custom script he will follow default web server's behaviour?, or will it respond 200 for every single request? It will look spammy with no specific content focus. –  Binarysurf Jun 26 at 15:45
    
Once you're running a script, the script is responsible for sending the status code, and the correct code to send when a query has no result is 404. –  Wossname Jun 27 at 2:49

I would personally build a message into the system that would let the user know that while their request was completed, there were no composers found for the parameters given. This will keep the user from thinking that the page/link is "broken".

share|improve this answer

If the page has no content, your CMS system should not be linking to it. Linking to blank pages is bad for both users and search engines.

I like Wayne Whitty's suggestion about how to put the meta tag on the composer page itself. You should find a way to do something similar in the place where you list all the composers and link to them:

<?php foreach ($composers as $composer) {
    if ($composer->countCount > 0)
         print "<li><a href=\"$composer->url\">$composer->name</a></li>";
    } 
} ?>

If you do this, then search engines will no longer find the blank pages to begin with.

share|improve this answer
1  
...that is, unless there's something meaningful that users could do even with a "blank" page, such as supplying information for it. (Even then, though, it may be better not to link to the blank page directly, but to some different URL that indicates the (possible) absence of data. Note that you'll need to deal with the edge case where a user follows such a link after the page has recently been created; see for example how Wikipedia does it.) –  Ilmari Karonen Jun 27 at 12:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.