Will indexing an underdeveloped website result in a Google penalty?

Question

My website is still undergoing multiple changes. When I search my website's indexed pages, I find many unwanted pages in search results. There are still many categories that need to be uploaded.

With as many pages as there are without content, is it possible that Google might slap a penalty on the site? Or is there a case that as soon as we complete the development and missing content, and then Fetch the pages as Googlebot and submit them for indexing, that everything will return to normal?

In short: should I not index my website until it is complete?

Answer 1

I'm asked this question a lot by customers.

I don't recommend or see a reason to block the google bots completely. You could use a robots.txt file to list and stop them indexing the parts of the site that are under development and are not content rich.

I've not come across any form of penalty for having part of the site under development and still being indexed by mistake, as all sites on the internet change in some way regularly.

You want to make sure the content that can be found is relevant/high quality and relates or serves a purpose to the visitor.

It sounds like you've got the right idea but fetching as google when you've completed an area of the site to push an update through a bit quicker.

Google luck with your development!

Answer 2

The "penalty" is that you are going to have pages, that you don't want indexed, indexed by the search engines. And this could reflect badly on you and your business. It could also expose security vulnerabilities in your site as it's being developed.

When the actual content is live, Google won't suddenly update its index. This could take days, weeks, ... Particularly since your site is new and probably isn't being crawled so often.

The site should be at least "presentable" before you open it up to the world.