3

On one of our websites, Google has been indexing something it wasn't supposed to. We fixed the problem so it shouldn't happen anymore, but are interested in requesting that Google de-index these pages.

The problem is that there are about 10,000 pages. They all look similar to this:

http://www.mysite.com/file.php?o=34995
http://www.mysite.com/file.php?o=4566
http://www.mysite.com/file.php?o=223af
http://www.mysite.com/file.php?o=6ga3h
http://www.mysite.com/file.php?o=sfh45a
etc...

All the pages are file.php with get parameters like above. Is it possible to put in a de-index request like: http://www.mysite.com/file.php* so that Google removes all 10,000 pages?

  • Are the files public? Or meant for members of the site? If for members you may want to add some kind of authentication to who can access those URLs. What happens if I link to http://www.mysite.com/file.php?o=6ga3h from my website? Now Google has a valid link to crawl and access that file. – Anagio Oct 20 '12 at 6:22
2

You can't. The removal request tool is intended for emergencies, not clean-up. What you should be doing is informing the crawler(when it comes back around) that those URLs are no longer good. The engines will then do the clean-up themselves. This can be done by sending something like a 410 Gone response for the URLs if they really just plain shouldn't exist. If the problem is that the URLs have changed(eg. there were multiple ways of accessing those documents), you should look at their canonicalization suggestions. Eventually, the "true" URLs will be the only ones indexed and you can get rid of those 301s or whatever.

| improve this answer | |
-2

Put in your robots.txt

Disallow: */file.php?o=

| improve this answer | |
  • 1
    This won't take the pages out of the index quickly. – paulmorriss Sep 20 '12 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.