How can I prevent Googlebot from indexing web service URL's?

Question

I have a problem with googlebot and bingbot accessing our web service. We have a search application created in aspx that is integrated into a drupal website. Users perform searches based on postal code or city through drupal and search query is outputted through aspx. The problem is that google/bing keeps indexing the aspx returned query pages and then we receive 404 errors.

How can I prevent googlebot from indexing the web service. We are on IIS 7.5. Do we setup a rule on the server? I have tried blocking it in robots.txt but am unsure if I have the correct syntax.

Disallow: /*?
User-agent: Googlebot
Noindex: /office/default.aspx
Noindex: /?keyword
Noindex: /el​ectronics/locationdetails.aspx?
Noindex: /electronics/communitydetails.aspx?
Noindex: /electronics/Communitylist.aspx?
Noindex: /office/communitysearch.aspx?
Noindex: /Electronics/Communitylist.aspx?
Noindex: /Office/LocationDetails.aspx?

Answer 1

Your robots.txt file does not appear to conform to the robots.txt specification (you must specify the user agent before the directives and I do not believe that Noindex is a valid directive).

Consider the following updates:

User-agent: googlebot
Disallow: /Office/LocationDetails.aspx
Disallow: /office/default.aspx
Disallow: /Electronics/Communitylist.aspx
Disallow: /electronics/locationdetails.aspx
Disallow: /electronics/Communitydetails.aspx
Disallow: /electronics/communitydetails.aspx

... alternatively, you could configure your webserver (or application) to add noindex/none X-Robots-Tag headers where appropriate as described in detail at Google's Robots meta tag and X-Robots-Tag HTTP header specifications document.

Answer 2

Use "NoINDEX, NOFOLLOW" Meta Robots tag in each pages under the head tag or certain directives under robots.txt will do the job for you.