I have a proxy at http://rahul2001.com/proxy/
I site can be 'proxied' by like this:
http://rahul2001.com/proxy/proxy.php/http://site-to-be-proxied.example.com/
The problem is that Google seems to be crawling pages through my proxy :/
[![uh oh..][1]][1]
I do NOT link a proxied version of Yahoo food or Google Careers or Google Search Hindi Help, but these still turn up in the search results...
PROBLEMS:
-I DO NOT want to block my website/proxy from search engines entirely, I still want the proxy service itself to be able to be located in search.
-I DO NOT want Google to use up my bandwidth by crawling useless sites.
-I DO NOT want to use captcha since a few of my apps use this proxy.
-I DO NOT want Google to spoil the search results of my website in this manner.
What do I do??
ALSO, why is Google entering random URLs into the form??
EDIT After adding the meta tag, I get an error :(
proxy.php (first few lines):
<head><meta name="robots" content="noindex, nofollow" />
</head>
<?php
/*
miniProxy - A simple PHP web proxy. <https://github.com/joshdick/miniProxy>
Written and maintained by Joshua Dick <http://joshdick.net>.
miniProxy is licensed under the GNU GPL v3 <http://www.gnu.org/licenses/gpl.html>.
*/
Error:
Warning: Cannot modify header information - headers already sent by (output started at /home/rahulcom/public_html/proxy/proxy.php:3) in /home/rahulcom/public_html/proxy/proxy.php
