0

I run a simple companion website to my book. To count the hits on the website, I use a simple PHP counter that is triggered by this command

<img src="counter.php?Identifier=name" height="0" width="0" alt="" />

Interestingly, about three years ago I did a major overhaul of the website and changed the names of the identifiers, e.g. from name_old to name_new.

However, my PHP counter still receives name_old requests, which make up between 10% and 50% of all requests, depending on the part of the website. After three years!

The only explanation I have for this is that my website is backed up / cached / mirrored somewhere on the internet and that this years old backup / cache /mirror is still being accessed by part(s) of the worldwide web.

Can anyone confirm that these backups / caches / mirrors actually exist? Is there any way to find and fight them as they eat up significant part of the traffic and also show outdated information?

PS: I know that access log data would be helpful, but with cPanel you only have access to "visitors" via the admin panel, and this data is useless (for a few hours at best).

EDIT: I have changed the PHP script to intercept these requests, and here's what I have intercepted so far (HTTP referrer is always empty):

petalbot-114-119-XXX-XXX.petalsearch.com, Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537... (2x)
114.94.12.10, Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) G...
49.51.204.252, python-httpx/0.26.0 (2x)
ec2-47-128-XXX-XXX.ap-southeast-1.compute.amazonaws.com, Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.3... (2x)

1 Answer 1

2

Try navigating to those URL's and see if they come up.

The problem would not be mirroring as they would not hit the counter (because its a relative URL)

It could well be caching - It would seem that counter.php exists - so if Google knows that URL and keep checking it it will continue to produce a result. If this is the case, maybe you can modify the script so it returns a 404 for identifiers that don't exist - and maybe after some years Google will "get the hint". You should also update/create a robots.txt and maybe use Googles removal tool for the URL and put a noindex tag in the HTTP header of the script. Have a read of https://developers.google.com/search/docs/crawling-indexing/prevent-images-on-your-page

2
  • +1 Hello, thank you for the answer. I have amended to my question and maybe this will help to find the exact cause of the problem.
    – Pygmalion
    Commented Mar 7 at 11:55
  • 1
    Your edit confirms (based on IP addrrsses and browser strings) that these are bots who remember the old URL. Per my answer you want to 404 replies on the old output and exclude it in robots.txt.
    – davidgo
    Commented Mar 7 at 18:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.