How do I identify and block a bot from my site?

Question

In Google Analytics I can see that I started getting traffic to one page on my site, which is /search. It is as if a bot is visiting the site, entering "" in the search bar and pressing enter to bring up all products on the site and then moving on to Page 2, Page 3, etc.

It started on 21st September, going from 0 to 500+ hits per day. I am trying to identify and block the bot from the site. I cannot identify it from within Google Analytics. It says it comes to the site from different cities and via different sources and devices.

I have asked web developer to see if he can use server logs to identify it but he says he is unable.

Anyone had this before?

Answer 1

There is no simple solution, and it depends on the source of traffic - and indeed (from apractical POV) may not be possible on the scale you are talking about.

Very often the browser identifier will provide clues. If its a legitimate or semi-legitimate bot it should set a unique and meaningful browser string.

IP address is another one, but from your question I'm guessing this is not going to help - although sometimes doing analytics will reveal a handful of locations. There are also lists which you can compare compromised IP addresses against. Depending on your site and the bit, you may be able to additionally work out what the bots are by whether they call elements in the page like CSS and images. Conversely you may be able to embed hidden links which identify a not if followed.