I noticed something strange in my server logs today. Several visitors were accessing the same pages with the same parameters at (basically) the same time.

I looked into it and found one that opened 29 different pages which was more than the 3-5 of everyone else. But they were too on an old version of Chrome, and most importantly none of the page hits indicated from what page they were navigating from (referer).

I left it at that, until I found another visitor accessing these same pages with even more of a history. I'll call them Visitor A. They first came to my website in August via a search engine and returned occassionally. Every single interaction they had with the site looked perfectly normal, actually entering some text into search boxes, following existing links, and the HTTP referer was always set. That includes today. Visitor A accessed the web site and interacted with it normally.

Time        Visitor     Page            Referer
03:03:00    Visitor A   index
03:07:10    Visitor A   view?id=2115    index
03:25:15    Visitor A   search          view?id=2115

Soon, 3-5 bots started accessing some of the pages that Visitor A was on ~5 minutes earlier.

03:31:54    Visitor A   list?page=1582  list?page=1581
03:36:26    Bot A       list?page=1582
03:36:29    Bot B       list?page=1582
03:36:33    Bot C       list?page=1582
03:36:39    Bot D       list?page=1582
03:38:55    Bot E       list?page=1582

03:31:55    Visitor A   list?page=1583  list?page=1582

03:43:48    Visitor A   view?id=9825    list?page=1585
03:48:37    Bot B       view?id=9825
03:48:40    Bot F       view?id=9825
03:48:45    Bot G       view?id=9825
03:48:45    Bot D       view?id=9825

This pattern repeats many times, but only for Visitor A. Their latest page hit:

10:23:28    Visitor A   list?page=1770  view?id=4758
10:27:48    Bot B       list?page=1770
10:27:52    Bot H       list?page=1770
10:27:53    Bot D       list?page=1770
10:27:58    Bot I       list?page=1770

Bot activity continued for ~20 more minutes and then ceased as well.

Has anyone seen this before? What am I looking at here?

  • 1
    Usually the actual identity of the bots, their IP address and request headers, could possible answer a question like this. But you stripped all that from your question. Commented Nov 11, 2023 at 16:43
  • As commented above, please provide snippets of your actual server access logs instead of "Bot A, Bot B, Bot C..."
    – dan
    Commented Nov 12, 2023 at 2:25


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.