0

I have a website with a around 100k unique pages that are regularly updated. Our search engine rankings have been steadily improving and so has our traffic. Search engines crawl the site constantly and account for a lot of activity in the logs. However, I am seeing really large portion of the traffic is single requests from unique ips. They look like real users from real isps with normal user agents and requesting various pages. A good portion seems geographically strange (ie from far away Countries when our site is Country specific) but it doesn't look like a ddos attack. Out of all the unique ip addresses that hit our site, 80% of them have single requests like this. Normally I wouldn't even notice nor care too much but our host charges based on the number of visitors (calculated as unique ip addresses per day) and needless to say we are blowing through our plan limits. Google Analytics is not recording these single requests.

I am not sure what would cause this. Would the browser website prefetch feature account for some of this? Looking at Google Search Console, we have 300k impressions each day with about 3k clicks.

2
  • Do those page hits just download the page, or do they also download things lmbedded in the page (like CSS, images, javascript)? Does the REFERER give you any hints? Also, why is your host charging by number of visitors - that would seem a very weird metric to charge on - could they be gaming you?
    – davidgo
    Jan 20 at 8:56
  • Just hit the page, no js/css/etc. Referer wasn't consistent but looked like regular traffic primarily mobile browsers. This was also occurring to a lesser extent on another similar site I manage but was able to use Cloudflare and it would block 20k+ daily using their managed challenge. Has now subsided significantly after about 6 weeks and moved to a different host that doesn't charge on visitors. I agree that is a horrible metric to charge on given the increasing amount of bot traffic. Still unknown what the cause was.
    – Marko1
    Jan 22 at 0:10

1 Answer 1

1

First thing is to identify the most common single request visits and see how/if they relate to your site.

Causes of Single-Request Visits:

Scrapers: Bots designed to harvest content might visit a single page, grab what they need, and move on. These can come from legitimate research bots or malicious actors.

Social media previews: Sharing your content on social media platforms can trigger bots fetching previews or checking metadata.

Referral spam: Malicious actors can send fake referral links to your site, generating single-request visits from nonexistent users.

Crawler misconfiguration: Some crawlers might be misconfigured to visit each page only once, even if they haven't fully explored your site.

Browser prefetching: As you mentioned, prefetching can cause browsers to make speculative requests for linked resources on your site, but this typically wouldn't account for 80% of your traffic.

You can do all of this by looking at your access logs. Check for patterns in IP addresses, user agents, and requested pages. Are there specific pages or sections attracting these single requests, maybe your business unit can provide additional insight to page popularity.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.