4

Since April 16th I've been getting alerts regarding a high number of 404s to pages that do exist on our website, but with /mobile appended. For example: www.mysite.com/some-page/mobile

There appears to be a variety of user agents being used, including Chrome and Firefox. IP addresses appear to be from legitimate sources (i.e. the general public).

Any ideas where this could be coming from? Has there been any new technology released that could be looking for these pages, or is there perhaps a tool that someone in the marketing department has installed...

Update: Answering a few comments here:

  • The Http Referrer data looks like these are legitimate users. The general activity on the site also makes it look like these are legitimate users - i.e. the number of pages visited and the traversal through those pages.
  • There is no consistency in referrers - most are from Google, which is normal, but some are from Bing or other websites.
  • Google Webmaster Tools does not indicate any crawl errors for these pages
  • No we are not a Tumblr site :)

My guess is this is a browser plugin of some sort...

  • Is GWT reporting these 404s? – DocRoot Apr 19 '16 at 8:29
  • There is not enough information for us to even guess. Some sample log entries can help. I will bet dollars to donuts that the IP addresses are not from subscriber blocks meaning that they are not human. As well, the referrer is an important bit of information. If a user is following a link, the referrer should reflect where they are coming from. If the referrer is the same, then you are being hit by a bot of some sort. Check the user agent carefully. For example, using a log file analyzer, you can easily see if there are any inconsistencies in the session data. Often there is. – closetnoc Apr 19 '16 at 14:16
  • 1
    fwiw - I'm also seeing this behavior in my logs starting recently. The Referer is always a real page on my site with a similar URL (e.g. /blog/mobile has a Referer of /blog). This makes it look like legitimate 404's due to a link, but it's all over the site and I can't find any links or JS causing this. The user agent's "look" real - e.g. Chrome on OSX. – weaverryan Apr 20 '16 at 13:55
  • Also, the requests are always POST requests in my logs. Also, they only come from a sub-set of pages. The only common thread I can see on those pages is the presence of Disqus commenting and an AddThis widget. I think it's nothing wrong with our site - but maybe this will help someone else :) – weaverryan Apr 23 '16 at 16:58
1

The first question I'd have is if you can identify the source leading to these error pages? Are they all direct? Or are they coming from a referral source? If you can pin down the source, you could correct the links at the source.

If they are all direct, then I'd be suspicious of some technology routing mobile visitors to a "/mobile" page. Can you tell what device these people are on who are reaching the "/mobile" pages? If you can, can you test your site on that device to see if you are redirected to that "/mobile" page? Perhaps there is some device targeting in the works.

I'm not sure where that would come from. Most of the mobile routing tools take you to a subdomain, not appending "/mobile". I would scan through builtwith.com to see what services are on your site currently and if any of those happen to do something where they'd try to create a mobile site.

And, just as a "throw it out there" idea, the only site I know of where "/mobile" works at the end is Tumblr. Adding /mobile to the end of a URL forces a mobile version of the page. As in:

  • example.tumblr.com/tagged/subfolder/
  • example.tumblr.com/tagged/subfolder/mobile

But I'm guessing you aren't on a Tumblr site. Though, if you recently migrated URLs over from your Tumblr site, perhaps some people are still looking for the "mobile" version? That seems like a reach, I'd check sources, devices, and what tools are on your site first.

  • Thanks, interesting insights - added an update to my question – cbp Apr 21 '16 at 5:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.