1

My website logs spit out "invalid content" messages -- they typically occur when someone types in a wrong hyperlink or a page has mistakenly changed URLs. Lately (about 20 times per day) I've been noticing a trend with these logs. It's probably best explained with an example:

Assuming a page URL is http://www.myswebsite.com/news/1048/title-of-some-news-item, I will receive one of those log messages that says someone tried to visit something along the lines of http://www.mywebsite.com/news/1048/titlee-of-some-news-itm or some variation that is incredibly close to the original slug, with just a few characters added, changed, or removed. The id (1048 in this case) is always the same.

I don't think people are mistake-prone enough to do this 20 times per day. Is this a bot? Is it common, and what are they trying to accomplish?

Improve this question
7
  • Have these slugs ever been incorrect on the site and they were later corrected? Are these titles user generated? Are users submitting this information and later correcting it (as they do on this site)? – MrWhite Jun 18 '14 at 14:57
  • There have been very few changes to the slugs through the life of the website. Each time there was a change, I made sure to add a 301 redirect to the original slug. – Nick S. Jun 18 '14 at 14:59
  • The titles are not user generated -- they are created by me internally. Users don't have the ability to edit any of the elements of the pages. – Nick S. Jun 18 '14 at 14:59
  • Is this happening to many different pages/urls or just a few urls repeatedly? Do the users appear real or bot-like? – MrWhite Jun 18 '14 at 15:03
  • I haven't noticed any trends in the urls. It seems to be a random page each time, which makes me think it is a bot. Also, a sampling of the IPs point to networks in Germany and China (mostly China). – Nick S. Jun 18 '14 at 16:22
1

A possibility is that these hits are made through links that people have added to their blogs/sites or shared with others using email, social networks etc.

It's happening to me all the time. They try to add a link to their website, but they do not make the copy paste correctly or they add extra characters or even delete some of them (accidental/mistaken keystrokes).

That's why some of the last characters are missing or url variations appear.

Suggestion

If you load the page based on post's id for example 1048, try to redirect to the correct slug if the slug is not the correct one

Also if the post's id does not exists but the slug exists, redirect to the correct url again.

If the given post's id is invalid or does not exist, and also the slug does not exists, try to find the most relevant url to redirect to, or display an error 404/410 with the urls that are the most relevant, so that the visitor can click and be redirected to the article that he wanted to read.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.