2

Lets say you write a script that checks a URL every few seconds, but on the website there is a counter for how many times a person visits the site.

As a side effect of you checking the site, this number increases.

Multiply this a few times a minute for how ever long the script is used, this could increase it drastically.

This does not DOS the site or anything like that.

Are there any legal ramifications for writing a script to check this site where this is an incidental side effect?

  • 1
    No. I am not even sure why you are asking the question. – Steve Jun 26 '18 at 21:52
  • It just seems like the owner of a website could get frustrated with the fact that their view counter has suddenly started going off the charts. @Steve – Bigbob556677 Jun 26 '18 at 21:55
  • 4
    Why in this day and age is anyone using a counter? – closetnoc Jun 26 '18 at 22:29
  • 4
    Does the website have a terms of service? If so, you might get some direction from the website owner directly about how much they might limit such uses of their website. Does the website have a robots.txt? You should obey it if it does, including using a descriptive user agent that could be blocked in robots.txt. – Stephen Ostermiller Jun 27 '18 at 11:57
  • 1
    Sometime we can forget that you guys in the field can see some weird-@$$ stuff. Don't let your dreams become nightmares! Cheers!! – closetnoc Jun 28 '18 at 15:15
4

As mentioned by Stephen Ostermiller, you have several options.

1. Asking Owner

Ask the owner of the site directly. Of course, this could be a scary thought... but I think that's the best way if you are to hammer the site with so many hits in a row.

2. Check the site legal document(s)

You will likely find some page with Terms of Service, Terms of Use, etc.

In these documents, unfortunately, the information is most often quite general. Once in a while, though, there are useful details, especially if that document is very long... (Yeah, work.)

3. robots.txt

Since you are talking about running a script, that means you are running a robot. You need to give it a specific name that won't change and check the robots.txt and make sure that your robot is allowed to check the site.

So if the user doesn't like it and sees your robot's name in their log, they can add a line or two to their robots.txt to block you.

4. Speed Issues

One thing that Google says about their robot (and I'm sure others do too) is that they will do the right thing to avoid flooding your website with hits that would slow down your website.

I have no idea how they have implemented that but I could imagine that they have some way to know how quickly a server answers one of their request and if further hits are answered slower, they probably slow down after that.

Because frankly, hitting a server the same URL every few seconds is probably going to be seen as an attack. When a robot, as in a spider, checks a website, at least it attempts to check all the pages of the site, not one single page.

5. Counters...

This sounds like very old technology! A good counter is one that at least doesn't count repeat from the same IP address when hits are very close to each others (i.e. you'd save the IP for a few minutes and update the date when it needs to be deleted if the user re-access the site). That being said, you should not be too worried about one specific problem.

Newer systems make use of JavaScript which means most robots would not generate an increase of the counter since most robots would never execute that JavaScript code. Oh well...

  • Thank you for the break down. Their site isn't under a threat of being DOS'd. It loads and operates fine. As for why they use a counter....I have no idea. – Bigbob556677 Jun 28 '18 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.