Our website is an agenda of events taking place in our city.
We have a datepicker for users to look at events scheduled in the next week, next month, etc...
We recently discovered that GoogleBot is crawling this datepicker, requesting events centuries in the future: it crawls URLs like
Is there a way for us to tell GoogleBot to not look that far in the future?
I apologize if this is a silly question, but I am a developer, not a webmaster thus I'm pretty noobie about this.
As suggested by Stephen in the comments, I should prevent both human and bot visitors from requesting events too far in the future. I can do this in code without having to rely on updating robots.txt periodically.
I have a doubt though: won't GoogleBot periodically request all the URLs in the future that it has already crawled?
If so, maybe I should complement this solution with a few rules in robots.txt to block requests further than say 10 years in the future, or even better, do this dynamically in code once again.