I have a web calendar, all my own work. It works fine. But I see in the logs a bunch of search indexers go round and round and round trying to chase every link. It could go forward to the year 3000 month by month and day by day. I don't think the indexers should follow this as it doesn't serve much purpose. What HTTP headers or META headers should I apply to the calendar to let them know it is a calendar, but they shouldn't follow every link on those pages?


Use rel="nofollow" on all links that you don't want spiders to follow or index (i.e. days, months, etc.).

Use robots.txt to specify paths that you want them to ignore.

If your site is a web application, consider using robots.txt to disallow indexing of the whole app with:

User-agent: *
Disallow: /
| improve this answer | |

Probably simply <a rel='nofollow' ... on the calendar navigation links (as opposed to the events that link to specific events) would be fine.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.