I have tracking redirects on my site. I use server-side tracking on them. They are used for things like:

  • Advertisements
  • Social media sharing

The URLs are of the forms:

  • /ad-click?id=12345
  • /social-share?site=twitter&content=23456

So that bots don't inflate the stats of them, I use the following in robots.txt

Disallow: /ad-click
Disallow: /social-share

The problem is that Google is now indexing some of them. Google reports they are "indexed but blocked by robots.txt." Having them indexed also messes up the stats because now people sometimes click on them from the search results.

The typical solution for that message in Search Console is to allow them to be crawled and use a noindex tag. However, that isn't going to work for tracking redirects because:

  • Having them crawled would mess up the stats.
  • Google doesn't index redirects anyway, so adding noindex wouldn't make any sense on URLs that redirect.

I really wish Google hadn't stopped supporting Noindex: in robots.txt, this would be the perfect use case for it.

At this point, what are my options for server-side tracked redirects that should be neither crawled nor indexed?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.