The scenario is

  1. My page is a music description page with proper title, description, keywords, h1 and a description as a paragraph
  2. Url of this page is a meaningful long slug(not very long) separated by hyphen
  3. This page has a download link(to download a small music file) which will force download that file using content disposition
  4. Download link will have title text like a sentence
  5. Download links hyperlink will have a meaningful slug separated by hyphen
  6. Hypertext like 'Click here to download'
  7. Sample of the url is sitename.com/module/categoryname/title-of-the-music-sample/
  8. Sample of the hyperlink slug is href='/module/get/title-of-the-music-sample/' and assume that the title attribute text is similar to the slug.

I dont want the download link to be indexed. that is i dont want bots to index anything starts with /module/get/ because

I am afraid that:

  1. that the slug for the link and the slug of that pages url are the same so it may be considered duplicate ... the only difference in the url is /categoryname/title-of-the-music-sample/ and /get/title-of-the-music-sample/
  2. that google may penalize thinking i am trying to populate duplicate keywords in the link tags which is similar to the current pages url.
  3. that directly accessing the /get/(.)/ will redirect to the actual music page of /categoryname/($1)/ if the user is not coming from /category/ ... i am restricting users to directly load the download page... so i redirect to the main page and only then they can download the music file...
  4. since i am redirecting /get/ to /categoryname/actualcontent i think there is a trouble i this which is not a good practice...

So thought of adding disallow for /module/get/ to prevent bots from indexing or using that /get/* links

can i use nofollow to my own website. like in the download link can in add rel='nofollow' if i do not want bots to crawl or index that link?

If i dont want those links to be indexed then why do i use a hyeprlink ... because thinking that those keywords would add more keywords to the document or some kind of gimmick i see in other sites which would boost ranking. i think this approach is not good... may be or may not.

If what i did is not a problem then i will leave it as it were... and add the condition in robots hinting not to crawl or index /get/* links

Else i will change it to a span tag with an onclick event yet writing in a way so that bots will not crawl or index window.location ... .

Will those links be penalized or indexed?

if it is going to be penalized to some extent or whole then what are all the points that i have missed to think to make the link proper?

if it is going to be indexed then still is there any flaws which people would say that doesn't matter?

Suggestions please.

  • I've posted an answer to your question - I think. Re-reading your question, I see that you are asking many things. Please summarize your question(s) in a list or something as it is a bit difficult to follow your reasoning at times, and this could be closed for not being a real question. Commented Jan 27, 2013 at 10:36

1 Answer 1


Disallow it in your yoursite.com/robots.txt, like so:

Disallow: /module/get/

Edit: Yes, you can use rel="nofollow" on your link. This means that Google will not attribute ranking to it, however it doesn't necessarily mean that it won't follow the link at all. It might still be indexed. Using Disallow in robots.txt is still the best option in my honest opinion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.