1

My traffic is not from search engine and I want to prevent bots or others from displaying or accessing it.

What would be a good .htaccess code to do so?

I've heard

User-agent: ia_archiver

Disallow: /

Don't really remove Alexa ranking because Alexa got the content from it's toolbar. Okay. What is then?

3
  • Please explain your reasons for blocking search engines. Just because they aren't sending you much traffic does not mean you should block them. Surely anything they send your way is good? Commented Jul 8, 2013 at 10:44
  • To prevent my competitors to from knowing my sites are and what countries it's promoted.
    – user4951
    Commented Jul 8, 2013 at 10:57
  • That seems very misguided to me: if you are hiding from competitors you are also hiding from customers. But this isn't really the place to discuss that. Commented Jul 8, 2013 at 14:43

3 Answers 3

2

Use a robots.txt file to prevent spiders from crawling and indexing your content.

From Alexa's "for webmasters" page:

User-agent: ia_archiver
Disallow: /

From Googe's robots.txt help page:

User-Agent: Googlebot
Disallow: /

If you want to block all robots and prevent all spiders from crawling and indexing your content:

User-Agent: *
Disallow: /

Alexa does not need to crawl or index your content to display an "Alexa Ranking" for your website. It bases its rankings on the number of people that visit your website. It gets this data from its toolbar or by purchasing data from ISPs. As such, blocking the crawler will not prevent Alexa from displaying the ranking. There does not appear to be a way to have your site opt-out of the Alexa rankings.

0

To prevent all robots from indexing a page on your site, place the following meta tag into the section of your page:

For more info read : https://support.google.com/webmasters/answer/93710?hl=en

I don't think its possible to prevent alexa from indexing a page. Since its indexing a page from it's toolbar, i don't know is it possible to prevent that..

0

All of those htaccess parameters are just requests, Google etc can still crawl your site since it is on the internet and publicly available. You should try to define what you wnat to allow them to crawl in a robots.txt file

This might help https://yoast.com/x-robots-tag-play/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.