2
votes
1answer
72 views

Hiding duplicate content - should I use meta tags or robots.txt?

Last week my site was utterly pummeled in Google's rankings - losing 95% of impressions overnight according to Google Webmaster Tools. It now only shows up if you search for the URL/site name itself. ...
0
votes
1answer
41 views

Hiding page content from Google when it is duplicated across several sites

I would like to hide some content from Google. Because a client has multiple websites with the same content then he gets a warning about duplicate content on some of them (multiple countries, same ...
1
vote
1answer
231 views

Redirect Google crawler to different robots.txt via .htaccess

I have googled for the answer all day and still couldn't find an answer. I have a virtual subdomain www.static.example.com which is a mirror site of www.example.com. It means I have just one root ...
1
vote
3answers
939 views

Remove subdomains from Google index and stop indexing them

I am running static content through a CDN, cdn1-cdn5 I am using such subdomains for that. I am loading just images, CSS and JS files this way, but apparently Google has indexed some pages on ...
1
vote
1answer
58 views

Removed pages from website - do I need to tell Google or will they fall off in time anyway?

This is a follow up question from here: Duplicate title tags due to translations Since asking the question I have removed the duplicated content that was being generated by language/translations on ...
1
vote
2answers
113 views

Handling duplicate content caused by 'new' smart-url structure

I used to have an URL structure like this: mydomain.com/search.php?key=value which I've since changed to: mydomain.com/key/value. I've changed all the internal links in my site, but Google ...
1
vote
1answer
106 views

Robots.txt Disallow command

How do I use robots.txt to disallow folders which are being crawled due to a bad URL structure? They're currently causing a duplicate page error. The URL which been crawled incorrectly is: ...
4
votes
1answer
274 views

Prevent Duplicate content when using CloudFlare

I have a website which uses CloudFlare CDN. At the CDN setup, CloudFlare created a sub-domain like direct.example.com to my website, which can be used to override CloudFlare and access the site ...
2
votes
2answers
87 views

Robots.txt Blocking Blog Tags?

Tags are causing duplicate content on my website and I want to make sure that I block them from being indexed. Is this the right syntax to use for blocking those Tags from being crawled? User-agent: ...
0
votes
1answer
76 views

How to overcome duplicate URLS

I am having a website which is 3 years old, now I wanted to do some SEO chages into it. so, changed the some urls. From: /psd-layouts.html to /psd-web-layouts.html Now my question is that, I had ...
7
votes
1answer
1k views

Should I block Wordpress archive pages from search engines?

I use WordPress and /sample-post/ URLs for my posts and /yyyy/mm/ for the archives. Google has indexed the site fully. Since the archives currently display full posts I've thought that I shouldn't ...
1
vote
1answer
141 views

How to disallow indexing but allow crawling?

In the front page of my website, I have some previews to articles (with a small introduction to them) that link to the full articles. I want to disallow the front page to prevent duplicate content. ...