301 redirect away from content that is served with a reverse proxy to prevent duplicate content

Question

I am currently struggling to resolve an issue stemming from a reverse proxy. We are running a reverse proxy from page.example.com to example.com/page/ . This original subdomain content is generated from Unbounce.com , and attached to the subdomain.

Problem: We want the subdomain ported to a sub-directory for SEO reasons. The reverse proxy works great, and properly ports the sub-domain, to the sub-directory. Our current issue is that the page.example.com is still getting indexed in Google. We'd like to install 301 redirects to allow only the sub-directory to get indexed. Because we do not have direct access to the content on page.example.com , I cannot put an htaccess file in place to redirect that traffic. Unbounce also charges based on "traffic", and the reverse proxy breaks their tracking information.

They have told us, if we put the 301's in place, their tracking info should push to the new URL structure, and work again.

If anyone could point me in the right direction, that would be fantastic.

Answer 1

You do not need to move a subdomain to a folder for SEO reasons. Content on subdomains can work great from an SEO perspective. See: Do subdomains help/hurt SEO?

You won't be able to issue 301 redirects from the proxied pages. Your proxy needs those pages up and running to be able to work correctly. You you had page.example.com issue a 301 redirect then your reverse proxy would no longer be able to access the content.

You have several possible solutions:

1. Implement meta link rel canonical tags. If you put those tags onto the pages served at page.example.com and they pointed to example.com/page then Google would do the right thing. You would be telling Google that there is duplicate content and the preferred URL is the subdirectory. Unlike 301 redirects, the meta tags won't prevent your reverse proxy from operating properly.
2. Rename the subdomain to something else like xyzzy.example.com and reverse proxy it from there instead. Then the page.example.com would 404 and fall out of the Google index. You might have to deal with the same problem again in the future because Google will eventually find the new subdomain too.
3. Use robots.txt on the subdomain to prevent Google from crawling it. You would still need to allow the reverse proxy to crawl the subdomain. The pages would eventually mostly fall out of the index. You could implement robots.txt in conjunction with changing the subdomain. Then Googlebot wouldn't index the new subdomain name.
4. Implement 301 redirects, but only when the content isn't being proxied. Your host would have to put in code that identifies the reverse proxy requests by user agent or other headers. It would serve the content for those requests, but issue 301 redirects otherwise.
5. Just let the content reside on the subdomain and redirect the subdirectory to it.