1

For our domain, example.com, we have 2 subdomains: a0.example.com and a1.example.com. These 2 are meant only for product images -- images stored here are accessed like a0.example.com/var1/var2/prod_id_img_1.jpg, and linked from products listed under www.example.com. There is no other usage of these subdomains.

Before release, we had a page at (a0|a1).example.com saying 'Coming Soon', which we missed out to remove later on (as not accessed from our main site).

Now, we find that when we search for our domain, 'example', then the 1st sitelink is 'Coming Soon' and links to a0.example.com (the other 5 sitelinks shown are valid).

Questions are:

  1. How do I properly get rid of this page so that it doesn't show up on google sitelinks (and any other searches, if any)? Should I just remove it to effect a 404 status or should I run it through 410, or do I need to do anything else -- for a proper removal (for ever)?

  2. Scanning through the nginx logs, I notice that google bot is periodically looking for sitemaps under (a0|a1).example.com. Am I missing any explicit configuration due to which google bot is looking for sitemaps here, or is this behaviour kind of standard and I need not worry?

  3. On webmaster, we have example.com and www.example.com properties. example.com is not configured but www.example.com is and is working fine. Do I need to do anything with example.com property that may help with our (s0|s1).example.com usage?

Thank you in advance for your help.

Improve this question

1 Answer 1

Reset to default
1

You have a few options.

  1. Remove and allow to all requests to issue a 404 error, however, if this is an index.html within the root of each sub-domain, I would highly recommend not doing this and keeping the file for security reasons.

  2. Exclude the page in a robots.txt file within each sub-domain web space. Here is example code to exclude the index.html file within each sub-domain web root.

    User-agent: *
Disallow: /index.html

  3. Use noindex within the HTML header. Here is example code to place within the HTML <head> tag.

    <meta name="robots" content="noindex">

  4. Redirect any request for the page to the site home page. Here is example code to do just that. You can put this in your .htaccess file within each sub-domain web root. (Assuming Apache)

    Redirect permanent /index.html http://www.example.com/
Improve this answer
10
  • Firstly, thank you for your answers. In your point #1 you mention of security concerns with removal of the index page on the subdomains (only for images). Can you elaborate on security issue?
    – Ethan Collins
    Mar 20, 2017 at 23:50
  • @EthanCollins Only in that you are exposing a sites structure and making it easier for hackers to see what vulnerabilities exist. That is all. I always recommend that an index.html exist even if you want to have a directory index that does not default to index.html. You may see this scenario on download sites.
    – closetnoc
    Mar 21, 2017 at 0:47
  • Thanks. I am still trying to understand (not understanding the directory structure decoding issue, as it's quite apparent by looking at the images link on main site). Let me checkout the download sites. An example can be wetransfer?
    – Ethan Collins
    Mar 21, 2017 at 0:52
  • @EthanCollins I cannot remember any sites specifically, but I do tend to go to the heavy hitters such as Apache, Ubuntu, MySQL, etc. Some of them use directory indexes. In many of them you may see an index.html. The reason for this is to make life easier for users by combining descriptions with links. However, if you are just creating a web space for standard HTTP image access for a web site and not for uses to browse, an index.html is always a good idea JIC (just in case).
    – closetnoc
    Mar 21, 2017 at 0:57
  • Thanks. Will you be able to answer the last 2 queries that I had? (I have edited my question with numbers now).
    – Ethan Collins
    Mar 21, 2017 at 4:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.