4

I updated my web site to be mobile friendly last year and everything seemed to be OK, Google and other tests reported that it was now mobile friendly.

However, I just logged into my Webmaster Tools account and found some messages about fixing mobile usability issues. The thing that is puzzling me is that the "page" that it says there are problems with is not a web page at all, it is a sub-folder in which I store images.

The sub folder is not linked to in the site navigation and I didn't intend any visitors to the site to ever see it, so I am puzzled as to why the Google bot is looking at it.

1

Without knowing which web server you are running, I am assuming Apache. Similar configuration options exist for all of the popular web servers.

My first guess is that you have the directory linked and that the Index option is allowed for this directory or for the site as a whole. It may be that you do not have a link, however, Google is finding the directory index anyway. It is recommended that you at least check to see if the Index option exists within your Apache configuration file or within an existing .htaccess file within your web space.

In the Apache configuration file, either httpd.conf, 000-default.conf, or example.com.conf, the Options directive may have Index enabled where -Index is recommended. For most installations, the Apache configuration files can be found in /etc/apache2/ or /etc/apache2/sites-available/.

Information on the Options directive within the configuration file can be found on the Apache core page here: https://httpd.apache.org/docs/current/mod/core.html

This can also be controlled within the sites .htaccess file or within an .htaccess file located within the sub-directory.

The Options directive can also be used within the .htaccess file. Information on the .htaccess file can be found here: https://httpd.apache.org/docs/current/howto/htaccess.html

  • Thanks for your comments, very helpful if a little above the scope of my current knowledge. I just ftp the files, but I do believe that the server is Apache. I didn't intend to present this sub-folder to anyone visiting the site and it is definitely not linked to in the site navigation or sitemap.xml which is why I was surprised that the Google bot even looked at it. As a quick fix I added it, and other folders that are not supposed to be exposed, as "disallowed" in the Robots.txt file, but I will also ask the host support about the Apache configuration file as you suggest. Thanks! – DavidCheshire Apr 25 '16 at 9:57
1

You can also control some HTML in the HTML header that is generated with each output that apache uses for its default directory listings. This code if used inside the httpd main configuration file (usually httpd.conf) will cause the directory listings to not be indexable or URLs within it to be followed, and it will add a viewport tag compatible with all mobile devices so they can see the directory listings normally.

IndexOptions Charset=UTF-8 SuppressHTMLPreamble 
IndexHeadInsert "<META NAME=\"ROBOTS\" CONTENT=\"NOINDEX, NOFOLLOW\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">"

Now if you didn't mean to present a sub-folder to a user, then you need to use the index file defined in the apache configuration file. You can usually get away with using index.html or index.htm or index.php in that folder.

  • Thanks for your comments. I didn't mean to present the sub-folder to a user, which is why I was surprised that Google found it. I have disallowed it in the robots.txt file, will that work as well as your suggestion, which is a bit advanced for me at the moment. – DavidCheshire Apr 25 '16 at 10:03
0

To answer your only question.

Your subdirectory is one containing lots of images which, presumably, is a lot of data. Perhaps megabytes, even gigabytes or terabytes, full of data. That wouldn't be a very mobile friendly territory for one on a cell phone now would it?

  • I see your point: I didn't phrase my question very well. I didn't intend to present this sub-folder to anyone visiting the site and it is not included in the site navigation or sitemap.xml so I was surprised that the Google bot even looked at it. – DavidCheshire Apr 25 '16 at 9:54
0

The simplest solution may be adding Options -Indexes to your .htaccess file. This prevents directory listings; simple as that.

You mentioned in a comment to another answer that you've disallowed it in the robots.txt file, but that might not be ideal. If you use any of these images within your other pages, then disallowing the directory stops search engines from crawling those images and rendering your pages correctly. Options -Indexes stops the "page" from being available without stopping crawlers from accessing its contents.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.