2

My website es.wikineos.com has been running for a few years with the MediaWiki Content Management System like the one they use at Wikipedia.

Google seems to be indexing my site correctly at a quick glance but after investigating further I have discovered that Google is not indexing URLS ending with image based file extensions such as jpg, png and gif etc.

MediaWiki creates file description pages with the following format:

domain.com/wiki/Archivo:\*.\*name.description

Which will look something like:

domain.com/wiki/Archivo:image.jpg

The site does not have any metas with with no-index nor does it have any blocked URLS in the robots.txt file. Google Webmaster Tools does not report any errors and this seems to be only happening on image based URLS. Google is also indexing SVG and PDF files correctly.

If you Google: site:http://es.wikineos.com + url:jpg vs site:en.wikipedia.org + url:jpg you can clearly see that my site is not indexing jpegs while WikiPedia is. How can I get Google to index these urls.

2

Generally Google does not like indexing URLs ending with certain file extensions, especially those ending with .JPG... regardless of the received header response. Typically you should avoid using file related extensions in the URL. The likes of Wikipedia is one of the biggest and busiest sites around and they most likely get special treatment, and wouldn't be surprised if the bot they receive is a dedicated one that only scans Wikipedia and is heavily optimized just for them.

In regards of this issue there's a few reports around the net with other people having similar trouble, your best bet would be to simply rewrite the file extension to something more friendly on the index front.

One user that reported a fix was simply adding a tailing slash to the URLS but this will affect all URLS:

SOURCE

To configure your wiki that way is quite simple. I give the description here for the standard way to rewrite MediaWiki URLs using mod_rewrite - if you used another technique, you have to adapt it to that tool.

In the file "LocalSetting.php" in your wiki directory there is a variable $wgArticlePath that you have to adapt. I added the slash there, so set it to:

$wgArticlePath = "/wiki/$1/";

That makes MediaWiki format all URLs correctly with the additional slash. Obviously, you also have to modify the mod_rewrite code, that resides in the .htaccess file:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^wiki/(.*)/ w/index.php?title=$1 [PT,L,QSA]
RewriteRule ^wiki/*$ wiki/Main_Page [L,QSA]
RewriteRule ^/*$ wiki/Main_Page [L,QSA]

Additionally, I added a few lines to rewrite the old URLs (without the slash), so that older links to my site don't end in nirvana but are forwarded to the new location. Acutally, I use a 301 http redirect to the new site, so I don't lose any link power. The code is not perfect, but works in most cases (it does not work when a slash is included in the article name itself):

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^wiki/(.*)$ http://www.domain.com/wiki/$1/ [R=301,L]

Then, your site will be index by google as well. Have fun!

Searching the net simply for the term "Powered by MediaWiki" I can see many other MediaWiki's operating with this problem.

EG.

  1. http://chrishecker.com/File:Chris_Hecker%27s_Home_Page_%2820070221%29.png
  2. http://wiki.c2b2.columbia.edu/califanolab/index.php/File:Header.png

Also it would seem that not even WikiMedia have there images indexed either > site:http://www.mediawiki.org/ + url:.jpg. It's also worth mentioning that Google will consider these pages low value so why would you want them to be indexed? chances are they will never be found in any case.

  • I want Google to index these pages because only the small thumbs of the images used in the articles are indexed. I would like Google Images to have my high quality images but these are only linked from the file description pages, which usually end with .jpg. – user1314836 Apr 28 '14 at 21:30
  • By the way, thank you for your help understanding the problem!!! Thinking about the solution... these files can have any extension. Indeed, they don't even need an extension. I guess that the best solution would be to rewrite all of the pages ending with .jpg to delete that extension and change that in the articles in which the images are inserted. I'm sure I can do this without much trouble. The pitty is that having it ending with .jpg helped me know it is an image I can insert in the articles. – user1314836 Apr 28 '14 at 21:33
  • Google is fully aware of your images, your rarely ever see a image in the organics on a 'Normal Search' using site: + url: is not a normal search... Your images are indexed under images google.co.uk/search?q=http://… use the / ending slash trick to get description pages indexed, but bare in mind.. these are low quality and its unlikely they will ever get visitors because Google will not rank them highly... Hope this helps, Good luck. – Simon Hayter Apr 29 '14 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.