Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

We use Joomla with Remository to store and manage publications (don't ask me why). Files (PDF) are stored in a database and can be accessed via dynamic, rewritten links of the form

http://domain.de/some/path/filename.html

Here is an example: some file

Current browsers reliably detect that they get a PDF. wget uses the .html filename but after renaming I get a working PDF file. curl behaves similarly; piping its output into a (suitably named) files gives a working file. All this leads me to believe that -- against all odds, one might say -- the data our system provides is generally valid and understandable for clients.

However, Google does not seem to index PDF files referenced by such links. Our publication list is indexed, but the PDFs linked there are not (they don't show up in web and Scholar searches).

How can we tell search robots to retrieve our files and index them?

share|improve this question
    
Have you considered embedding the PDF files on your website? This could encoverage people to return ot the site while giving them the option to print the pdf's as well as giving Google access to them. –  bybe Apr 3 '13 at 16:20
    
@bybe I think an obstacle-free download is the best choice, especially regarding resource overhead and usability. –  Raphael Apr 4 '13 at 9:18
    
Well a possibility is to embedd and include a download, surely this is best of both worlds? I understand its important to make things simple, but other than you'd need to make sure that Google can be redirected to the PDF on the GET, using the sitemap. –  bybe Apr 4 '13 at 10:10
    
@bybe I'm not sure embedding PDFs is all that useful (consider the new built-in PDF viewer in FF), but thanks for the suggestion. In any, it's the boss who decides the larger stuff. –  Raphael Apr 4 '13 at 10:38
    
Embedding PDFs can gain backlinks, ofc direct download links can do but I believe that juice will flow better if its on the page. So if SEO value is a high priority embedding is fair better for obtaining backlinks from visitors, ofc its not required - just my 2 cents. –  bybe Apr 4 '13 at 11:02
show 2 more comments

1 Answer

You cannot tell them but give them a strong hint by providing a sitemap. Google may or may not index those these even with a sitemap. It will tell you how many of the sitemap files were indexed. You need a Google Webmaster Tools account and register your website with them. Once done, sitemap submissions and index status appears the reports.

From a search engine's perspective it really does not matter where the data comes from, only that it is accessible. You may be doing something fancy that Google does not like but it is not the fact your documents are in the database.

From the link you provided, I see something automatically trying to download when clicking on your links which may count as an unwanted drive-by download, so be careful and is really a poor user experience. If the link is meant to be a download, then there are pages too many. Check your mime-types too as they may simply be confusing the Google crawler.

share|improve this answer
    
The download links are reachable from the sitemap. It seems we had rather dumb problem with nofollow directives in placed we did not want them; investigating now. –  Raphael Apr 4 '13 at 8:40
    
bare in mind that nofollow doesn't stop Google indexing files, pages... if its not indexed already then there's a problem. You should try to do a Google fetch within webmaster tools. –  bybe Apr 4 '13 at 10:11
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.