Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

In the little summary extract which Google displays about pages from a site I administer, there's frequently a Flash error message:

Page Title

www.example.com/.../Slug_of_the_page
Flex Error #1001: Digest mismatch with RSL http://www.example.com/framework_3.2.0.3958.swf. Redeploy the matching RSL or relink your application with the ...

I know people will want to tell me what the error message means and how to recompile, but there are plenty of instructions about that elsewhere, and that's not my question.

I see this happening in search summaries for pages on this site which I think don't have any Flash objects in them - and certainly Firebug's Net tab doesn't show any. Google's cached version of the page doesn't include the error messages in the HTML or text-only versions.

Trying things at random and waiting for Google to re-index is clearly not a sane strategy. Does anyone know a tool which reproduces as closely as possible Google's web page to text converter, or which in some way would allow me to reproduce the error message locally so that I can debug and test that I've fixed it?

share|improve this question
    
What about GWT's "Fetch as Googlebot" tool? –  w3d Mar 14 at 18:59
add comment

1 Answer 1

Googlebot uses code that is custom to Google for converting the HTML pages into text. Google has not released this code, and nobody is sure exactly what they do in all cases.

Usually Googlebot just sees the page source and removes the HTML tags. There are many programs that can download the source code of a page (even masquerading as the Googlebot user agent.) I tend to use curl like this:

curl -s -A 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' 'http://example.com'

There are not many cases in which Googlebot indexes text that is not in the page source. The only ones that I have ever seen:

  • Meta description taken from a listing on dmoz.org rather than from the page source
  • A page that is blocked by robots.txt may have anchor text of inbound links indexed as a stand in for page content.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.