Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

When I wanted to fetch a certain page with htm extension I noticed an attachment after the htm extenstion http://www.example.com/IP_814.htm#.Uk1_MtKsh8E Google could not fetch this page it gave the 502 error code.

When I fetched the same page as http://www.example.com/IP_814.htm Google fetched it and gave me the green check.

When I created the page, I named it IP_814 with extension htm as this is a individual internal product page. I believe that the code or number behind the htm changes daily.

What should I do so Google crawls my individual product pages with the htm extension?

share|improve this question
When you say "google fetched it", are you referring to the "fetch as Googlebot" feature in Google webmaster tools? – Stephen Ostermiller Oct 3 '13 at 15:29
Yes I was referring to "Fetched as googlebot" – Lexi Oct 3 '13 at 16:01

A # and anything following in a URL is a fragment identifier. It instructs the browser to fetch the page from the server that is specified by the portion before the # and then show the user the portion of the page identified by the fragment. This is often implemented as scrolling the page to the appropriate section, but the JavaScript on the page may examine the fragment and take another action such as showing a specific DHTML tab.

When URLs are requested, the fragment identifier is never sent to the server. Browsers must strip this information from the URL before making the request. Googlebot should never be crawling URLs with fragments. It should be stripping the fragment off before crawling the URL.

share|improve this answer
OH I think you answered my question, because I was the one who entered the whole link in webmaster google for the link to be fetched as google. The second time without knowing I stripped the fragment identifier, and that's why google was able to fetch my page. Thank you so much for explaining it to me! – Lexi Oct 3 '13 at 16:00
I would expect that even if you entered it into the "fetch as Googlebot" tool that it would strip it off before it fetched it, but they may have a bug in that tool caused by an oversight of that case. – Stephen Ostermiller Oct 3 '13 at 16:02

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.