In Google Webmaster Tools you can use the "Fetch as Google" feature to have Google download a page from your site. Once you have done so, you can click the "Submit to Index" button which gives you the following options:

If you select "Crawl this Url and Direct Links" does it only crawl links that are part of your domain or all links (even external links)?

Google's documentation for this feature is here and it does not address this question. It only says:

Select Crawl this URL and its direct links to submit the URL as well as all the other pages that URL links to for re-crawling. You can submit up to 10 of requests of this kind per month.

It doesn't say whether or not it will re-crawl URLs linked from the page to other domains.

  • Only the links associated with your site. – closetnoc Jun 24 '15 at 22:17
  • I assume it doesnt traverse over subdomains ? – Paul Taylor Jun 25 '15 at 13:13
  • Sub-domains? Probably does not crawl them. – closetnoc Jun 25 '15 at 14:44

IIRC, there is an option that lets you choose whether you want to crawl only the specified URL or all the URL in that domain.

  • That's the question; what does that do? How does it work? – Andrew Lott Nov 23 '15 at 12:20
  • It doesn't say anything about "in that domain" that I can find. The documentation says "all the other pages that URL links to". – Stephen Ostermiller Dec 23 '15 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.