In Google Webmaster Tools you can use the "Fetch as Google" feature to have Google download a page from your site. Once you have done so, you can click the "Submit to Index" button which gives you the following options:

If you select "Crawl this Url and Direct Links" does it only crawl links that are part of your domain or all links (even external links)?

Google's documentation for this feature is here and it does not address this question. It only says:

Select Crawl this URL and its direct links to submit the URL as well as all the other pages that URL links to for re-crawling. You can submit up to 10 of requests of this kind per month.

It doesn't say whether or not it will re-crawl URLs linked from the page to other domains.

  • Only the links associated with your site.
    – closetnoc
    Jun 24, 2015 at 22:17
  • I assume it doesnt traverse over subdomains ? Jun 25, 2015 at 13:13
  • Sub-domains? Probably does not crawl them.
    – closetnoc
    Jun 25, 2015 at 14:44

1 Answer 1


IIRC, there is an option that lets you choose whether you want to crawl only the specified URL or all the URL in that domain.

  • That's the question; what does that do? How does it work? Nov 23, 2015 at 12:20
  • It doesn't say anything about "in that domain" that I can find. The documentation says "all the other pages that URL links to". Dec 23, 2015 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.