There are several pages on the web that are not under my direct control but contain content about a product I'm involved with.

I'd like make Google robots crawl and index those pages, is there a simple way to nudge Google's spiders towards those pages? I don't have robots.txt access, and I can't provide a sitemap through Google Webmaster Tools since I don't control those websites. Also those are 100 webpages on 100 different domains that I'd like to Google to index so a single sitemap is not an option, I don't want to submit them one by one using the Google "Add URL" page since it is very time consuming.

Do you know a simple way to accomplish this task? To somehow make Google scan a list of URLs and send it's spiders to index them?


3 Answers 3


Just make a page that links to those pages. Google fill follow those links and find those new pages. Just as with any page, it won't guarantee those pages get crawled or indexed, though (but those are true even with pages you do control).

  • That's the first thing I've tried. Google seem to take it's sweet time...
    – Aviran
    Sep 28, 2013 at 12:35

Unfortunately, you can't force Google to crawl webpages. Google uses its own assortment of systems and protocols for crawling and searching/indexing. You can ask Google to crawl pages from webmaster tools but that is the only method to do it. Without access to that, your best bet would be to create pages on your site with links, or to promote more traffic to the pages in other ways. The more traffic a page gets, the more it is crawled, read, and saved. You could even promote your website(s) with Ads on places like Social Networks.


As John Conde suggests, you should make sure all those pages have links to them. If Google is being slow about indexing them you can:

  • Ensure that the page(s) that are linking are popular and have pagerank
  • Make the links prominent on the page
  • Make the pages you are linking from useful for users
  • Put links in additional places

Not the answer you're looking for? Browse other questions tagged or ask your own question.