Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

There are several pages on the web that are not under my direct control but contain content about a product I'm involved with.

I'd like make Google robots crawl and index those pages, is there a simple way to nudge Google's spiders towards those pages? I don't have robots.txt access, and I can't provide a sitemap through Google Webmaster Tools since I don't control those websites. Also those are 100 webpages on 100 different domains that I'd like to Google to index so a single sitemap is not an option, I don't want to submit them one by one using the Google "Add URL" page since it is very time consuming.

Do you know a simple way to accomplish this task? To somehow make Google scan a list of URLs and send it's spiders to index them?

share|improve this question
add comment

3 Answers 3

Just make a page that links to those pages. Google fill follow those links and find those new pages. Just as with any page, it won't guarantee those pages get crawled or indexed, though (but those are true even with pages you do control).

share|improve this answer
    
That's the first thing I've tried. Google seem to take it's sweet time... –  aviran Sep 28 '13 at 12:35
add comment

Unfortunately, you can't force Google to crawl webpages. Google uses its own assortment of systems and protocols for crawling and searching/indexing. You can ask Google to crawl pages from webmaster tools but that is the only method to do it. Without access to that, your best bet would be to create pages on your site with links, or to promote more traffic to the pages in other ways. The more traffic a page gets, the more it is crawled, read, and saved. You could even promote your website(s) with Ads on places like Social Networks.

share|improve this answer
add comment

As John Conde suggests, you should make sure all those pages have links to them. If Google is being slow about indexing them you can:

  • Ensure that the page(s) that are linking are popular and have pagerank
  • Make the links prominent on the page
  • Make the pages you are linking from useful for users
  • Put links in additional places
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.