0

I work on an exotic pet website which currently has several types/species of reptiles. It has done well in SERP for the first couple of types of reptiles, but I am continuing to add new species and for each of these comes the task of getting ranked and I need to figure out the best process.

We just released our 4th species, "reticulated pythons", about 2 weeks ago, and I made these pages public and in Webmaster tools did a "Fetch as Google" and index page and child pages for this page: http://www.morphmarket.com/c/reptiles/pythons/reticulated-pythons/index

While Google immediately indexed the index page, it did not really index the couple of dozen pages linked from this page despite me checking the option to crawl child pages. I know this by two ways: first, in Google Webmaster Tools, if I look at Search Analytics and Pages filtered by "retic", there are only 2 listed. This at least tells me it's not showing these pages to users. More directly though, if I look at Google search for "site:morphmarket.com/c/reptiles/pythons/reticulated-pythons" there are only 7 pages indexed.

Any advice on what could be going wrong here. I really want Google to index the top couple of links on this page (home, index, stores, calculator) as well as the couple dozen gene/tag links below.

Update about 1 Month Later:

I'm now seeing pages getting indexed like I had hoped. It's hard to tell if submitting the URLs even helped, since Google at this point could have been picking it up on its own.

Improve this question
3
  • Fetch as Google will only fetch one page per online request. Fetch and Render is only a temporary thing. The regular googlebot will come by in 1-2 days and fetch the page and make it official at that point. If you say that you want Google to fetch "the linked pages", it will when the regular googlebot visits and not when you do the Fetch and Render. Cheers!!
    – closetnoc
    Commented Nov 24, 2016 at 23:38
  • Okay, but it's been 2 weeks. Google bot has come many times by now. It does not appear to be fetching the linked pages, is my point. Any ideas? Thanks.
    – John Lehmann
    Commented Nov 25, 2016 at 1:45
  • 1
    It will. Google works on it's own schedule, no ours. ;-) If you try and hurry G up, it gets mad. Even vindictive sometimes. I have learned that patience is the best tool in the box. Cheers!!
    – closetnoc
    Commented Nov 25, 2016 at 4:52

1 Answer 1

Reset to default
0

Have you checked "Crawl this URL and its direct links" on submit option?

enter image description here

You can do that only 10 times per month. And additionally you can submit 500 single URL in fetch and render tools.

What I want to say is, if there are more links on that page, then Google will not crawl and index all the pages on same time/day, They will schedule their works, your links are queue in cralwer pipeline, you just have to wait, there are thousand of people submit URLs to Google and crawler have to crawl only specific number of pages on certain time.

Improve this answer
4
  • Yes, that is what I had done. It is just so slow it's hard to tell if that was even helpful.
    – John Lehmann
    Commented Dec 19, 2016 at 13:40
  • Actually it's purely depend on how much other webmaster request same thing in one day. If there are too many then your URL's will be on queue on Google crawler.
    – Goyllo
    Commented Dec 19, 2016 at 18:53
  • I appreciate your help. Do you have any references or is this just speculation?
    – John Lehmann
    Commented Dec 19, 2016 at 19:30
  • Google said that many times on official hangout of Google webmaster, but have not written any article about that but my blog post on crawling might helpful.
    – Goyllo
    Commented Dec 20, 2016 at 6:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.