Google is not indexing the entirety of my site [duplicate]

Question

This question already has an answer here:

• Why aren't search engines indexing my content? 1 answer

I require google to index the entirety of my site: 10 or so basic sections with 80,000+ individual job offer pages. However, to date, only 2,900 pages have been indexed by google.

I'd like to know why... only 2,900 but not 80,000+. I figure, if there were something wrong with my job offer pages, then there wouldn't be those 2,900 values indexed but rather 0...

I have a 'page' sitemap which, link by link, leads to each and every ad (paginated) My XML sitemap is lacking though. So does this mean you absolutely need an XML sitemap when you're dealing with tens of thousands of unique pages?

http://haytrabajo.mx is the URL

http://haytrabajo.mx/mapa is the 'page' sitemap

Answer 1

Here's my generic answer which usually helps: sign up for Google Webmasters and Google Analytics as they may be able to give you some more information.

I'd also generate an XML sitemap with XML-Sitemaps (you could also look at automating generating this file) and upload it to /sitemap.xml - you can tell Google the location of your sitemap in Webmaster tools to aid them in finding your content and it can also show you which of your sitemap pages it has actually crawled.

Answer 2

When I check your site map I did not see that you have 80 000 + pages there, you got 6 categories pages, and that's it where all of your 80000 pages?.

Suggest that you expand the map and even create a unique map for each category and send it to GWT.

In addition, I suggest you build a more professional Footer at the bottom to show Google the site structure and improve the indexing your interior pages. You'll find excellent Footer examples here.

I have a couple more suggestions for you to improve navigation and you want i could expand a little more

Answer 3

The 'false' sitemap found in /mapa did not work. I had to create an XML sitemap using one of those free xml generators and I spread my 80,000 entries into 4 pieces, so that now I have:

sitemap-index.xml which points to: sitemap-1.xml sitemap-2.xml sitemap-3.xml sitemap-4.xml

I then had to submit all these in google webmaster tools and set a custom crawl rate. Now it's just a matter of waiting....and waiting for the whole site to be indexed.