1

I have a website that sells 100,000 items. The item data is stored in a mysql database. So, there aren't 100,000 html files on my site. Therefore, google can't see them. I read that the solution to this is to create site maps. I seem to have done that part correctly. On the google search console, now it says 100,000 items discovered. It's been that way for months and says 0 pages indexed. It doesn't say why. I read that the reason for this could be it's just too many items and it's overloading the system. I think it's referred to as "Crawl Budget." I didn't see a solution though. Should I be be submitting SOME site maps, wait for SOME pages to get indexed, then submit more. Seems rather tedious. And how much is too much to submit at once?

Improve this question
4
  • 2
    "The item data is stored in a mysql database. So, there aren't 100,000 html files on my site. Therefore, google can't see them." - Whether your pages are .html files or served from a database has no bearing on Google's ability to crawl and index. Furthermore, site maps are not a solution for forcing Google to index pages. Does your site have 100,000 unique URLs that each return unique pages? If so, Google may simply be refusing to index your site for one of many reasons.
    – Maximillian Laumeister ♦
    Commented Nov 3, 2023 at 18:09
  • @MaximillianLaumeister, maybe I misunderstood you or the question. I interpreted the OP as having a single webpage that could query the database for the products. In this case, I'm not sure how Google would know about 100,000 products without some kind of clue, like a parameter being passed to the page.
    – Trebor
    Commented Nov 4, 2023 at 15:36
  • 2
    @Trebor True. If all products are only available at a single URL, then there would be no way for Google to index them. You'd have to change your site so that each product is available at a different URL, whether that be through rewrites or simply using URL parameters. Then, you'd likely want to add product listing pages so crawlers can discover all those links. Google crawler of course doesn't interact with web pages, so it can't navigate input elements or fill out text boxes - it can only follow links to different URLs. It all has to do with how the site is currently set up.
    – Maximillian Laumeister ♦
    Commented Nov 4, 2023 at 16:55
  • Your question needs clarification for any further help. Do you have a page that accepts parameters to your 100K items? How does your sitemap point to these items?
    – Trebor
    Commented Nov 10, 2023 at 0:57

0

Reset to default