I am reaching out for some guidance regarding the Google Search Console (GSC) indexing of my website. I launched this site about a month ago, and I'm encountering some challenges with Google's indexing process.
Background:
Nature of Website: My website connects to a database created by a scraper that collects public company information, making it easier for users to search through these listings.
Technical Specs:
Scraper: Built in Python
Database: SQL
CMS: WordPress
Total Listings: Approximately 570,000 company listings/pages
Sitemap Structure: Sitemap contaning 570 sitemaps, each containing 1,000 pages
Current Indexing Status: Google Indexing Rate Started with 13 pages every 3 days, gradually increased to 100 pages every 3 days GSC Sitemap Recognition: Only 11,000 pages are recognized out of 570,000
Challenges and Queries:
Indexing Rate: The indexing started slowly and has only marginally improved. Given the size of my website, is this a normal progression, or could there be technical issues hindering the process?
Time Factor: Should I simply wait for Google to increase indexing rate or are there strategies I could implement to expedite this process?
GSC Sitemap Limitation: Why does GSC recognize only 11,000 out of the 570,000 pages listed in my sitemaps? Could this indicate a potential issue with how my sitemaps are structured or submitted?
The site is 100% legitimate and not a spam site. It's designed to provide a valuable resource by enhancing the accessibility of public company information.
I would greatly appreciate any insights, suggestions, or experiences you can share, especially if you have dealt with similar challenges in indexing large websites.
Thank you in advance for your time and assistance!
