I have a hypothetical page, that contains pages like:
example.com/see/cat
example.com/see/mouse
example.com/see/<basically anything>
I can have thousands of such pages. The number of pages can increase slightly day from day, however, the existing pages should not change drastically too often. There will be no index of those pages (to limit exploitation/scrapping of the data on pages). Users should be able to find the page only with search on this website, or preferably with Google (or other search engine).
What is the best practice to tell google or other search engines about these pages and their content, so they can offer them in search results?
Is dynamic generation of sitemaps enough? Is there some advanced stuff about this topic I should study?
Note: I realized that if I dynamically create sitemap on the fly for everyone who want it, than the whole "knowledge" in the site is scrapable/exploitable by anyone. So now I need to add second question: Is there any standard way how to authorize only friendly crawlers?
