0

Let's say I have a listing website with thousands of pages. I don't want to make all those pages(items) crawlable by others, unless they search for those items. But, still I like to have all those pages indexed in google. Please note that those pages do exist, and it is not a broken link, just that there is no direct link to those pages unless one uses search function of the website.

How can I achieve that? One things that comes to my mind is to create a complete sitemap with all pages included and submit it to google for indexing. would it work!? Or google would complain that it cannot follow a path of links from home page to those (unlinked) pages?

Thanks

Improve this question
2
  • 1
    Google will index all the pages within a sitemap without a link to the page, however, for the past couple of years Google has penalized sites with a lot of pages without links delisting those pages in the end.
    – closetnoc
    Nov 4, 2017 at 0:49
  • 1
    Google doesn't usually index pages unless they are well linked. The best way to achieve what you want is to link to them in places that users don't usually use. Things like the sidebar below the fold, bottoms of drop down menus, or footers.
    – Stephen Ostermiller ♦
    Nov 4, 2017 at 10:16

1 Answer 1

Reset to default
0

Besides penalizing, when a visitor could not find a certain route to your pages which aren't defined by category or an index somehow, you may lose some page visits.

Also performing a "site:example.com" search in Google will reveal which URLs are indexed.

In addition, in normal ways of having a sitemap, anyone cloud find the sitemap and use it by typing the address and the filename like "Example.com/sitemap.xml".

With that in mind, if you are looking to protect your data against piracy, you can register your site within the search engines, or adding meta-tags to declare the content's author, or use some tools like Google Alerts, Copyscape, &... that are described here.

And if you want to hide your sitemap there are some suggestions like: Do not use frequent used address/filenames, check the User-Agent of the bots you want to allow &... that are described here.

Improve this answer
1
  • 1
    Using the site:example.com directive shows indexed pages that pass all the SERP filters. For the true index number, Google Search Console gives you the number.
    – closetnoc
    Nov 4, 2017 at 14:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.