I have a URL pattern that was found by the search engine, I've tried the link directive to find backward links. But to no avail.

So how could I know where / how the search engine found those page?

By search engine I specifically refer to Bing and Google

  • 2
    Welcome to the world of crawlers. A crawler indexes your main page, then follows links leading off that page.
    – Nathan C
    Jun 20 '13 at 11:22

Your best bet would actually be to sign up for a Google Analytics accounts and place tracking cookies on your websites. That way you can see where referring traffic is coming from. The link directive is good but an actual tracking device such as Analytics will show you those that might not necessarily show up in the link searches.


To see which of your pages have been indexed you should use the site operator:


This will return all of the pages Google has crawled and indexed within your site. You can also modify the search to look for certain keywords or urls

The Google reference may be useful


The only time that I have found that Google tells you where it found links to your site, is when the page on your site is 404 Not Found. In that case, Google has a 404 report in Webmaster Tools that includes information about what links to that URL.

If you have a valid page on your site, I've never seen Google tell you this information. The referrer field that Googlebot sends when it crawls is always blank. Google specifically makes the link: directive incomplete and useless.

Bing on the other hand has a inbound links tool available in their webmaster tools. When you log in and validate your site, they will tell you about links they find to your pages.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.