Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a URL pattern that was found by the search engine, I've tried the link directive to find backward links. But to no avail.

So how could I know where / how the search engine found those page?

By search engine I specifically refer to Bing and Google

share|improve this question

migrated from superuser.com Jun 22 '13 at 15:19

This question came from our site for computer enthusiasts and power users.

2  
Welcome to the world of crawlers. A crawler indexes your main page, then follows links leading off that page. –  Nathan C Jun 20 '13 at 11:22
add comment

3 Answers 3

Your best bet would actually be to sign up for a Google Analytics accounts and place tracking cookies on your websites. That way you can see where referring traffic is coming from. The link directive is good but an actual tracking device such as Analytics will show you those that might not necessarily show up in the link searches.

share|improve this answer
add comment

To see which of your pages have been indexed you should use the site operator:

site:www.domain.com

This will return all of the pages Google has crawled and indexed within your site. You can also modify the search to look for certain keywords or urls

The Google reference may be useful

share|improve this answer
add comment

The only time that I have found that Google tells you where it found links to your site, is when the page on your site is 404 Not Found. In that case, Google has a 404 report in Webmaster Tools that includes information about what links to that URL.

If you have a valid page on your site, I've never seen Google tell you this information. The referrer field that Googlebot sends when it crawls is always blank. Google specifically makes the link: directive incomplete and useless.

Bing on the other hand has a inbound links tool available in their webmaster tools. When you log in and validate your site, they will tell you about links they find to your pages.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.