5

It's a question I wouldn't have asked 10 or 15 years ago but now I notice it's being generalized... at least on the search engines I'm using or have tested, namely Google, DuckDuckGo, Yahoo and Bing.

I grew up with the known fact that a search engine by default performs a lookup based on all of the terms you put in a query. More than a decade ago, I could easily verify it, i.e. there was more than 95% chances results included all of the terms I queried.

Nowadays it's become rather trivial to demonstrate results do not include all of the terms but only some of them. FTR the first time I've raised the issue by my team I was first laughed at... until I demonstrated was I was talking about.

The usual counter-argument I was exposed with is to add the boolean operator "AND". It's easy to demonstrate that it doesn't change anything. Neither does an advanced search using the "Find all terms" input box (Google)... just because search engines, by definition, are supposed to use all of the query terms by default. But most often than not only part of the terms appear in the results. It's rather easy to spot results that do miss at least one term.

Examples

Of course in these cases the terms do exist in non-visible text elements (e.g. hyperlinks). However I did happen to find results that lack at least one term even from the HTML source (no example to give at the moment, sorry).

It becomes even more obvious with more search terms. The more terms the more probable it becomes to get "irrelevant" results. The consequence, at least on my own, is I stopped a long ago trusting the first result and often wade past the first, second and even third page of results.

Even though one could argue terms may still be found in non-visible parts of a page, the consequence is an ever decreasing trust in the results. I have noticed I happen to not find what I'm looking for even after the third results page. And not because I mistyped (which is also another annoyance of crawling engines, thinking I mistyped and spontaneously correcting my spelling, not even bothering to ask "Did you mean xxx?").

Why do search engines lie?

Improve this question
6
  • 2
    They do not lie imo - at least google tells me next to a search result that a specific word is missing. Search engines perform a lookup based on all of the terms. But if they can't find results with all of them, it is better to show results which could help me rather than showing nothing.
    – Marvin
    Jun 21, 2015 at 10:26
  • 1
    Well I for one disagree with that policy, if that's the one in place of course. I'd rather have no results than incorrect ones. Unfortunately that behaviour doesn't fit everybody and there are cases where it is useful, others not, but no way to choose which one to apply. It's especially bad usability case given that the "search for all terms" clause is misleading in most situations.
    – user53437
    Jun 21, 2015 at 10:32
  • 2
    FWIW I see the same result for your first example. "Android" only appears in the menu - a link to another page - which does indeed seem irrelevant. However, if in-bound links to this page contain the keyword then that could also sway results. Sometimes there can be a mismatch with the search results (the page that is "cached") and the live page, if the page has recently change. However, that does not seem to be the case in this instance, although can explain instances when the "keyword" does not appear at all and it is not explicitly stated as "Missing".
    – MrWhite
    Jun 21, 2015 at 13:58
  • 1
    It goes beyond links and HTML. It is about semantics. One simple example: I have a page on my site that has several other pages linking to it. My page and none of the links contain the word android, however, more than half of the pages that link to my page from other sites and a few on my site do. This is a semantic signal. The term is simply used in content. Better yet, the term is used near a few of the links to my page giving yet an even stronger signal. So in other words, a page does not have to be fully on-topic to appear in the SERPs, however, the semantic link map must be on-topic.
    – closetnoc
    Jun 21, 2015 at 15:06
  • 1
    Search engines don't work the way they used to 10 to 15 years ago. Also, don't log in with an account on the service to avoid "learning" enhanced results.
    – Fiasco Labs
    Jun 22, 2015 at 5:59

1 Answer 1

Reset to default
5

"They" [that means google, everyone else mirrors the methods hoping to mirror the success] are factoring in two things (quite interconnected to each other actually):

  1. Users' true intent: sometimes relevant results may not contain some of the terms put in the query, so they run a sort of fuzzy logic. In your example, maybe they thought that even if you searched for some android gadget, you might happily settle with it's windows alternative ;)

  2. Overoptimization/spam: Content creators sometimes focus on having all relevant keywords in place rather than fine info, so to keep the search results relevant search engines sometimes judge that dismissing some of the keywords is the way out.

To bybass that, there's always the "advanced" toolset - using + until a couple of years back, quotes, choosing to display the "literal" results etc.

Improve this answer
1
  • 1. makes some sense indeed. As for 2., I wish there were an option to select between "strict" and "lax" results.
    – user53437
    Jun 22, 2015 at 9:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy