Adding a new site to a Google Custom Search Engine gives fewer hits

Question

I have a Google Custom Search Engine (CSE) with one "site". Adding a second site gives me fewer hits. Removing the second site gives me back the hits I had with only one site.

What is going on there?

I add the sites with XML to be sure of what I am doing, like below:

<Annotations>
  <Annotation about="site1url" score="1">
    <Label name="_cse_MYID" />
  </Annotation>
  <Annotation about="site2url" score="1">
    <Label name="_cse_MYID" />
  </Annotation>
</Annotations>

UPDATE: Here are instructions for anyone who wants to test:

1. Add these two sites to a new custom search engine (https://www.google.com/cse/): www.zotero.org/groups/from_some_psychologists/items/itemKey/* www.zotero.org/groups/minding_my_mitochondria_terry_wahls/items/itemKey/*

2. Search for food (you will get 2 hits)

3. Now delete the first site from the custom search engine.
4. Search for food again (you will now get 19 hits)

UPDATE 2:

I posted the question also to Google Product Forums > Google Custom Search: https://productforums.google.com/forum/#!category-topic/customsearch/indexing-and-results/8Qs60jdiFZQ

UPDATE 3: I previously thought that the hits when having both sites were always only from the second site. That was wrong (so I removed the part saying so).

Answer 1

This is an issue that seems to date back to at least last yr: Google custom search showing no results. For the example above, I'm getting no results. I've tested "food" with wikipedia.org and get a random number of results.

By random, I mean 0 results or 550k+ results.

---

Follow-up 1: Now I'm really confused...The issues I'm having are with searches done in the Public URL that is provided. If I do the search in the embedded web page under Setup --> Basics (tab), I have no issues. I'll test on my domain and report back.

---

Follow-up 2: The issue for me has to do with using Google's Public URL. When I put the code in my own domain, I have less issues. I don't understand why I get no results when I click "Search the entire web but emphasize included sites."

Answer 2

Please read the discussion in the link in @myidealab's answer. The person named "maitchy" there says the problem probably is that Google aborts the search when it takes too long time at their servers.

maitchy also says it worked better when s/he took out some sites from the CSE search engine definition.

This is exactly what I see too.

So I agree with maitchy. In addition to what maitchy said I have some thoughts about the possible implementation at Google. I suspect they are searching each site in the CSE by itself (technically, of course). They are then adding the time it takes for each site and comparing this with a fixed time limit. Or at least a time limit which does not take the number of sites into full consideration.

That does not work. If there are several sites in the CSE it takes longer time (if my technical guess is correct).

Now, what we end up with is a rather brutal abort (from Google) of the search. This brutal abort tends to make the resulting hit list more or less useless when there are several sites in the CSE.

This is of course not Google's intention. ;-)

That is has been this way for quite some time points to that Google does not get the feedback needed to fix this.

Google have actually prevented feedback from free CSE users! What can we do? Someone suggested trying to give the feedback through the Chrome developers by reporting this as a Chrome bug - which it of course not is. That would be a rather desperate thing to do, but are there alternatives? :-)