I have seen evidence that Google relies heavily on click through rate when determining the position of a page in the search results. If they put a site in position #1 but nobody clicks on it, it is certainly not going to stay there. Similarly if they try it out the whole way down at #10 but people find it there anyway, it will rise up the page.
To make this work, Google has to figure out a set of pages to test for the query to begin with. Then it has to get enough people searching for that term that it can gather data about the sites it has listed. Then there has to be a big enough difference between the click through rates to be statistically significant.
Click through rate ends up coming into play the most for queries with navigational intent. That is, the user is searching for a site, brand, or page. In that case, one result will get a better than expected click through rate no matter where in the results it lands initially. That is how Google so accurately gets the "official" site when you search for a brand.
It is also how Google determines whether the listing deserves "site links". Site links only show up when you search for something that has a very high click through rate for a specific search result.
As for "#1 ranking factor", I'm not sure. I'd certainly put it in the top ten overall. For the brand queries, it is clearly 75%+ of the algorithm, but CTR won't come into play at all for:
- Tail term searches
- New sites without click history
In those cases Google has to rely on other signals.
When Google doesn't have click history, I've seen case where they will "try a site out" higher up in the results for a while. I've monitored the search results for queries relevant to my sites. Occasionally I'll see the ranking improve for a day. If the site gets clicks, it may stay there (or at least remain higher than it used to be.)
Google relying on click through rate is even more plausible when you consider how their advertising system works. Advertisers on Google are ordered by multiplying how much they are willing to pay by their click through rate. This algorithm ensures both that higher quality ads rise to the top and that Google maximizes the amount of money that it can make per page view. (It's a bit more complicated now, Google has replaced straight click through rate with "quality score" which is a blend of factors dominated by click through rate.) Since this system works so well for ads, Google would be silly not use click through rate to some extent in the organic listings as well.
There is another ranking factor based on searcher behavior that I believe that Google relies on heavily: bounce back rate. That is the rate at which searchers back off a result and click on something else or refine their query. Google even has the technology to tell this type of behavior apart from users that open multiple search results in new tabs. Google used to show "block this site" links when you used the back button (but not when you opened multiple in tabs). Here is a previous answer I wrote where I go into detail about the evidence I have for Google using bounce back rate as a strong signal.