I have looked around a lot myself for this answer and if Bing or Yahoo does take it into account then they have not release to the public that they do.
Also, take a look at http://www.free-seo-news.com/newsletter416.htm. Google has stated that adding in page speed will change less than 1% of search results.
Google uses over 200 criteria to return results for pages. Page speed is probably one of the most minor.
In general you will lose more visitors because your page is to slow and people will leave, rather than Google reducing your ranking because of how slow your site is.