I know that OpenID would benefit more from a larger number of sites adopting it, than it would from more people being providers.
But even still, I am curious as to when it would be good for a site to be the provider.
If you look at it from the standpoint of Return on Investment (ROI) it makes little sense to become an OpenID provider unless you have a massive user base. The value of companies like Google being an OpenID provider is that they get more people to visit their site and more people to tightly integrate and work with them.
This offsets the technical, monetary, and legal expenses that are the cost of being an OpenID provider. You have to remember that you have to build a technical team to develop and manage your OpenID servers, you have to pay a ton of money to buy or rent server that will always be up and can handle the load, and finally OpenID has to be on. If Google's OpenID service goes down people have no access to ALL of the sites that use OpenID. This can come with legal ramifications if it costs someone money and they are below their service agreement up-time levels.
Also, there is the issue of adoption. SO and Stack Exchange only promote certain sites and Companies for OpenID. So unless your site is on that short list people won't realize you are an option and may have more issues using your system unless they are technically savvy. Having a quick link on the SO page to Google has made my life 100 times easier when trying to log in to SO, I know that.
Finally, as others have said, most people are not going to trust any company that is going to close down. Fortune 500 level companies are usually where people put there trust in these kinds of things because even if a company like Google goes bankrupt someone will buy them and continue the service. When you get to small companies there are a ton of issues.
From my standpoint of a user, I expect my OpenID providers to be around a long time. I wouldn't want the service that's allowing me to log into a number of sites to disappear overnight. That's why I use Google and Verisign as my providers (Verisign being my primary simply because they offered it before Google did). I would never use a new upstart service as my provider, unless I was using myself as my provider (and I know people who do that).
Honestly, it just seems like more of a hassle to become a provider than it's worth. Becoming a consumer would be most beneficial at this point. Your target audience is probably a user of at least one of AOL, Google, MySpace, Verisign, Yahoo!, or WordPress. Other people jumped onto MyOpenID or one of the other providers early in the game, too. The world doesn't need more providers, but does need more consumers.
On the other hand, though, if you are competing with any of those companies I named, then perhaps you should become a provider. From a business standpoint, it's a feature that you can check off as "they do this, so we do it as good or better". From a business standpoint, RandomBen's answer to this question provides a better answer than I can. In short, it's all about the Return on Investment.
Actually there is one reason that I can think of, as I am in this situation now:
You open one or more "sister sites" to your main site, and you want users from your main site to be able to login to your partner sites.
So I won't publicize the fact that we're OpenId much, but internally I'll use OpenId.