Artlung covered some support considerations, but there's also the fact that it's just not going to matter most of the time. The important bit in the question is "until it's fully loaded." The progressive format isn't an aesthetic nicety that makes the image blur in, it's functional:
If an image is large enough(by filesize) that it take a while to download, then you'll see the progression.
Even if every JPEG everywhere were saved as progressive, at current common connection speeds the overwhelming majority of images you're going to encounter just aren't large enough that you'd ever see the effect to any significant degree. This was a great idea back when everybody was on slow phone modems, but is increasingly a curiosity of limited practical application beyond sites that really do require very large images like a picky photographer or something.
The same goes for interlaced GIFs, since someone tacked them on in the comments.
There's a potential audience factor here, say if you consider countries with a less-developed net infrastructure where they may get significantly slower speeds, but I can't speak to that. I could see them as possibly having some application for mobile browsing, but then we loop back to the spotty support for the feature.
