Not that difficult,
first you need a good design. If the original visuals are bad you cannot do anything at all.
then you need to know which file format is good for where, PNG for most at the moment, GIF for very low color counts and animations, JPGs for Photos which thumbs PNG as PNG file size becomes too big at photos.
And finally compression of JPGs. Every image should be checked with human eye for exporting with right compression values, for example JPGs are bugged with red if you compress it too much reds in jpgs become very nasty looking.
Good designers with sense of file size still pay attention to compression values but most are now just using PNGs and no compression values for JPG images to avoid such decays in quality.
And also using the right software is essential, Photoshop and Fireworks are 2 most popular ones that handle image exporting pretty decently. For me Fireworks knocks out PS bad for web exporting :)