Favicons help give a website it's identitiy. They are often a scaled-down version of the logo. Favicons are normally given in ICON format, as ".ico", but they can be other types of image (ie. GIF or PNG).
When using ".ico" files, these can hold multiple sized images (usually 16x16, 32x32 and 48x48 pixels) for different contexts such as in the browser or on the desktop.