A favicon is a small graphic normally 16x16 pixels, though it can be larger, often displayed in the address or tab bar (left of the url) of your browser.
Favicons help give a website it's identitiy. They are often a scaled-down version of the logo. Favicons are normally given in ICON format, as ".ico", but they can be other types of image (ie. GIF or PNG).
When using ".ico" files, these can hold multiple sized images (usually 16x16, 32x32 and 48x48 pixels) for different contexts such as in the browser or on the desktop.