BMP, TIFF, GIF, JPEG – Bitmap Standards
The Bitmap Standards – BMP, TIFF, GIF and JPEGTom Arah looks at the four main bitmap file standards – BMP, TIFF, GIF and JPEG.
When you come to save a bitmap file the sheer number of file types on offer is daunting. What exactly is the difference between a PNG, PCD, PSD and PDF and does it matter? In fact the question is largely irrelevant, at least as you work on your image, as you should always save to your editor’s native format to ensure that you aren’t throwing away information. However images are different from most other files, such as spreadsheets and databases, in that the data they contain usually can’t remain specific to one application – after all an image only makes sense when it is seen by others. This is where the various bitmap file standards come into play enabling the reliable exchange of graphical information between different applications and different media. But don’t panic – your Save As dialog might bombard you with twenty or thirty alternatives, but you really only need to know about four.
Before looking at these in detail, it’s worth going back to basics and asking just what is a bitmap in the first place. Essentially a bitmap is an array of pixel values. At its most basic a simple black and white bitmap of a circle, for example, is just a grid of squares or pixels where each pixel is either on or off, black or white. The simplest file format necessary to describe this requires a file header with a few fixed fields defining such essentials as the width and height of the grid and then, below this, the image data can be stored as a stream of single bits with 0 representing white and 1 representing black.
Zooming in on a simple bitmap shows its essential nature as a grid of pixel values.
With black and white printers and monitors this was all that was required of bitmap formats, such as Microsoft Paint’s MSP, but the advent of colour monitors changed all that. With CGA, EGA and then VGA screens capable of showing 16, 64 and 256 different colours, the simple on-off bitmap was no longer adequate. The solution was simple enough – increase the bit-depth. Rather than storing one pixel value per bit, you could simply allocate more bits per pixel. With two bits you could define four colours, with three you could define eight and by the time you reached 8-bits you had the necessary 256. Various bitmap standards emerged to support the new display-based demands, such as PCX, but the most important survivor today is Microsoft’s BMP, Windows’ native bitmap format.
What made the BMP format successful was its support for indexing through 4 and 8-bit palettes. Rather than storing each of the necessary RGB values for each pixel, Microsoft instead added first a fixed and then a customisable palette to its BMP format. Each pixel’s colour could then be defined by storing its associated index number rather than the much longer RGB value. This look-up table approach is inherently more efficient for handling images with up to 256 colours as each pixel can be stored in eight bits of information rather than twenty four. Microsoft was able to further optimise the BMP format by tying it even more closely to Windows’ API, adding features such as control over which colours to dither when images are viewed on limited colour displays.
The BMP’s indexed format and basic compression is built for fast screen display
Such indexed bitmaps are ideal for images with limited numbers of colours, but the advantage eventually turns into a disadvantage. In particular, when 24-bit displays and output devices appeared capable of producing photographic continuous tone colour, to index each colour you would have to have a palette for over 16 million colours. Even the smallest 24-bit indexed image would require a massive in-built palette and each indexed entry would be no smaller than the original RGB value. The obvious solution is simply to give up on indexing and store the actual RGB values. This can be done in planar fashion so that the Red, Green and Blue components are stored separately – RRRRGGGGBBBB – but that means that you don’t actually find out any final pixel value until you have read two thirds of the file. A much simpler option is to store the information as rows or scan lines – RGBRBGRGBRGB – and this is the option that the vast majority of bitmap formats have adopted.
The Windows 2.x BMP format added support for 24-bit RGB colour in this way and then the Windows 3.x version added basic compression and the Windows 95 version added colour management but, even so, BMP never took off as a universal 24-bit true colour standard. The problem is that its efficiency at reading data and getting it onto the screen as quickly as possible makes it far less efficient when it comes to more demanding tasks such as handling photographs and print. Its compression scheme is typical. The only compression BMP offers is Run Length Encoding (RLE) which works by taking repeated sequences and expressing them in two bytes so that three hundred white pixels in a row can be represented as 300W. That works fine for images such as screen shots but for the continuous tones of scanned photographs, for example, it is pretty much useless.
BMP remains important for fast and efficient screen display, but another standard for print-based work was clearly required for the burgeoning computer-based publishing industry. The format that came to fill this role was the TIFF (tag image file format). What made the TIFF so different was its tag-based file structure. Where the BMP is built on a fixed header with fixed fields followed by sequential data, the TIFF has a much more flexible structure. At the beginning of each TIFF is a simple 8-byte header that points to the position of the first Image File Directory (IFD) tag. This IFD can be of any length and contain any number of other tags enabling completely customised headers to be produced. The IFD also acts as a road map to where image data is stored in the file as the tagged nature of the format means that this needn’t be stored sequentially. Finally the IFD can also point to another IFD as each TIFF can contain multiple subfiles.
The TIFF’s flexible tagged nature has a whole host of benefits. On the structural level, the fact that multiple images can be contained in the one file is useful when it comes to storing alpha channels. Much more fundamental is the way that data isn’t stored scan line by scan line but is instead broken up into tagged strips of multiple scan lines. This doesn’t sound that revolutionary but it’s crucial for handling the large files necessary for print work as it allows for easy buffering and random access. In particular it means that, to load just the bottom of an image, the whole file needn’t be read and held in memory. Other tags – and there are now over seventy public and any number of customised private tags – are used for defining such features as the colour mode of the file and the choice of compression method.
The TIFF format’s strength is its print-oriented flexibility through features such as tiling, multiple colour spaces and LZW compression.
Originally, like BMP, TIFF only offered a variation on RLE compression called PackBits which is only efficient for indexed images. With the 1988 release of TIFF version 5, however, optional support for LZW (Lempel-Ziv-Welch) compression was added. LZW is a substitutional or dictionary-based encoding algorithm which effectively creates its own look-up dictionary on the fly. Patterns of data are identified in the data stream and matched to entries in the dictionary and, if there is no existing entry, a code phrase based on the substring is automatically added to the dictionary. The easiest way to understand the difference to RLE is if you think about encoding this article. RLE encoding would only be able to compress consecutively repeated letters whereas LZW would be able to compress repeated combinations of letters wherever they appeared.
The flexibility of TIFFs was enhanced further with the 1992 release of the current version 6 specification. This added support for tile-based storage, where the image data is split into columns as well as rows, but more importantly it added support for new colour spaces. The new tags allowed both for 16-bit colour channels and also two completely new non-RGB colour spaces. The first of these, YCbCr (more familiar as LAB), allows colours to be stored in a device-independent format. The second is even more beneficial as it allows pixel values to be stored as 32-bits made up of four channels – the ubiquitous CMYK format so essential for print work.
The TIFF’s flexibility is undoubtedly the secret of its success, but it is also its major weakness. The first problem is that there are just so many “flavours” of TIFF. If your application doesn’t support LZW compression, for example, it will throw up a “bad TIFF” report. In fact the limitation isn’t in the TIFF itself but in your application. Perhaps the most common problem of all is the inability to read Mac-produced TIFFs on the PC and vice versa because of the byte-order in which data is stored. In fact the very first field in the TIFF image file header is the byte-order identifier so really the programmers have no excuse. Even so, it has to be a criticism of the most important print-based bitmap file standard that many programs refuse to open its files or render them as onscreen garbage! The only reliable solution is to stick to professional applications that cannot afford not to fully support the full specification.
The TIFF’s flexible nature has a further drawback in that its tag-based structure inevitably adds an overhead to each file. For print work that’s a price well worth paying, but it’s not acceptable for Web work where every byte counts. The main format that has come to dominate here is the GIF (graphic interchange format) thanks to its combination of palette-based indexing and LZW compression. GIFs can contain a maximum of 256 colours with an 8-bit palette though, unlike other formats, takes full advantage of storing fewer colours at lower bit-depths as well. The fact that the number of colours is restricted automatically suits the typical flat-colour GIF for LZW compression, especially as the system has been tweaked. With the TIFF format’s data packing each byte can represent single, multiple or fractional pixel values which means that repetitions are much less likely to occur so reducing compression rates. With GIFs, the LZW dictionary can automatically be initialized before compression begins with between two and 256 symbols depending on the chosen bit-depth. The end result is that for a typical 8-bit image, GIFs can be losslessly compressed by 40% or more.
Before saving to GIF, the colours in an image must be restricted to 256 or fewer colours.
Apart from this minimised file size, the GIF format offers a number of other optimisations for browser-based display. The use of interlacing, for example, means that all even scan lines are stored first and then the odd rows are split into three so that the image appears in four passes in a venetian blind effect. The advantage is that the viewer can interpret the whole image after only 50% of the data has downloaded. The GIF89a release took things further with the addition of various control extensions. Flat transparency was relatively easy to add through palette-based indexing while control over overlaying and disposal method meant that the multiple images that the GIF87a format could already store could now be turned into a slideshow. Suddenly the static rectangular GIF could become an integral and animated part of your Web page.
However it wasn’t all good news. In December 1994 Compuserve, the format’s original developers, announced that they had entered into a licence agreement with Unisys for the use of the LZW compression/decompression system and that users and creators of GIF should now enter into a licensing agreement with them to pay royalties. The result was panic in the graphics community as it seemed that GIF was being removed from the public domain. This gave a major impetus to the development of a rival standard, PNG (portable network graphics), which at one time looked as if it might topple GIF from its pedestal. Eventually both Unisys and Compuserve back-pedalled by lowering fees and making them only applicable to programs released after 1995. Now, with no question of the end user being charged apart from when they purchase their originating software, the GIF has been saved.
Or rather it’s been saved for the moment as, in many ways, the format is still under threat. The problem is that GIF by its nature falls between two stools. On the one hand its palette-based nature is really only optimised for images based on flat areas of colour, which would in fact be better handled through a vector format like the forthcoming scalable vector graphic format, SVG (see issue 61). On the other hand, for continuous tone photographic images, where pixel-based handling is the only option, the format’s 256-colour limit and LZW compression are hopelessly unsuitable resulting in both poor image quality and large files. In fact it was to solve just this problem that the fourth of the crucial bitmap standards, JPEG, was developed back in 1987 by the Joint Photographic Experts Group who gave it its name.
What made the JPEG completely different was the acceptance that its compression could be “lossy”, that is that information could be thrown away. Of course the trick was to make this loss as imperceptible as possible and the result was a five-stage process. To begin with the image is converted to a colour space (LAB again) where colour and luminance are handled separately. The two chrominance channels are then downsampled as the eye is not as attuned to changes in colour as to changes in brightness. In this way pixel data can be cut by 50% with almost no perceived loss of quality. Next the image is broken into 8×8 tiles that are transformed through a Discrete Cosine Transformation (DCT) into a mathematical equation representing the relevant brightness and colour values. The high frequency information can then again be discarded without the eye noticing. Each block is then quantized according to the user-set, 1-100 quality setting. Finally the resulting co-efficients are losslessly compressed using Huffman compression. The end result is a compression ratio of up to 25:1 with very little perceived loss of quality on continuous tone files.
The JPEG’s lossy compression becomes noticeable at high compression settings.
In fact, like TIFF, JPEG also offers its own flavours through the use of extensions. Variations include a version that stores multiple versions of the same image at different resolutions in a hierarchical format and another that uses an entirely different Predictive Lossless Coding rather than DCT. Much the most important extension beyond the baseline format, however, is the Progressive option. This stores multiple versions of the same file at different quality settings. The lowest quality image is read first which means that a blurred image displays quickly and then appears to come into focus. It can be useful for large images, but unlike interlacing, it means that it actually takes longer for the final image to appear – and depends on your browser supporting the feature.
Generally it’s probably wiser to stick with the baseline specification, sometimes referred to as the JPEG File Interchange Format (JFIF). Or rather of course you shouldn’t stick with JPEG at all. It’s easy to forget, but every time that you save to JPEG you are throwing away more information so you should never work on an existing JPEG image, but should immediately convert it to your editor’s own proprietary format instead. Just as with the other bitmap standards, JPEG’s greatest strength, its lossy compression, turns into its greatest limitation and its Achilles’ heel.
In a way that’s inevitable. Ultimately it’s important to recognise that the four bitmap giants BMP, TIFF GIF and JPEG are powerful, but only as exchange formats. Each format is so targeted at a particular function – screen, print, flat colour Web and continuous tone Web – that outside this narrow function huge shortcomings are unavoidable. As such, while it is true that these are the four standards you really need to know about, I’m afraid that they aren’t the be-all and end-all. If you want to move above standard class you do still need to know about the likes of PNG, PCD, PSD and PDF after all. But that’s another story and a future article.