What is Entropy Coding

Only a few decades ago, the human-computer interaction was based on a rudimentary text user interface, a convenient method compared to the punch card era, but too tedious and not very appealing for the nonspecialist, and thereby, not suitable for the mass market. Later on, the multimedia era arrived, with personal computers and other devices having powerful graphic capabilities, plenty of full-coloured pictures shown to the user. Although images made more pleasant the interaction with computers, their use represented a new challenge for electronic engineers; while only a few bytes are needed to represent a text (typically one byte per character in extended ASCII), lots of data must be employed for images that, in a “raw” representation (i.e., uncompressed) for colour images, need as many as three bytes per single pixel (picture element, each dot forming an image). Thus, there was a clear urge to reduce the amount of bytes required to encode an image, mainly so as to avoid an excessive increase in both memory consumption and network bandwidth required to store and transmit images, which would limit or prevent their use in practice. In general, in order to exploit the information system resources in an efficient way when dealing with images, compression is almost mandatory. Fortunately, most images are characterized by highly redundant signals (especially natural and synthetic images), since pixels composing an image present high homogeneity, and this redundancy, often called spatial redundancy, can be reduced through a compression process, achieving a more compact representation.