Abstract
This chapter provides a review of Shannon and other entropy measures in evaluating the quality of materials used in perception, cognition, and learning processes. Energy-based metrics are not suitable for cognition, as energy itself does not carry information. Instead, morphological (structural and contextual) metrics as well as entropybased multiscale metrics should be considered in cognitive informatics. Appropriate data and signal transformation processes are defined and discussed in the perceptual framework, followed by various classes of information and entropies suitable for characterization of data, signals, and distortion. Other entropies are also described, including the Rényi generalized entropy spectrum, Kolmogorov complexity measure, Kolmogorov-Sinai entropy, and Prigogine entropy for evolutionary dynamical systems. Although such entropy-based measures are suitable for many signals, they are not sufficient for scale-invariant (fractal and multifractal) signals without corresponding complementary multiscale measures.
TopIntroduction
This chapter is concerned with measuring the quality of various materials used in perception, cognition and evolutionary learning processes. The multimedia materials may include temporal signals such as sound, speech, music, biomedical and telemetry signals, as well as spatial signals such as still images, and spatio-temporal signals such as animation and video. A comprehensive review of the scope of multimedia storage and transmission is presented by Kinsner (2002). Most of such original materials are altered (compressed or enhanced) either to fit the available storage or bandwidth during their transmission, or to enhance perception of the materials. Since the signals may also be contaminated by noise during different stages of their processing and transmission, various denoising techniques must be used to minimize the noise, without affecting the signal itself (Kinsner, 2002). Different classes of coloured and fractal noise are described by Kinsner (1996). The multimedia compression is often lossy in that the signals are altered with respect not only to their redundancy, but also to their cognitive relevancy. Since the signals are presented to humans, cognitive processes must be considered in the development of suitable quality metrics. This chapter describes a very fundamental class of metrics based on entropy, and identifies its usefulness and limitations in the area of cognitive informatics (CI) (Wang, 2002).