Abstract
This chapter presents an approach to unsupervised digital movie restoration. The approach is based on the idea of recovering the appearance of color instead of the original color signal. The rationale behind this choice is that very often the original color reference is missing in old films, and new films or digital coding can be subject to important gamut transformations. The authors apply algorithms that are designed to reflect the capabilities of the human vision system in automatically adjusting color and lightness variation in the scene.
TopIntroduction
Since 1980, UNESCO has recognized “moving images” as an integral part of world’s Cultural Heritage (UNESCO, 1980). Cinema, since its invention in the 19th century, has in fact become one of the most important media of popular culture. Moreover, it becomes - next to books - our historical memory. However, due to the aging of dyes contained in the emulsion and to the support itself, color films are subject to a natural decay process. This process introduces a color dominant, loss of contrasts and/or color desaturation. This affects in the same way different type of supports: color negative and positive film, color print material, interpositives, and color motion-picture release print (Reilly, 1998). Unfortunately, the decaying process is irreversible and it can be slowed down only by storing photographic and cinematographic material at controlled temperature and humidity (Harrison, 1997).
As stated by the European Broadcasting Union (2001): “Differential dye fading is the most common image degradation phenomenon in archived color films”. Also Martin Scorsese (The Film Foundation, 2013) has raised attention on this problem, stating that “Movies touch our hearts, and awaken our vision, and change the way we see things. They take us to other places. They open doors and minds. Movies are the memories of our lifetime.”.
In (Read & Meyer, 2000), it is estimated that about 80% of the movies produced between 1910 and 1920 have been lost and, if not restored, a large part of the film cultural heritage will be lost in a relatively short time. This urgency calls for strong ethical obligation to preserve and restore those materials that are aimed at representing our social historical memory.
Due to the enormous technical progresses in digital color restoration, this replaced some classic techniques that were based on the use of optical and photochemical tools used directly on the original film material. Those restoration techniques involved a significant human intervention and constant supervision of the work and not all artifacts are treatable by traditional photochemical techniques. Digital techniques allow an easier removal of color casts, equalization of the frame histogram, expansion of the dynamic range and adjustment of desaturated colors. Furthermore, they allow easily multiple attempts and a digital pipeline can address effectively the scaling of the restored content to the different formats and gamuts, necessary for the multiple different visualization devices currently available in cinemas. However, it is important to point out that constant supervision of the entire process by qualified personnel is still required and of fundamental importance.
In this chapter, the authors’ experiences and results obtained so far in the projects conducted in the last years in the digital film restoration domain are illustrated and discussed. The used method is described: it is based on the approach of recovering the appearance of color rather than the original color signals. This because, most of the time, in old films the original reference is missing, and new films or digital coding can be subject to severe gamut transformations.
The adopted approach is based on the application of Spatial Color Algorithms (SCAs) (Rizzi & McCann, 2007), a family of algorithms inspired by the capabilities of the Human Vision System (HVS) of autonomously adjust to the variation of color and lightness in the scene. The digital restoration of color and dynamic is usually a manual or semi-automatic delicate operation performed by experts. What the authors propose is an unsupervised (or eventually, if preferred, a semi-unsupervised) method for digital color, contrast and dynamic restoration. It can be used both as a tool for complete restoration or for an unsupervised kickoff followed by further manual refinements. The advantage of the SCAs algorithms is that they adjust automatically the strength of correction according to the enhancement required by the input.
The chapter is organized as follows. First, three Spatial Color Algorithms - namely, RSR, STRESS, and ACE - are presented and later their use in movie restoration is explained. Then, the proposed approach and related pipeline are presented. Finally, the application of the approach and its results are illustrated and discussed.
Key Terms in this Chapter
Local Linear Look Up Table: An approach to software speed-up in which the spatial color algorithm is applied to a subsampled version of the original image, and in a second step, local interpolation functions are determined in order to propagate the color computation effect to all the pixels of the original image, keeping the local filtering effect.
Automatic Color Equalization (ACE): Algorithm for digital images unsupervised enhancement based on a computational approach that merges the “gray world” and “white patch” equalization mechanisms, while taking into account the spatial distribution of color information.
Digital Movie Restoration: The restoration of movies with the application of digital techniques that allow an easier removal of color casts, equalization of the frame histogram, expansion of the dynamic range, and adjustment of desaturated colors.
Random Spray Retinex (RSR): A simplified reformulation of the MI-retinex algorithm where paths are replaced by two-dimensional point sampling (called sprays) across the image.
Color Appearance Model: The attempt of modeling colors as detected by the human visual system (HVS).
Spatial Color Algorithms: A family of algorithms inspired by the capabilities of the human vision system (HVS) of autonomously adjust to the variation of color and contrast in the scene.
Spatio-Temporal Retinex-Inspired Envelope With Stochastic Sampling (STRESS): Algorithm that uses the same spray approach of the random spray retinex algorithm, but with an alternative technique that exploits two envelopes to characterize the local visual context. It calculates for each pixel not only the local maximum (reference white) like RSR, but also the local minimum (reference black), for each chromatic channel.