A Biologically Inspired Smart Camera for Use in Surveillance Applications

Introduction

Flying insects have extraordinary visual capabilities that allow them to navigate in cluttered environments without collisions, perform spectacular aerobatic manoeuvres (Land & Collett, 1974) and discriminate the motion of visual targets camouflaged within complex background textures (visual clutter) (Nordström, Barnett, & O'Carroll, 2006). These are all challenging tasks for artificial vision systems that have attracted substantial attention from scientists and engineers.

One such challenge is discerning detail in scenes with widely varying lighting levels, given that typically only 8-bit (256 level) luminance is usually available in digital imaging despite the huge range of possible luminance levels that natural lighting conditions provide (range in the order of 10⁸) (Brinkworth, Mah, & O'Carroll, 2007). This presents itself as a limitation on the information content, by which we mean, specifically, the ability to distinguish objects within a scene under difficult lighting conditions. Many conventional camera systems tend to have poor performance when capturing images of scenes with complex lighting conditions. As illustrated in Figure 1 traditional camera systems tend to either struggle to capture details in the darkest or brightest parts of the scene, and occasionally in both. The reason for this is that these conventional cameras generally use a relatively simple global adjustment to the luminance on each frame as a whole, in an attempt to improve the final image output quality. However, in some cases the resulting image quality is poor due to a compromise needing to be reached between information in the bright and dark areas.

Figure 1.

Images captured with a conventional camera using different exposure settings and global gain adjustments. Left) short exposure time which lets the bright background be seen but misses the darker foreground. Right) long exposure time which over-exposes the background saturating it and resulting in a loss of information, however more foreground detail is present. Traditionally camera operators had to decide which one of these modes they would operate in due to the limitations of the image capturing and processing technology.

High dynamic range (HDR) imaging processes allow capture over a much larger luminance range than traditional low dynamic range (LDR or 8-bit), however such images are not supported by the majority of image display and storage media (Seetzen et al., 2004). There are conventional engineering solutions, such as gamma correction and tone mapping (Pattanaik & Yee, 2002; Reinhard, Stark, Shirley, & Ferwerda, 2002), however they tend to be focused on maintaining the psychophysical look of an image to a human observer rather than maximising the information contained within the scene. There are also custom CMOS sensors (Pixim, 2006) that approach the problem differently. By using a different integration time (shutter speed) for every pixel in the image they compress the image before it leaves the camera. This means it is possible to capture information from both the dark and light parts of a scene in a low dynamic range format that is more suitable for transmission, storage and display. While this approach can result in an improvement in the information content in a captured scene it is only part of the solution that exists in biology.

Here we consider a biologically-inspired approach based on photoreceptor cells, whose purpose is precisely luminance range compression (Laughlin & Weckström, 1993) while maintaining information content. As with the custom sensors by Pixim the integration time of each receptor in biological eyes does change (Payne & Howard, 1981) independently of the global illumination (Matic & Laughlin, 1981); a large number of cascaded non-linear adaptive processes also occur in order to maximise the information content and reduce the bandwidth of the signal (de Ruyter van Steveninck & Laughlin, 1996; Laughlin, de Ruyter van Steveninck, & Anderson, 1998).