Sprite generation is the process of aligning, warping, and blending of pixels that belong to an object in a video. The evaluation of the correctness of a sprite is usually accomplished by a combination of objective and subjective evaluations. Availability of ground-truth image would help mere objective evaluation. In this paper, the authors present video generation from an image based on various camera motion parameters to be used as ground-truth for the sprite evaluation. This paper introduces a framework for evaluation of sprite generation algorithms. Experiments under the proposed framework were performed on the synthetic videos of different camera motion patterns to reveal the components of the sprite generation algorithm to be improved.
TopIntroduction
The term “sprite” refers to the composition of pixels that belong to a video object in a video. Sprite generation (Lu, 2003) is the process of generating sprites for objects in videos. The most common object for sprite generation is the background, and the corresponding sprite is usually referred as the background sprite. It was used for video compression under MPEG-4 video standard (Sikora, 1997). Sprite coding is only supported by MPEG-4 Main Profile and requires the availability of the sprites. In MPEG-4, a sprite may be generated for each object, and objects are layered on top of each other. Instead of sending the complete background scene for every frame, sprite coding encodes and transmits static background sprite once. The individual frames can be regenerated from the sprite with the support of motion compensation. Thus, sprite coding leads to good subjective quality with very low bitrates (Jinzenji, 2001).
Sprite generation has been also studied as background extraction (Lai, 2008), photo stitching (Baudisch, 2007; Brown, 2007; Baudisch, 2005; Zomet, 2006), and panoramic image generation (Farin, 2008). Sprite generation research has gained significant attraction since late 1990s due to its application in many domains such as object-based coding (Wantanabe, 2001), video compression (Jinzenji, 2001), video indexing (Grammalidis, 1999), virtual environments (Jaillon, 1994), object tracking (Lin, 2002), and security surveillance (Cheng, 2007). Sprite generation is mostly applied to the background in the video. In the rest of the paper, we will use the term sprite for the background sprite unless stated otherwise. Since background scene may not be captured in a single frame of a video, sprite is generated by correctly overlapping between sequential frames. Sprite is usually generated when there is a significant global motion and the background regions eventually become visible. Therefore, sprite generation algorithms process every frame (usually in order) to incorporate occluded backgrounds as they become visible and newly visible backgrounds. Sprite generation is mostly applied in order sequentially for all frames in the video, since the global motion is limited for consecutive frames. The consecutive frames are aligned and the newer one is blended into the sprite.