The fundamental principles of the coding/decoding H.264/AVC standard are introduced emphasizing the role of motion estimation and motion compensation (MC) in error concealment using intra- and inter-frame motion estimates, along with other features such as the integer transform, quantization options, entropy coding possibilities, deblocking filter, among other provisions. Efficient MC is one of the certain reasons for H.264/AVC superior performance compared to its antecedents. The H.264/AVC has selective intra-prediction and optimized inter-prediction methods to reduce temporal and spatial redundancy more efficiently. Motion compensation/prediction using variable block sizes and directional intra-prediction to choose the adequate modes help decide the best coding. Unfortunately, motion treatment is a computationally-demanding component of a video codec. The H.264/AVC standard has solved problems its predecessors faced when it comes to image quality and coding efficiency, but many of its advantages require an increase in computing complexity.
TopIntroduction
Because 3D video has gotten crescent importance in multimedia and there are lots of legacy 2D videos, both 2D+depth and the H.264/AVC standard still need attention. Most multimedia data streams contain audio, video, and some metadata, but for these streams to be useful in stored or transmitted form, they must be encapsulated together in a container format.
To offer better video quality and more flexibility compared to previous standards, the H.264/AVC (Wiegand et al., 2003) video coding standard has been devised by the Joint Video Team (JVT) to deliver significant efficiency, simple syntax specifications, and seamless integration of video coding into all current protocols and multiplex architectures. H.264/AVC supports several video applications like broadcasting, streaming, and conferencing over fixed and wireless networks with different transport protocols.
Error Concealment (EC) involves recreating lost video data using already received information. Due to redundancy in both the spatial and temporal domains, the lost data can be estimated from existing information via Motion Estimation (ME), also known as Motion Prediction. The main types of EC include the following techniques: spatial (intra-frame), temporal (inter-frame) and hybrid. The last technique is a combination of the spatial and temporal strategies (Fleury et al., 2013).
Efficient Motion Compensation (MC) is one of the key reasons for the H.264/AVC superior performance compared to its predecessors. Unfortunately, motion estimation/compensation (MEMC) is the most computationally-intensive part of a video encoder.
A video is organized as a sequence of frames, where each frame is an image consisting of pixels. The H.264/AVC divides every frame into several Macroblocks (MBs). A MB is a Processing Unit (PU) in video compression formats relying on linear block transforms, such as the Discrete Cosine Transform (DCT). The H.264/AVC main profile supports a 4×4 transform block size while its high profile allows for a transform block size of either 4×4 or 8×8, tailored on a per-MB basis (ITU-T, 2013). The MBs are grouped into partitions called slices.
There are three different types of frames:
- •
Intra-picture frames (I-frames),
- •
Unidirectional predicted frames (P-frames), and
- •
Bidirectional predicted frames (B-frames).
These frames form a sequence called a Group of Pictures (GOP) as shown in Figure 1.
In preliminary standards such as H.261, MPEG-1 Part 2, and H.262/MPEG-2 Part 2, MC is performed with one Motion Vector (MV) per MB. Diversely from the division into transform blocks, an MB can be split into prediction blocks with multiple variable-sized prediction blocks (partitions) in the H.264/AVC.
The H.264/AVC videos coding layer design is based on the concepts of Motion Compensated Prediction (MCP) and transform coding. Within this framework, some important functionalities deserve mention such as:
- •
Intra-prediction in spatial domain,
- •
Hierarchical transform with (4×4, 8×8) integer DCT transforms,
- •
Multiple reference pictures in inter-prediction,
- •
Generalized bidirectional prediction,
- •
Weighted prediction,
- •
Deblocking filter,
- •
Context-Based Adaptive Variable Length Coding (CAVLC)
- •
Context Adaptive Binary Arithmetic Coding (CABAC),
- •
Entropy coding, and
- •
Parameter setting.