Quality of Service (QoS) and Quality of Experience (QoE) are two different approaches that evaluate the quality level of a given service. In general, QoS models analyze objective network parameters, while QoE techniques focus on what the user actually perceives when consuming the service (subjective perspective). In this chapter, it is presented a comparative study of two voice codecs employed by two well-known VoIP applications, namely Skype and Jitsi, analyzing the provided quality from two perspectives: objective and subjective evaluations. To this end, several network metrics have been taken into account; in addition, the participants in this study completed different quality surveys in order to obtain their opinion about the evaluated services. Concretely, 60 quality tests with 60 subjects have been performed in a controlled wired scenario. The obtained results suggest a clear relationship between QoS and QoE. This chapter aims to present a detailed description about the process to perform comparative quality evaluations of multimedia services from both QoS and QoE perspectives.
TopIntroduction
Voice over Internet Protocol (VoIP) is a widespread technology that has received great attention due to the numerous benefits that delivers to end-users, e.g., low-cost calls, multiple-device accessing, etc. Whereas the traditional public telephone service, a.k.a. Public Switched Telephone Network (PSTN), transports the analogue voice signal through a twisted pair cable within one unique stream, the VoIP technology fragments the voice signal into chunks ready to be delivered through packet-switched networks, such as the Internet. Thus, it is expected that VoIP will widely replace the current residential telephone service by means of protocols allowing real-time multimedia communications, being able to offer new attractive services to its users. Such services can be phone-number portability, no geographical limits, integration with other applications, etc. However, the unreliable nature of the IP network-level protocol, which only provides best-effort delivery, poses some issues to deploy a VoIP service with the sufficient quality level demanded by users. Mainly, the impairments introduced by the network, among other factors, cause a notable degradation in the Quality of the Service (QoS), which has motivated the still slowly ongoing market penetration of VoIP (TelcoBridges, n.d.).
The QoS term refers to the capacity that a system presents to ensure the accomplishment of the minimum quality requirements for a given service. These requirements are usually contained in Service Level Agreements (SLA). From a technical perspective, QoS is assessed by monitoring some key parameters that allow extracting information related to the level of quality provided by the corresponding service. Therefore, the integration of both voice and data flows through a shared network involves some requirements that must be considered during the design of a multimedia communication system. Among others, crucial metrics are: the required bandwidth, the voice-packets delay and jitter, as well as the Packet Loss Rate (PLR). Such metrics should be considered in order to achieve a good system’s performance.
Nowadays, due to the uprising involvement of users in the forthcoming Internet, for instance in the creation and distribution of multimedia content, assessing objectively the quality of the service might not be enough. Instead, knowing what the end-users actually perceive, when consuming the service has become of great interest to service and network providers. This concept is gaining momentum recently and it is known as Quality of user Experience (QoE). The methodologies focused on measuring QoE are classified in two families, namely, subjective and objective approaches. The former directly assesses the perception of quality that a group of users have about the service under evaluation. According to the ITU-T Rec. P 800 (1996) the output of this approach is usually the average of the users’ opinions in a scale from 1 (poor quality) to 5 (excellent). This value, so-called Mean Opinion Score (MOS), is the most extended scale to evaluate QoE.
Although subjective tests provide the most accurate results, they are time-consuming, expensive, and do not allow continuous monitoring of the service. For that reason, objective QoE-estimation approaches have been proposed (Hsu & Lo, 2014). In order to guarantee the accuracy of these objectives techniques, these models try to obtain the greatest correlation between their estimations and the results obtained by the subjective methods (Chang et al., 2012). The objective QoE models are divided into:
- 1.
Intrusive models, which need the source voice-signal to be compared with the received stream.
- 2.
Non-intrusive models, which make use of network parameters or only need the received voice-signal to obtain their QoE estimations.
Deeper insights of objective QoE estimation models for VoIP can be found in (Jelassi et al., 2012). Figure 1 shows some factors that impact on QoE. The nature of these factors is multiple, contributing to quality degradation in different ways: