Guaranteeing User Rates With Reinforcement Learning in 5G Radio Access Networks

Guaranteeing User Rates With Reinforcement Learning in 5G Radio Access Networks

Ioan-Sorin Comşa (Brunel University London, UK), Sijing Zhang (University of Bedfordshire, UK), Mehmet Emin Aydin (University of West of England, UK), Pierre Kuonen (University of Applied Sciences of Western Switzerland, Switzerland), Ramona Trestian (Middlesex University London, UK) and Gheorghiţă Ghinea (Brunel University London, UK)
DOI: 10.4018/978-1-5225-7458-3.ch008


The user experience constitutes an important quality metric when delivering high-definition video services in wireless networks. Failing to provide these services within requested data rates, the user perceived quality is strongly degraded. On the radio interface, the packet scheduler is the key entity designed to satisfy the users' data rates requirements. In this chapter, a novel scheduler is proposed to guarantee the bit rate requirements for different types of services. However, the existing scheduling schemes satisfy the user rate requirements only at some extent because of their inflexibility to adapt for a variety of traffic and network conditions. In this sense, the authors propose an innovative framework able to select each time the most appropriate scheduling scheme. This framework makes use of reinforcement learning and neural network approximations to learn over time the scheduler type to be applied on each momentary state. The simulation results show the effectiveness of the proposed techniques for a variety of data rates' requirements and network conditions.
Chapter Preview

1. Introduction

The accelerated acquisition of powerful mobile devices is significantly contributing to the growing market of immersive multimedia applications. According to Cisco (2017), it is envisioned that by 2021 more than 80% of total mobile data will be represented by video traffic at different data rate requirements. In this context, the end-user Quality of Experience (QoE) will make the difference between network operators while providing these pretentious services (Trestian, Comsa, and Tuysuz, 2018). According to Ghinea, Timmerer, Lin, and Gulliver (2014), the concept of Multiple Sensorial Media (Mulsemedia) can enhance the user perceived QoE when experiencing poor video quality by incorporating additional senses such as: olfaction, wind, haptic, etc. However, the real factor that impacts the video quality degradation is denoted by the QoS provisioning schemes on the wireless interface that can differ from one operator to another. By providing higher video rates than the requested limit, the rate of packet drops is increased in order to keep the normal functionality of video decoders. On the other side, lower data rates for video services will increase the packet delays which have as a consequence a larger number of lost packets at the radio interface. Thus, guaranteeing certain data rates for video traffic is crucial in order to avoid the degradation of user perceived quality.

In 5th Generation (5G) of mobile communications standard, guaranteeing certain bit rate requirements is even stricter especially with the popularity of the new bandwidth hungry applications (i.e. high definition video, virtual reality traffic) (Elbamby, Perfecto, Bennis, and Doppler, 2018). This puts a significant pressure on Radio Resource Management (RRM) to provide these immersive services with very stringent QoS in multi-user scenarios (Li et al. 2017). Alongside of other RRM functions, the packet scheduler is in charge of allocating user data packets in frequency domain at each predefined Transmission Time Intervals (TTIs). According to Comşa (2014a), the scheduling process is conducted based on the scheduling rules aiming to maximize the satisfaction of particular QoS requirements. In literature, several scheduling rules are proposed to deal with the Guaranteed Bit Rate (GBR) objective. For example the scheduling rule proposed by Lundevall et al. (2004) is designed to work for WCDMA access networks and very low data rates of video services. Andrews, Qian, and Stolyar (2005) propose a scheduler for CDMA downlink networks in which a maximum number of 40 users are scheduled with the maximum rate of 160kbps. In the same type of access networks, the scheduler proposed by Kolding (2006) outperforms other GBR oriented schedulers for the considered networking scenarios. However, the proposed schedulers work appropriate only for particular scheduler states in terms of: access technologies, user rates, channel conditions, traffic load, etc. On one hand, these scheduling techniques must be upgraded for the novel access technologies imposed by 5G standard. On the other hand, the aim would be to use each of these scheduling rules on the best matching scheduler state in order to maximize over time the satisfaction of user rate requirements for various traffic types.

Key Terms in this Chapter

Reinforcement Learning: Training/learning method aiming to automatically determine the ideal behavior within a specific context based on rewarding desired behaviors and/or punishing undesired one.

Unsupervised Learning: Type of machine learning algorithm that is capable of learning a function that best represents a model from datasets consisting of unlabeled input data.

Multi-Objective Optimization: Mathematical optimization problems involving more than one objective function to be optimized simultaneously.

Machine Learning: Category of algorithm that uses statistical techniques to analyze data and make predictions.

Radio Resource Management: Function of a mobile communication systems used for establishing, maintaining, and releasing of radio resources.

Complete Chapter List

Search this Book: