Guaranteeing User Rates With Reinforcement Learning in 5G Radio Access Networks

1. Introduction

The accelerated acquisition of powerful mobile devices is significantly contributing to the growing market of immersive multimedia applications. According to Cisco (2017), it is envisioned that by 2021 more than 80% of total mobile data will be represented by video traffic at different data rate requirements. In this context, the end-user Quality of Experience (QoE) will make the difference between network operators while providing these pretentious services (Trestian, Comsa, and Tuysuz, 2018). According to Ghinea, Timmerer, Lin, and Gulliver (2014), the concept of Multiple Sensorial Media (Mulsemedia) can enhance the user perceived QoE when experiencing poor video quality by incorporating additional senses such as: olfaction, wind, haptic, etc. However, the real factor that impacts the video quality degradation is denoted by the QoS provisioning schemes on the wireless interface that can differ from one operator to another. By providing higher video rates than the requested limit, the rate of packet drops is increased in order to keep the normal functionality of video decoders. On the other side, lower data rates for video services will increase the packet delays which have as a consequence a larger number of lost packets at the radio interface. Thus, guaranteeing certain data rates for video traffic is crucial in order to avoid the degradation of user perceived quality.

In 5^th Generation (5G) of mobile communications standard, guaranteeing certain bit rate requirements is even stricter especially with the popularity of the new bandwidth hungry applications (i.e. high definition video, virtual reality traffic) (Elbamby, Perfecto, Bennis, and Doppler, 2018). This puts a significant pressure on Radio Resource Management (RRM) to provide these immersive services with very stringent QoS in multi-user scenarios (Li et al. 2017). Alongside of other RRM functions, the packet scheduler is in charge of allocating user data packets in frequency domain at each predefined Transmission Time Intervals (TTIs). According to Comşa (2014a), the scheduling process is conducted based on the scheduling rules aiming to maximize the satisfaction of particular QoS requirements. In literature, several scheduling rules are proposed to deal with the Guaranteed Bit Rate (GBR) objective. For example the scheduling rule proposed by Lundevall et al. (2004) is designed to work for WCDMA access networks and very low data rates of video services. Andrews, Qian, and Stolyar (2005) propose a scheduler for CDMA downlink networks in which a maximum number of 40 users are scheduled with the maximum rate of 160kbps. In the same type of access networks, the scheduler proposed by Kolding (2006) outperforms other GBR oriented schedulers for the considered networking scenarios. However, the proposed schedulers work appropriate only for particular scheduler states in terms of: access technologies, user rates, channel conditions, traffic load, etc. On one hand, these scheduling techniques must be upgraded for the novel access technologies imposed by 5G standard. On the other hand, the aim would be to use each of these scheduling rules on the best matching scheduler state in order to maximize over time the satisfaction of user rate requirements for various traffic types.