Article Preview
TopIntroduction
Mobile edge computing (MEC) (Shahidinejad et al, 2021) is an ultra-low delay technology that can perform data analysis near the data source. Specifically, MEC pushes the frontier of services and data away from centralized cloud to the edge of mobile network with computing and storage resources. Then, these small edge clouds (ECs) are used to support resource-intensive and delay-sensitive applications. ECs install the MEC server in a fixed cellular base station (BS). When providing MEC services for mobile users (MUs) in the Internet of Things, it is necessary to have a dedicated wireless connection between ECs and MUs (Pan et al, 2020). However, due to signal blockage and shadow occlusion, it is difficult to provide MEC reliable services for MUs.
In order to solve these problems, unmanned aerial vehicles (UAVs) (Pfeifer et al, 2021) are deployed as the miniaturized ECs with embedded computing modules (Dai et al, 2021). Nowadays, UAVs have been widely concerned as an effective technology in the wireless network (Zeng et al, 2016). In UAV-assisted MEC system, MUs need to process the generated data quickly, but their communication, computing and storage resources are very limited. The UAV uses its own computing module to offer their computing service to MUs. Considering the limited battery life and computing power of UAVs, it is necessary to design an efficient task offloading and power allocation (TOPA) scheme in UAV-assisted MEC system.
However, since the joint TOPA problem in UAV-assisted MEC system has a strong non-convex characteristic, it is difficult to obtain the joint optimal strategy. Some researchers have proposed iterative algorithm (He et al, 2021; Ji et al, 2020), Lyapunov optimization algorithm (Wang et al, 2020) to deal with this issue. Some scholars have combined a variety of algorithms in order to approach the optimal solution. A two-level optimization method was proposed by combining differential evolution algorithm with elimination operator and efficient greedy algorithm in (Wang et al, 2019). The authors in (Li et al, 2020) and (Lim et al, 2019) combined Dinkelbach algorithm with successive convex approximation technology and closed-form method, respectively. However, it is very challenging to find the optimal strategy without the complete network information of the system. Reinforcement learning (RL) method can obtain the optimal strategy of intelligent decision-making through interaction with the environment. The Q-learning algorithm was proposed to solve the approximate optimal solution of the non-convex problem in (Elgendy et al, 2021). However, these RL methods are difficult to solve the problems of large state-action space or continuous state-action space (Zhao et al, 2020a). By combining deep neural network with RL, deep reinforcement learning (DRL) has received extensive attention in the field of wireless communication (Volodymyr et al, 2019). In our previous work, a DRL method has been proposed to solve the task offloading problem in UAV-assisted MEC system (Yu et al, 2021). However, DRL-based TOPA scheme has seldom been studied in UAV-assisted MEC system.
In this article, we propose a DRL method for the TOPA problem in UAV-assisted MEC system. The main contributions of this article are summarized as follows: