Deep Reinforcement Learning Methods for Energy-Efficient Underwater Wireless Networking

Introduction

During the recent years, Underwater Wireless Sensor Networks (UWSNs) have gained a lot of attention. These networks provide a large number of applications including, for example in studies on marine mammals as well as investigating commercial exploitations strategies. The main driving force for investigating the design process for underwater wireless sensor networks involves the need for collecting oceanography data in order to provide the opportunity for human beings to study large areas underwater in various ways (Jin, Ma, Su, Li, & Fu, 2017). The UWSN constitutes a number of sensors and vehicles specifically designed for the target underwater domain in order to collect the required data and provide the opportunity for collaborative monitoring. In a general sense, in order to proceed with monitoring the ocean bed, there have been a number of oceanographic sensors developed and proposed that are capable of recording data in a stable manner, while also providing the opportunity to recover the utilized instruments once the task of collecting the required data is completed. One of the problems with using conventional methods involves the fact that there is no interactive communication between the two ends, increasing the likelihood of losing the recorded data during the transmission stage. Moreover, in such methods, the recorded data will be damaged in case of failure (Awan, et al., 2019). On the other hand, some of the major concerns and problems influencing underwater routing involve the issue of the mobility of the nodes, the limited available bandwidth, the low speed of sound, major possibilities of failure, and the limitations in energy consumption. In general, when developing routing algorithms, keeping a complete picture of the problem in mind is a highly difficult task. Therefore, there is a need for achieving a tradeoff among various limitations. Similar to any other network, in UWSNs, it is highly important and integral to set up appropriate paths and provide a secure and efficient way for transmitting the collected data (Dosaranian-Moghadam & Amo-Rahimi, 2018). The schematic architectural configuration of UWSNs is presented in Figure 1 below. There is a strong similarity between Underwater Wireless Sensor Networks (UWSNs) and Terrestrial Wireless Sensor Networks (terrestrial WSNs). However, the challenges facing the UWSNs differ from those facing the terrestrial WSNs due to difference in the technology utilized in the physical layers along with the difference in the propagation medium. In clearer terms, radio signals are used in terrestrial WSNs to transmit the data; however, such a technology is not appropriate for underwater settings since it requires a lot of energy and radio signals dissipate rapidly in water. Therefore, acoustic signals can constitute a much better choice as a communication medium when working underwater. However, the main issue with the acoustic signal is that its low speed compared to radio signals results in a high level of propagation delay. Moreover, the multipath fading channel results in slightly higher error ratios and low bandwidth. These issues cause a number of problems including low levels of reliability, high energy consumption, and the high rate of retransmission. Therefore, designing appropriate routing protocols for UWSNs is one of the main challenges in this field (Zhu & Wei, 2018).

Figure 1.

The architecture for the underwater sensor network (Zhu & Wei, 2018)

In recent years, a number of protocols for efficient energy consumption have been proposed. Among these protocols, the Q-learning-based energy-efficient and lifetime-aware routing (Hu & Fei, 2010) and the VBF protocol (Xie, Cui, & Lao, 2006) can be mentioned. The VBF protocol reduces energy consumption by limiting the number of senders. In this protocol, suitable senders are the ones which have the shortest geographical distance to the destination node, with the shortest distance being referred to as the ‘routing vector’. In this protocol, the routing pipe is shaped around the routing vector. Some of the nodes might be located outside the routing pipe, and thus they will not be considered as appropriate senders. Moreover, they are not allowed to send any received packages. In this way, the system can save on energy consumption by reducing the number of transmission at unsuitable nodes.