Stochastic Approximation-Based Transport Profiling for Big Data Movement Over Dedicated Connections

Introduction

Extreme-scale scientific applications are generating colossal amounts of datasets, now frequently termed as “big data”, which must be transferred over long geographical distances for distributed processing and analysis. Such big data transfer requires stable and high-speed network connections, which, unfortunately, are not readily available in traditional shared IP networks, e.g., the Internet. High-performance Networks (HPNs) such as ESnet (ESnet, 2017), Internet2 (Internet2, 2017), and Google’s B4 (Jain et al., 2013) that provide on-demand dedicated high-speed network connections realized by technologies such as MPLS (Andersson & Swallow, 2003) and OpenFlow (McKeown et al., 2008) have emerged to support these data- and network-intensive applications. More recently, significant progress has been made for big data movement in various aspects including the deployment of 100Gbps networks with future 1Tbps capacity, the increase in end-host capabilities with multiple cores and buses, and the use of parallel file systems. However, such technology advancement and infrastructure investment have not led to corresponding improvement in big data transfer performance, especially at the application level. Maximizing the data transfer performance over such dedicated connections is still challenging, mainly because: i) the optimal operational zones of transport protocols are affected by the complex configurations and dynamics of the network segments, end-hosts, and protocol itself; ii) different parameter settings may lead to very different performance and oftentimes the default parameter setting does not yield the best performance; iii) due to the lack of accurate models for high-performance transport protocols such as UDT (Gu & Grossman, 2007), a widely used protocol in the HPN community (UDT, 2017a), and the complex dynamics of network environments, it is generally very difficult to model and derive the optimal parameter values analytically. For a given data transfer protocol, a careful selection of parameter values may result in a significant performance improvement over its default settings. As a motivating example, Figure 1 shows the instantaneous UDT throughput performance with different block sizes over a local back-to-back 10Gbps connection, where the performance is improved by three times on average through a simple adjustment made on the block size.

Figure 1.

Instantaneous throughput measurements of UDT over a 10Gbps back-to-back connection with different block sizes

Transport profiling, which sweeps through the combinations of parameter settings such as socket options, application-specific parameters, and protocol-specific configurations, enables users to determine the “best” set of parameter values for the optimal data transfer performance. There exist several bandwidth estimation tools such as ESnet iperf3 (Iperf3, 2017), which uses continuous data transfer to estimate the achievable throughput along an end-to-end network path. It provides users with various functions and options for tuning TCP, UDP and SCTP. The Transport Profiler Generator (TPG) (Yun et al., 2015) complements ESnet iperf3 by providing throughput estimation for UDT, and additionally supports parallel data streams over multiple NIC-to-NIC connections. A survey of bandwidth estimation tools can be found in (Prasad et al., 2003). These bandwidth estimation tools could be utilized to conduct exhaustive transport profiling to optimize big data transfer performance. However, such exhaustive-based profiling method is prohibitively time consuming when there exists a large control parameter space, which is almost always the case in most big data transfer scenarios. In general, users may not be in favor of performing transport profiling if the profiling overhead is comparable with the time needed for their actual data transfer.