Article Preview
TopIntroduction
In recent years, increasing scientific fields employ workflows to analyze large amounts of data and to perform complex scientific computing applications. A process in such scientific applications is described as a workflow by partitioning it into atomic tasks. These tasks are further distributed to multiple computing resources (Rodriguez & Buyya, 2017). They are usually represented as directed graphs with their nodes representing discrete computational components and the edges representing connections along which data and results are passed. They have varying types and usually their execution needs computing platforms with different QoS requirements (Yunni Xia et al., 2015; Xia, Zhou, Luo, Pang, & Zhu, 2015; Xia, Zhou, Xin Luo, Pang & Zhu, 2015; Xia, Liu, Liu, & Zhu, 2012; Xia, Xin Luo, Jia Li, & Zhu, 2013)).
Recently, cloud computing is recognized as a suitable solution and paradigm for providing a flexible, on-demand computing infrastructure over the Internet for large-scale scientific-workflow-based applications. The services that can be provided from the cloud include Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS). SaaS and PaaS clouds offer web applications/software over the Internet, running on cloud infrastructure. They are therefore less suitable for scientific workflows than IaaS ones because they mainly offer an environment to design, develop and test web-based applications. Instead, IaaS clouds offer an easily accessible, flexible, and scalable infrastructure suitable for the deployment of large-scale scientific applications based on on-demand and pay-per-use patterns (Buyya, 2009).
One of the most challenging NP-hard problems that researchers try to address is how to schedule large-scale scientific applications to distributed and heterogeneous computational nodes, e.g., IaaS clouds, such that quantitative goals such as process make-span are optimized, and certain constraints such as communication cost and storage requirements areas followed ((Wu, Zhou, Zhu, & Xia, 2018; Zheng, Trivedi, Qiu, & Xia, 2017; Deng, Wu, Taheri, Zomaya, & Wu, 2016)). From the end-user perspective, a low make-span is always preferred, whereas from the system’s perspective system-level efficiency and fairness are often considered as a good motivation such that the scientific applications and tasks are supposed to be fairly distributed among computational resources to avoid hot spots and performance bottle-necks. However, a careful investigation into related work indicates that only a few schemes are able to deal with both perspectives, such as optimizing user objectives (e.g., make-span) while fulfilling other constraints, and providing a fair workload distribution among physical computational resources of clouds.
The primary aim of the paper is thus to provide a multi-objective scheduling framework to address the real-time workflow-scheduling problem on multiple IaaS cloud. Specifically, we consider a multi-objective optimization workflow scheduling approach based on dynamic game-theoretic model. It aims at reducing workflow make-spans, reducing cost, and maximizing system fairness in terms of workload distribution among heterogeneous VMs. We conduct extensive case studies as well based on various randomly-generated scientific workflow templates and heterogeneous VMs created on real-world third-party commercial IaaS clouds, i.e., Amazon, Tencent, and Ali clouds. Experimental results clearly suggest that our proposed approach outperforms traditional ones by achieving lower workflow make-spans, lower cost, and better system fairness.