Construction of Lightweight Big Data Experimental Platform Based on Dockers Container

Construction of Lightweight Big Data Experimental Platform Based on Dockers Container

Youli Ren
Copyright: © 2020 |Pages: 14
DOI: 10.4018/IJISMD.2020070106
(Individual Articles)
No Current Special Offers


To reduce the large data experiment platform construction cost and reduce the learning difficulty big data, this article is based on virtualization technology through the Docker software installed on the Linux system, using the Open VpN routing forwarding, using Java web technology, realizing the big data within the local area network (LAN) cluster environment fleetly, and constructed of lightweight data experiment platform. Through this platform, we can create a big data cluster with one key, provide a variety of experimental environments matching the courses, focus on the technology itself, and greatly improve learning efficiency. Experimental analysis shows that the proposed construction method has a host occupancy rate of around 10% and a memory occupancy rate of around 10%, and the system runs stably.
Article Preview

1. Introduction

Presently, laboratories working with virtualization technique utilizing OpenStack platform for the virtulization of server resources (Gebreyohannes, 2014). These resources are then allocated to the students for making of clusters. Virtulization technique consist of OpenStack platform does not meet the requirement of resource utilization. The configuration of file is not satisfactory flexible for the operation of resource utilization. Docker provides a efficient solution as its an open source application container engine (Nasim et al., 2014). Docker provides the pacakage of applications in portable container and afterwards distributing tasks to machines. The virtulization can easily be achieved in docker, without single interface among containers. Therefore, the complete operating system can easily be developed automatically and much faster. The construction of specialized data science and big data analysis laboratories is the key to training applied talents in short supply (Pahl, 2015). Virtualization is the provision of centralized physical computing resources on the server side that can be decomposed into smaller units to serve different users independently, while ensuring isolation and security. Because the use of common PC hardware configuration of the machine to reduce the cost, through the means of virtualization to meet the user's demand for resources, very suitable for education and teaching experimental environment. Docker-based virtual experiment platform construction is a solution of simple experiment platform construction and deployment (Boettiger, 2015). In terms of the system architecture, the deepened service-oriented architecture design is adopted to cooperate with the service discovery mechanism, which makes the system more flexible, more convenient to maintain, and the coordination of the experimental platform more smooth (Gutsche et al., 2017). In the application of the experimental environment, container-based virtualization technology makes the system more compatible and portable, with the characteristics of high cohesion and low coupling. Therefore, it is widely used in the construction of various enterprise applications.

In order to reduce the cost of laboratory construction and the difficulty of big data learning, lightweight Docker container technology is used to provide users with a virtualized operating environment, which can ensure an efficient environment for users to conduct the same experiment is the same, and ensure that each user's virtual learning environment is independent and does not interfere with each other (Manu et al., 2016). Because Docker uses virtualization technology at the operating system level, it has higher utilization of system resources such as memory and CPU, and can support more users to use online at the same time under the same server cluster (Liu et al., 2020).

Complete Article List

Search this Journal:
Volume 14: 1 Issue (2023)
Volume 13: 8 Issues (2022): 7 Released, 1 Forthcoming
Volume 12: 4 Issues (2021)
Volume 11: 4 Issues (2020)
Volume 10: 4 Issues (2019)
Volume 9: 4 Issues (2018)
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing