Welcome to the InfoSci Platform
Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data

Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data

Padmashree Ravindra, Kemafor Anyanwu
Copyright: © 2018 |Pages: 28
ISBN13: 9781522551911|ISBN10: 1522551913|EISBN13: 9781522551928
DOI: 10.4018/978-1-5225-5191-1.ch035
Cite Chapter Cite Chapter

MLA

Ravindra, Padmashree, and Kemafor Anyanwu. "Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data." Information Retrieval and Management: Concepts, Methodologies, Tools, and Applications, edited by Information Resources Management Association, IGI Global, 2018, pp. 811-838. https://doi.org/10.4018/978-1-5225-5191-1.ch035

APA

Ravindra, P. & Anyanwu, K. (2018). Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data. In I. Management Association (Ed.), Information Retrieval and Management: Concepts, Methodologies, Tools, and Applications (pp. 811-838). IGI Global. https://doi.org/10.4018/978-1-5225-5191-1.ch035

Chicago

Ravindra, Padmashree, and Kemafor Anyanwu. "Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data." In Information Retrieval and Management: Concepts, Methodologies, Tools, and Applications, edited by Information Resources Management Association, 811-838. Hershey, PA: IGI Global, 2018. https://doi.org/10.4018/978-1-5225-5191-1.ch035

Export Reference

Mendeley
Favorite

Abstract

Graph and semi-structured data are usually modeled in relational processing frameworks as “thin” relations (node, edge, node) and processing such data involves a lot of join operations. Intermediate results of joins with multi-valued attributes or relationships, contain redundant subtuples due to repetition of single-valued attributes. The amount of redundant content is high for real-world multi-valued relationships in social network (millions of Twitter followers of popular celebrities) or biological (multiple references to related proteins) datasets. In MapReduce-based platforms such as Apache Hive and Pig, redundancy in intermediate results contributes avoidable costs to the overall I/O, sorting, and network transfer overhead of join-intensive workloads due to longer workflows. Consequently, providing techniques for dealing with such redundancy will enable more nimble execution of such workflows. This paper argues for the use of a nested data model for representing intermediate data concisely using nesting-aware dataflow operators that allow for lazy and partial unnesting strategies. This approach reduces the overall I/O and network footprint of a workflow by concisely representing intermediate results during most of a workflow's execution, until complete unnesting is absolutely necessary. The proposed strategies are integrated into Apache Pig and experimental evaluation over real-world and synthetic benchmark datasets confirms their superiority over relational-style MapReduce systems such as Apache Pig and Hive.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.