Our Knowledge Management Hubble May Need Glasses: Designing for Unknown Real-Time Big Data System Faults

Our Knowledge Management Hubble May Need Glasses: Designing for Unknown Real-Time Big Data System Faults

William H. Money (The Citadel, Charleston, USA) and Stephen J. Cohen (Microsoft Corporation, Oakton, USA)
Copyright: © 2018 |Pages: 21
DOI: 10.4018/IJKM.2018010103

Abstract

This article analyzes the properties of unknown faults in knowledge management and Big Data systems processing Big Data in real-time. These faults introduce risks and threaten the knowledge pyramid and decisions based on knowledge gleaned from volumes of complex data. The authors hypothesize that not yet encountered faults may require fault handling, an analytic model, and an architectural framework to assess and manage the faults and mitigate the risks of correlating or integrating otherwise uncorrelated Big Data, and to ensure the source pedigree, quality, set integrity, freshness, and validity of the data. New architectures, methods, and tools for handling and analyzing Big Data systems functioning in real-time will contribute to organizational knowledge and performance. System designs must mitigate faults resulting from real-time streaming processes while ensuring that variables such as synchronization, redundancy, and latency are addressed. This article concludes that with improved designs, real-time Big Data systems may continuously deliver the value of streaming Big Data.
Article Preview

1. Knowledge Management Challenges From The Third Generation Processing Of Real-Time Big Data

This paper agrees with the basic proposed reconceptualization of the data underpinning the knowledge pyramid as a better representation that documents (and records) added aspects of reality because of the inclusion of Big Data. The arguments presented in this paper extend our understanding of the intricacy and evolution of the knowledge pyramid with an in-depth analysis of the risks of Big Data when the Big Data are incorporated into the data, information, knowledge, and wisdom hierarchy without standards that recognize the characteristics and control or mitigate potential risks associated with such data.

The new proposition and basic premise offered by Jennex and Bartczak (2013) is that Big Data represents an unpresented data expansion via massive increases in the number of different views, availability and depth of the reality images, and an increased resolution of points depicting reality. The data expansion creates additional data layers through greatly expanded transaction/operational data, and requires the addition of a lower or bottom layer of Big Data. This layer is acquired from sources such as sensors and social media to better depict the knowledge pyramid. Our agreement with this new pyramid and its conceptualization can best be described with an analogy that compares the Big Data knowledge age we are entering to the launch of the Hubble telescope over 25 years ago. The Hubble was a huge leap for astronomy because it removed Earth’s atmospheric changes and light blocking distortions from our view of the stars, and employed digital cameras to construct our views of the universe (from the collected data) through visible, infrared and ultraviolet light. However, Hubble’s data had initial imaging defects and the telescope received ongoing servicing with improvements for many years to make the data usable via the additions of a number of improvements such as corrective lens, cameras, gyroscopes, stabilizing arrays, and corresponding software modification to correctly account for the objective sensory improvements. This paper correspondingly analyzes possible faults or misconceptions with the Big Data that are now being incorporated into the revised knowledge-KM pyramid. This new conceptualization theorizes that there is more knowledge than data, and illustrates that knowledge management is an organizational learning extraction of the knowledge pyramid. Our contribution is to begin to describe how the revised knowledge pyramid has been enhanced and what is still unknown. We address when Big Data, composed of data sets from sources including the Internet of Things, sensors, and social data sources, introduces risks and as yet unknown dependencies and/or possible errors because these Big Data are not yet well understood.

Knowledge management, an approach that adds or enhances value by leveraging data, analysis, experience, and knowhow inside and outside of an organization (Ruggles, 1998), has been greatly impacted and enhanced in the recent past by the processing of Big Data including unstructured text, sensors, and data from systems utilizing known data elements. The introduction of these Big Data is significant and has caused Jennex and Bartczak (2013) to propose a revision to the knowledge pyramid. They have argued that learning, filtering, and transformation processes create a significant difference between the KM knowledge pyramid and the general knowledge pyramid. Their 2013 model views the processes of KM delivering actionable intelligence and identified filters, processes, and technologies as integral delivery processes. However, Jennex and Bartczak (2013) did not originally consider Big Data, analytics, and the Internet of Things in their model. In a recently published paper (Jennex, 2017); Big Data are incorporated into a further evolved model KM pyramid. This paper holds a similar perspective arguing that the impacts on KM have and will be further exacerbated by a recent enormous inflow of social media data that will be continually added to the data from other collection sources. For example, they posit that impacts are enormous and will generate business value from the knowledge gleaned from these combined sources by applying conventional (pre-Big Data) sentiment analysis techniques. However, a significant problem arises: the computational tools used to analyze the Big Data are not ideally suited to leverage Big Data (Gandomi, & Haider, 2015).

Complete Article List

Search this Journal:
Reset
Open Access Articles
Volume 15: 4 Issues (2019): 2 Released, 2 Forthcoming
Volume 14: 4 Issues (2018)
Volume 13: 4 Issues (2017)
Volume 12: 4 Issues (2016)
Volume 11: 4 Issues (2015)
Volume 10: 4 Issues (2014)
Volume 9: 4 Issues (2013)
Volume 8: 4 Issues (2012)
Volume 7: 4 Issues (2011)
Volume 6: 4 Issues (2010)
Volume 5: 4 Issues (2009)
Volume 4: 4 Issues (2008)
Volume 3: 4 Issues (2007)
Volume 2: 4 Issues (2006)
Volume 1: 4 Issues (2005)
View Complete Journal Contents Listing