Competency matters. Social media, customer transactions, mobile sensors, and feedback contents are all piled up with data. This might be unstructured and complex data in voluminous quantity, often called Big Data. However, if this Big Data is managed, it might bring competency for organizations. This chapter introduces the must-know concepts and materials for organizational managers who face Big Data. Through the chapter, Big Data is defined and its emergence over the time is reviewed. The four Vs model in Big Data literature and its link to a banking system is analyzed. The chapter concludes by making a managerial awareness concerning ethical issues in Big Data. This is of high priority in public sectors as data relies for every individual in the society.
TopIntroduction
Without Big Data analytics, companies are blind and deaf, wandering out onto the Web like deer on a freeway (Geoffrey Moore1).
Organizations are facing every 18 months with doubled data (Moore, 1965) in unlike formats and in different departments such as marketing, research and development, procurement, warehousing, production, sales, customer satisfaction and retention, etc. These inconsistent, unstructured and massive data- first called “Big Data” by Cox and Ellsworth (1997, p. 235)- must be managed to avoid digital disruption. Managers are responsible for providing necessary infrastructures, hardware, software or applications, and most of all, data analysts who are familiar with massive data concepts.
Although Big Data is introduced in 1997, it is still a popular and developing subject in the field of analytics (Sathi, 2012, p. 73). Complex and massive structured/unstructured data will not be processed due to memory limitations and must be processed in a place other than memory. The best suitable place is where they reside (Prajapati, 2013, p. 4). For instance, around 20 million storage cabinets are needed to file text documents or more than 13 years of video files in high-definition (HD) quality when talking about data in petabytes (Hurwitz, Nugent, Halper, & Kaufman, 2013, p. 15). This size of storage is rarely processed in memory with current available technologies.
This chapter is intended to present a holistic managerial view over Big Data with emphasis on integration issues in public sector. A definition and an overview of the emergence of Big Data are presented. The 4Vs model introducing variety, volume, velocity and veracity of data is deeply reviewed with examples. Following, Big Data applications and management challenges are discussed and finally, the importance of ethical issues in Big Data management is highlighted.
TopWhat Is Big Data?
Big Data refers to the storage, management and manipulation of different types of data in vast quantities (Hurwitz, Nugent, Halper, & Kaufman, 2013, p. 1). This includes different types such as text files, images, voices, and videos. By investigating these vast amounts of data, hidden patterns that would lead important changes might be revealed. Hurwitz et al. (2013, p. 16) define Big Data as a process in which huge and different types of data are timely managed to meet real time responsiveness in analytics. Figure 1 illustrates the emergence of Big Data over time. Famous relational databases were introduced in 1980s and 1990s which were or presumably are handling complex data. The exponential growth of data in 2000s led to massive and various data types. This lets very complex and unstructured data to be managed.
Figure 1. How Big Data emerged over time (Mohanty, Jagadeesh, & Srivatsa, 2013)
This unstructured data will be obtained either from the inside of a corporation or outside. Sathi (2012) defines these two sources as:
... a fair amount of data within the corporation that … is increasingly shared. This includes emails, mainframe logs, blogs, Adobe PDF documents, business process events, and any other structured, unstructured, or semi-structured data available inside the organization.
… a lot more data outside the organization—some available publicly free of cost, some based on paid subscription, and the rest available selectively for specific business partners or customers. This includes information available on social media sites, product literature freely distributed by competitors, corporate customers’ organization hierarchies, helpful hints available from third parties, and customer complaints posted on regulatory sites.