Abstract
Among the foremost challenges with big data is how to go about analyzing it. What new tools are needed to be able to properly investigate and model the large quantities of highly complex, often messy data? Chapter 4 addresses this question by introducing and briefly exploring the fields of Machine Learning, Natural Language Processing, and Social Network Analysis, focusing on how these methods and toolsets can be utilized to make sense of big data. The authors provide a broad overview of tools, ideas, and caveats for each of these fields. This chapter ends with a look at how one major public university in the United States, the University of Texas at Arlington, is beginning to address some of the questions surrounding big data in an institutional setting. A list of additional readings is provided.
TopRecall that in chapter three, we proposed a rough heuristic for what “big data” might mean in an educational research context: data sets which are too large to process efficiently in an Excel spreadsheet and thus ultimately require higher level programming skills and newer, statistically-defensible methodologies in order to derive insight from the dataset.
So, what are some of the tools we can turn to when the old ones can’t keep up? What can we use to deal with large, complex datasets, and extract meaning from them that was previously unavailable? We need to look beyond the familiar worlds of Excel and simple statistical tests. As it happens, there are a great many tools that fit our criteria, far too many to cover comprehensively. We will focus in on what we think are the most useful offerings from just a few fields:
We will only briefly survey the contributions of these fields (otherwise this chapter alone would be several hundred pages!). The purpose of this chapter is to describe the value and the content of each field in practical terms and in broad strokes, arming the unfamiliar reader with enough detail to venture further on their own. For those interested in more information on these subjects, the Additional Readings section of this chapter will contain a list of resources that provide a more thorough and technical treatment of these topics than we provide here.