Abstract
This chapter focuses on quantitative content analysis of online interactions, in particular, asynchronous online discussion. It clarifies the definitions of quantitative content analysis and provides a summary of 23 existing coding schemes, broadly categorized by the theoretical constructs under investigation: (1) (Meta) cognition, (2) knowledge construction, and (3) presence. To help interested researchers harvest the rich source of data in online communities, guidelines for using quantitative content analysis of online interactions were provided. In addition, important methodological considerations and issues were discussed, including the issues of validity, reliability, choice of unit of analysis, and latent versus manifested content.
TopBackground
In this section, we first review content analysis literature to clarify some fundamental concepts before describing some recent applications in asynchronous online discussion.
Just like in many fields of research, the key players in content analysis have different definitions of content analysis. The main contention lies in the debate about whether it is a quantitative or a qualitative method (or both). Researchers using or advocating quantitative content analysis method described the method as “objective”, “systematic”, and “scientific” (Berelson, 1952; Neuendorf, 2002; Riffe, Lacy & Fico, 1998). Neuendorf (2002, p.10), for example, offers the following definition: “Content analysis is a summarizing, quantitative analysis of messages that relies on the scientific method (including attention to objectivity-intersubjectivity, a priori design, reliability validity, generalizability, replicability, and hypothesis testing).” To Neuendorf (2002), qualitative methods like discourse analysis or conversational analysis should not be called content analysis because “content analysis has as its goal a numerically based summary of a chosen message set. It is neither a gestalt impression nor a fully detailed description of a message or message act” (p. 14).
Key Terms in this Chapter
Inter-Coder Reliability: The degree of agreement or co-variation (whether the scores go up or go down correspondingly) of scores given by two or more coders.
Validity: an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores and other modes of assessment. (Messick, 1989, p.13)
Latent Content: Characteristics of content (text) that requires the coder to put together several manifest features to form a pattern or to make inferences beyond the manifested characteristics.
Reliability: A measure of how close the observed scores are to the true scores, usually indicated by the consistencies among several measurements of the same construct.
Unit of Analysis: smallest parts that the content of interactions (text) can be divided for analysis based on syntactic features (linguistic features like paragraph) or semantic features (meaning of the text).
Asynchronous Online Discussion: A computer-mediated communication that can support interactions among participants separated by time and space.
Quantitative Content Analysis: Content analysis that includes a measurement process of assigning numbers to properties of text based on a set of rules (an analysis or a coding scheme).
Content Analysis: A research technique for making valid and reliable inferences from the content of communication or interaction, usually in the form of texts. It requires interpretation of the meaning of the texts, taking into consideration the context of the interactions.
Manifest Content: Surface characteristics of content (text) that can be objectively counted, for example, the number of messages posted, the number of messages read or the number of words in each message.