Micro-Analysis of Collaborative Processes that Facilitate Productive Online Discussions: Statistical Discourse Analyses in Three Studies

The Value Of Temporal Analysis For Online Discussions

Studying time with advanced statistics has the seductive allure of showing cause-effect relationships in large online data sets. Unlike data collected at one point in time (e.g., student tests on July 1, 2014), longitudinal data (e.g., tests on multiple dates) enable analyses that can provide stronger evidence of cause-effect relationships. Whereas one-time data can at best yield correlations, longitudinal data can eliminate potential reverse causation. For example, in an online discussion, if disagreements precede correct, new ideas (or micro-creativity; Chiu, 2008a) but not vice-versa, we can be confident that micro-creativity does not cause disagreements. Thus, eliminating the possibility of reverse causation provides stronger support for cause-effect hypotheses (e.g., disagreements cause micro-creativity).

Furthermore, the explosion of enormous online, longitudinal data provide analytic opportunities for advanced statistics that greatly extend the scope of process analyses traditionally conducted via qualitative analyses. Online forums, twitter tweets and other internet interactions can quickly generate millions of data that cannot be analyzed qualitatively in a reasonable amount of time. Exploratory qualitative studies can examine a small subset of the data (e.g., several threads of online messages) to generate hypotheses that can be tested with statistical methods across many people and across many contexts. Conversely, the outcomes of large scale statistical analyses can point to specific portions of online discourse suitable for detailed, qualitative analysis.

In particular, analyzing time with statistical methods can address three important sets of research questions regarding sequences, time periods and phenomena at different time-scales. First, there are questions regarding sequences (the relative order of a series of events). For example, the likelihood of a target process (e.g., micro-creativity) might be explained more fully by a sequence of groupmates’ recent messages (micro-context), rather than a single prior message. For example, a disagreement often precedes micro-creativity, but it might be the sequence of a wrong idea followed by a disagreement that truly increases the likelihood of micro-creativity ([wrong idea → disagreement] → micro-creativity; Chiu, 2008a). Likewise, other sequences might reduce the likelihood of micro-creativity (e.g., [rude disagreement → rude disagreement] –X→ micro-creativity; Chiu, 2008a).

Second, we can ask questions about time periods and how target processes and relationships might differ across them (meso-time context). Within an online discussion of a problem for example, there might be less micro-creativity at the start when students are trying to understand the problem and more micro-creativity at the end when they are close to a solution. These can be considered distinct time periods. Furthermore, sequences that occur often at one time period might be rare at another time period; for example, disagreements might often precede micro-creativity in the middle of the discussion but not at the start or at the end. This raises the question of how to identify time periods. One way is to use pre-determined time periods such as instructional units (classes) and time units (days and weeks). Another way is to determine when the target process is mostly homogeneous, such as a low micro-creativity time period, and the pivotal moments that separate them from high micro-creativity time periods. The latter way of setting time points helps us model relationships across phenomena at different time scales.

Third, we can ask questions about how phenomena at different time-scales influence one another (such as time period ↔ message). For example, students might not understand the problem well during the beginning (initial time period), which might reduce the likelihood of micro-creativity in a message. On the other hand, a micro-time-scale phenomenon (e.g., an insult in a message) might radically alter the medium time-scale social interaction of a time period (argument). This message with an insult is a pivotal message that separates the previous time period from the subsequent time period, each with substantially different characteristics.