What is Inter-Rater Reliability

Previous fluency studies typically used small datasets to analyze L2 learners' oral performance with objective fluency measures, and found positive correlation between objective measures and subjective ratings. Findings from a small dataset are difficult to generalize. However, it takes a great deal of time and effort to build a large dataset with various measures. To help facilitate this process, CAF Calculator, which outputs 50 fluency measures, has been developed. In this chapter, CAF Calculator and a workflow to compute fluency measures are introduced along with a study investigating utterance fluency of L2 learners of Japanese at two proficiency levels performing two tasks. The study found significant differences in speed, breakdown, and composite fluency measures between the two groups in both tasks. It also found that task type affects pause locations. It is hoped that the research tools introduced in this chapter will encourage more research on fluency.