Outcomes Assessment in Japanese Language Instruction

Context And Challenges

On two recent occasions Japanese language instructors at the University of Guam have been asked to conduct and submit the results of student learning outcomes assessment. The first was for a mandated Japanese Studies program review, while the second was for a 2007 Western Association of Schools and Colleges (WASC) accreditation visit. These two occasions made us wonder why the administration was suddenly starting so forcefully to request that we do such assessment and thus supposedly demonstrate faculty accountability for student learning. We also wondered how the administration intended to use the assessment results – to truly enhance the classroom experience for both students and teachers, or to close classes and programs whose results were somehow unsatisfactory. As Barbara Walvoord (2004) points out, one of the challenges facing any assessment process is that teachers tend to place a “high value on faculty independence and autonomy” (p.7). Therefore, when administrators or external agencies require assessment, faculty members are almost automatically suspicious of any so-called “systematic procedure that supposedly can be applied to what they view as the subtle art of teaching” (Walvoord, 2004, p.7). In other words, faculty, perhaps especially in the Liberal Arts, are not clear as to how quantitative methods can be used to evaluate an essentially qualitative experience.

However, instead of resisting or ignoring or minimizing the administration’s request for assessment data, the Japanese Studies program chose to view it as an opportunity to evaluate student learning and strengthen courses in a way that had not yet been tried. No less importantly, the program also saw it as a chance to eventually increase the number of students majoring in Japanese Studies or minoring in Japanese. Numerous scholars (Bok, 2005; Friedman, 2005; Lederman, 2006) have argued that a major reason why the demand for more accountability in higher education is growing is a concern about the United States’ future economic competitiveness in a global society. In addition, in the aftermath of the terrorist attacks of 9/11 the National Language Conference published a White Paper that stressed America’s “critical need to take action to improve [its] foreign language and cultural capabilities” (2005, p.4). From the particular standpoint of FL instruction (in this case, Japanese), both the economic and national security arguments for more accountability offer great potential for recruiting and retaining quality students. But that potential also challenges FL programs to produce learning environments where those students can thrive. As both John Norris (2006) and Barbara Wright (2006) make clear, one way to provide those learning environments is through assessment initiatives designed and implemented by the teachers themselves.

Of course, the decision to proceed immediately presented the Japanese Studies program with another, more methodological challenge – namely, how to successfully implement an assessment project. We were forced to ask ourselves what might be the most effective strategies for assessing learning outcomes in Japanese language classes, given a limited amount of time and resources. First of all, we chose to keep this initial project fairly simple, following Walvoord’s reminder that “assessment need not be complicated” (2004, p.6). Therefore, we selected more standardized (product-driven) assessment methods, what might be termed “pen and pencil” evaluation, such as multiple-choice, true/false or short answer tests. In so doing, we were well aware of various criticisms of such an approach, as well as alternative (qualitative) approaches that seek to assess, for example, problem solving skills through real-world, task-based language performance (Herman, Aschbacher and Winters, 1992; Bachman and Palmer, 1996; Norris, Brown, Hudson and Yoshioka, 1998; Bachman, 2002). Nevertheless, especially at an elementary level, the primary purpose of FL instruction often involves rote learning, or rather, the necessary memorization of appropriate lexical, morphological and syntactic choices. Learning Japanese, for instance, requires an eventually extensive acquisition of kanji vocabulary, so relying on traditional methods (assessing student command of the target language, as opposed to critical thinking skills) seemed justified.