Questionnaires to Inform a Usability Test Conducted on a CALL Dictionary Prototype

Questionnaires to Inform a Usability Test Conducted on a CALL Dictionary Prototype

Marie-Josée Hamel
DOI: 10.4018/ijcallt.2013070104
(Individual Articles)
No Current Special Offers


Questionnaires are often considered a suitable instrument to gather data on language learners’ experiences. To test the usability of an online dictionary prototype, a series of questionnaires were distributed to a class of language learners before and after they completed a language task at the computer using the dictionary prototype. Measures of task effectiveness and efficiency were obtained and correlated with the questionnaires’ results. This study shows how the questionnaire results informed the overall measure of usability and, in particular, addressed user satisfaction, a more subjective yet a central component of this measure. Pre-questionnaires prepared learners for the task, whereas post-questionnaires fostered a reflection about the task process and its outcome. Hence, it is argued that combining observation and questionnaire techniques in that context was effective in providing a fuller insight into the learner-task-tool interaction at the computer. In this CALL research and development context, questionnaires served a double function as an evaluative and a pedagogical instrument.
Article Preview

Questionnaires In Call Research And Evaluation

Questionnaires are common instruments in second language research and are typically used for data collection on learner background, preferences, attitudes, beliefs, etc. (Dornyei, 2010; Nunan, 1992). Their versatility is alleged to increase consistency and reliability of anticipated results when triangulating data with regards to undermining biases for drawing conclusions on factual, behavioural, and attitudinal findings (Brown, 2001). Moreover, open-ended items in questionnaires, for instance, can play a significant role in providing real perspective, perhaps countering the aforementioned superficiality of user answers (Dornyei, 2010). Questionnaires are used for similar purposes in CALL research and evaluation (Hémard & Cushion, 2001; Hubbard, 2006; Jamieson, Chapelle & Preiss, 2005; Levy, 1999;).

For instance, in Huh and Hu’s (2005) chapter on criteria for effective CALL research, the authors suggest focusing on what happens through, with, or around the CALL tool through valid research design. Quoting Surry and Ensminger (2001), the authors warn against techno-centrism in CALL research, adding that researchers must take into account the instructional method as well as learner characteristics, by “using a variety of experimental, quasi-experimental and qualitative research designs” (Surry & Ensminger, 2001, p. 35) in order to correctly address the question of CALL evaluation. They also point out that statistical results do not suffice in providing the in-depth explanation and solid evidence needed to investigate the impact of computer technology use in language learning. Adding that certain methods result in more accurate answers to research questions, Huh and Hu (2005) call for a balanced quantitative and qualitative method in CALL research.

With regards to CALL evaluation, Raby (2005) states that such a task requires contrasting the evaluated activity with a user-centered model of reference. The model involves reflexive thinking on behalf of the learner through questionnaires, which allow structuring different qualitative and quantitative data into a coherent framework. More specifically, she recommends the use of questionnaires for information on how a user redefines his task and how a user feels about the achieved task, which the author believes to affect the mental processes in progress Questionnaires, hence, should ask for information on the user’s perception of goal achievement, difficulties met while at task, competencies demonstrated, motivation and reasons for motivation, and complimentary conditions needed to accomplish the task at hand successfully.

Complete Article List

Search this Journal:
Volume 14: 1 Issue (2024): Forthcoming, Available for Pre-Order
Volume 13: 1 Issue (2023)
Volume 12: 5 Issues (2022)
Volume 11: 4 Issues (2021)
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing