Using the Item Response Theory (IRT) for Educational Evaluation Through Games

Using the Item Response Theory (IRT) for Educational Evaluation Through Games

Marcelo Henrique Euzébio Batista (FACENSA College, Gravatai, Brazil), Jorge Luis Victória Barbosa (University of Vale do Rio dos Sinos (UNISINOS), Sao Leopoldo, Brazil), João Elison da Rosa Tavares (FACENSA College, Brazil) and Jonathan Luís Hackenhaar (University of Vale do Rio dos Sinos (UNISINOS), Sao Leopoldo, Brazil)
DOI: 10.4018/jicte.2013070103
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

This article shows the application of Item Response Theory (IRT) for educational evaluation using games. The article proposes a computational model to create user profiles, called Psychometric Profile Generator (PPG). PPG uses the IRT mathematical model for exploring the levels of skills and behaviors in the form of items and/or stimuli. The model was integrated with an educational game. The game was created considering a test applied by the Brazilian Government to evaluate students in national level, called Brazil Exam. The integration was used to evaluate the model in an educational scenario involving one hundred and thirteen students with an average age of 11 years old of a school in the south of Brazil. The results show PPG conducts an accurate evaluation because it considers not only the number of questions correctly answered but the proportional difficulty of each question in the evaluated group.
Article Preview

Introduction

Rossous (2007) affirms the information and communication technologies have assumed a critically important role in almost every facet of our society. He also highlights the competence in using computers is not only an advantage, but also often a necessity.

When the student is evaluated without a methodology, ends up getting a concept note or other classification which had no sense to improve the knowledge or to enhance the review of what was not understood to be rethought and improved. An evaluation methodology should provide subsidies, favoring the development and evolution, in order to make the most of the student.

In last years, the Brazilian government has implemented a form of evaluation at national level. Once a year, a test is applied to students in public education. This test is called Brazil Exam (in Portuguese “Prova Brasil”) and its main goal is to get data on the ability and aptitude of the students. Currently, the Brazil Exam is implementing the Item Response Theory (IRT) (Hambleton & Swaminathan, 1985; Rainer & Miller, 1996), that consists of a mathematical model which exploits the level of skill or learner behavior. According to Zlomke (2009), the IRT may be a useful framework for examining the reliability and latent traits of the measures assessed.

Based on the IRT, this article proposes the PPG (Psychometric Profile Generator) model. The model is composed by different evaluation environments and prospect profiles. In order to assess the PPG model, we constructed an educational game based on it. The game was used in a case study involving one hundred and thirteen students in the 5th grade of basic education with average age of 11 years old.

The article is divided into seven sections. The next section presents the psychometrics and mathematical model used. The following section describes the computational model proposed and its operation. Afterwards we present the case study based on the educational game. Next we discuss the approach to the results of the case study. Followed by a section that compares the PPG with three related works. Finally, the last section approaches the final considerations.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 14: 4 Issues (2018): 1 Released, 3 Forthcoming
Volume 13: 4 Issues (2017)
Volume 12: 4 Issues (2016)
Volume 11: 4 Issues (2015)
Volume 10: 4 Issues (2014)
Volume 9: 4 Issues (2013)
Volume 8: 4 Issues (2012)
Volume 7: 4 Issues (2011)
Volume 6: 4 Issues (2010)
Volume 5: 4 Issues (2009)
Volume 4: 4 Issues (2008)
Volume 3: 4 Issues (2007)
Volume 2: 4 Issues (2006)
Volume 1: 4 Issues (2005)
View Complete Journal Contents Listing