Principles of Natural Language Processing and Adaptive Courseware in E-Assessments: Empirical Evaluations

Introduction

While used in everyday learning environments, the knowledge e-assessment encompasses several contexts: the type of the delivery platform, the task types used during the assessment and the role of time in the process of assessment. Intelligent tutoring systems are effective e-learning platforms that provide immediate and customized instruction and feedback to students, but these tools are often customized for a single knowledge domain due to higher development costs. Over the last few decades, the researchers put efforts to improve intelligent tutoring systems’ abilities with the aim to get them as close as possible to the ultimate goal of one-to-one tutoring. These enhancements include the communication between the learner and the system in a more natural way and the adaptive behavior, which both are still the ongoing research topics in the scientific society.

Although in the beginnings the natural language was declared as Achilles' heel of intelligent tutoring systems (Rickel, 1989) the numerous language technologies were developed, prevalently for the English language. Despite it is known that intelligent tutoring systems with the ability to process even simplest form of natural language are more effective than those without such ability (Eugenio, Fossati, Yu, Haller, & Glass, 2005; Fox, 1993), only a handful of intelligent tutoring systems implement some form of natural language processing.

The additional improvement of intelligent tutoring systems would be to adapt the process of learning, teaching, and testing of knowledge to the different characteristics of a student. If a courseware represents the domain knowledge of the subject we want to teach the students, the automation of courseware adaptability in every stage of learning, teaching and testing process is the desired intelligent tutoring system functionality. More specifically, the automatic generation of courseware elements, dynamic selection and sequencing of courseware elements, and automatic generation of tests and questions should be implemented using adaptation to student knowledge.

For the last few decades, the researchers from the Faculty of Science, University of Split, are focused on the design, development, and research of intelligent tutoring systems that share the characteristic of ontological domain knowledge representation. The designed and developed systems include the idea of iterative processes of learning and testing until the learner is taught at a certain knowledge level. These systems are not domain specific and therefore can be used with various knowledge bases. Controlled Language Based Tutor (CoLaB Tutor) was developed in 2010 and its forte lies in teacher-learner communication in controlled natural language (Žitko, 2010). Adaptive Courseware Tutor (AC-ware Tutor) was developed in 2012 and it is focused on the automatic and dynamic generation of adaptive courseware based on learner stereotypes, Bayesian networks and Bloom's knowledge taxonomy (Grubišić, 2012). In term of knowledge e-assessment, CoLaB and AC-ware Tutor include different approaches. CoLaB Tutor uses tutorial dialogue in controlled Croatian language, while AC-ware Tutor adaptively generates several types of objective test items. In this chapter, the several experimental environments that investigated CoLaB Tutor and AC-ware Tutor supported education will be presented, mostly using the summative knowledge e-assessment method with different learning approaches and various education levels.

In terms of evaluations of the intelligent tutoring systems supported education, there are some common beliefs in the research society about the effectiveness of different types of tutoring. In the following ‘Background’ section, the related research work on the effectiveness of intelligent tutoring systems will be presented. Furthermore, in the ‘Ontological domain knowledge representation in CoLaB Tutor and AC-ware Tutor’ section, the descriptions of two examined systems will be presented. Following this, five empirical evaluations of CoLaB Tutor and AC-ware Tutor conducted since 2010 will be described including the details of methodologies and results in ‘The empirical evaluations of CoLaB Tutor and AC-ware Tutor’ section. After that, the discussion and some concluding remarks regarding intelligent tutoring systems’ effectiveness results in various environments will be offered in the ‘Discussion’ and ‘Conclusion’ sections.