On-Line Testing is that sector of e-learning aimed at assessing learner’s knowledge through e-learning means. In on-line testing, due to the necessity of evaluating a big mass of learners in strict times, the means for knowledge evaluation had to evolve to satisfy the new necessities: objective tests, more rapidly assessable, started gaining more credence in the determination of learners’ results. In this chapter, the authors present an On-Line Testing system, named eWorkbook, which can be used for evaluating learner’s knowledge by creating (the tutor) and taking (the learner) on-line tests based on multiple choice question type. Its use is suitable within the academic environment in a blended learning approach, by providing tutors with an additional assessment tool, and learners with a distance self-assessment means. Among other features, eWorkbook can record and visualize, in a suitable graphical format, learner’s interactions with the system interface during the test session. This is valuable information for understanding the learner’s behaviour when taking a test. In particular, the graphical analysis of the test outcomes has helped us in the discovery of several strategies employed by the learners to perform the test. In the paper, the main characteristics of the system are presented together with a rationale behind them and an outline of the architectural design of the system.
In blended learning the electronic means are mixed with the traditional didactics, in order to train and to assess the learners. Learning Management Systems (LMS), enhanced with collaborative environment support, and On-Line Testing tools are more and more widely adopted in the academy. At the University of Salerno some systems and platforms have been tested to support blended learning. Even if some good existing LMS with On-Line Testing capabilities, such as Moodle (Moodle, 2005) and Sakai (Sakai, 2005) have been used, none of them satisfied us at all: we needed an advanced assessment tool which could have helped the lecturers to speed up the onerous task of assessing a huge mass of learners and should have provided the tutor with valuable information for evaluating the whole assessment process.
A state of the art analysis undertaken at our department, which involved several lecturers and students, allowed us to identify the following important requirements for an effective environment for developing and using assessment tests:
Item sharing features;
Didactics organized in courses and classes;
Possibility of administering both self-assessment tests and proctored laboratory exams;
Availability of statistics on tests and questions;
Availability of a rich reporting section on test outcomes.
A project for a comprehensive Web-based assessment system, named eWorkbook, was then started. The system can be used for evaluating a learner’s knowledge by creating (the tutor) and taking (the learner) on-line tests based on multiple choice question types. Even though eWorkbook enables the creation of on-line tests for both assessment and self-assessment, it was planned above all for summative (evaluation) purposes. The questions are kept in a hierarchical repository, that is, it is tree-structured, in the same way as the file system of an operating system. In such a structure, the files can be thought of as questions, whether the directories can be thought of as macroareas, which are containers of questions usually belonging to the same subject. Every item (a macroarea or a question) has an owner, which is the tutor who authored it. The tutors can choose whether to share their questions or not, assigning a value to the permissions associated to each item. Permissions are for reading, writing and using the items.
The tests are composed of one or more sections. This structure facilitates the selection of the questions from the repository, but it is still useful for the assessment, where it can be important to establish if one section is more important then another to determine the final grade for the test. The selection of the questions can occur both statically, by directly choosing the questions from the repository, and dynamically, leaving the system to choose the questions randomly.
Didactics are organized into courses and classes: the tutors responsible for a course manage its class and choose the tests that must be taken by the learners of that class. With such an organization, the system can be used by a large set of users, such as the learners and the tutors of an entire faculty. Within a course interface, the learner can easily access the self-assessment tests. Restrictions on the access rules can be defined for proctored laboratory tests.
Different assessment strategies can be bound to a test, before it is published in a course. The assessment strategy affects the way in which some parameters concur to determine the grade of the test. Some strategies are preloaded in the system and are referred to as predefined assessment strategies. Others can be defined by the tutors and saved in his/her reserved area. We will refer to them as customized assessment strategies.