eWorkbook: An On-Line Testing System with Test Visualization Functionalities

Introduction

In blended learning the electronic means are mixed with the traditional didactics, in order to train and to assess the learners. Learning Management Systems (LMS), enhanced with collaborative environment support, and On-Line Testing tools are more and more widely adopted in the academy. At the University of Salerno some systems and platforms have been tested to support blended learning. Even if some good existing LMS with On-Line Testing capabilities, such as Moodle (Moodle, 2005) and Sakai (Sakai, 2005) have been used, none of them satisfied us at all: we needed an advanced assessment tool which could have helped the lecturers to speed up the onerous task of assessing a huge mass of learners and should have provided the tutor with valuable information for evaluating the whole assessment process.

A state of the art analysis undertaken at our department, which involved several lecturers and students, allowed us to identify the following important requirements for an effective environment for developing and using assessment tests:

• •

  Item sharing features;

• •

  Didactics organized in courses and classes;

• •

  Possibility of administering both self-assessment tests and proctored laboratory exams;

• •

  Availability of statistics on tests and questions;

• •

  Availability of a rich reporting section on test outcomes.

A project for a comprehensive Web-based assessment system, named eWorkbook, was then started. The system can be used for evaluating a learner’s knowledge by creating (the tutor) and taking (the learner) on-line tests based on multiple choice question types. Even though eWorkbook enables the creation of on-line tests for both assessment and self-assessment, it was planned above all for summative (evaluation) purposes. The questions are kept in a hierarchical repository, that is, it is tree-structured, in the same way as the file system of an operating system. In such a structure, the files can be thought of as questions, whether the directories can be thought of as macroareas, which are containers of questions usually belonging to the same subject. Every item (a macroarea or a question) has an owner, which is the tutor who authored it. The tutors can choose whether to share their questions or not, assigning a value to the permissions associated to each item. Permissions are for reading, writing and using the items.

The tests are composed of one or more sections. This structure facilitates the selection of the questions from the repository, but it is still useful for the assessment, where it can be important to establish if one section is more important then another to determine the final grade for the test. The selection of the questions can occur both statically, by directly choosing the questions from the repository, and dynamically, leaving the system to choose the questions randomly.

Didactics are organized into courses and classes: the tutors responsible for a course manage its class and choose the tests that must be taken by the learners of that class. With such an organization, the system can be used by a large set of users, such as the learners and the tutors of an entire faculty. Within a course interface, the learner can easily access the self-assessment tests. Restrictions on the access rules can be defined for proctored laboratory tests.

Different assessment strategies can be bound to a test, before it is published in a course. The assessment strategy affects the way in which some parameters concur to determine the grade of the test. Some strategies are preloaded in the system and are referred to as predefined assessment strategies. Others can be defined by the tutors and saved in his/her reserved area. We will refer to them as customized assessment strategies.