Reliability and validity have a well-established place in the development and implementation of educational assessment devices. With the advent of electronic delivery and assessment some of the factors that influence reliability and validity have changed. In order to understand the process involved the authors have suggested that a socio-technical approach to these educational issues gives an economical explanatory system. Within this socio-technical system, the authors show that the way the students extract information from sources is changing to an extent where it is difficult to distinguish between cheating and poor quoting behavior. This has led them to postulate a new classification within validity and reliability – knowledge validity and reliability. They argue that electronic delivery and assessment have not changed their core structures, but rather require revised education and training for both staff and students.
What is validity and reliability in e-learning and assessment? To answer this question, we shall first provide an introduction to the broad conceptual structure that defines reliability and validity in educational assessment. We will then be in a position to begin the process of mapping this structure into the evolving world of e-learning. We shall attempt to illustrate key points through case studies from real world teaching issues so that the ideas being presented do not stand in theoretical isolation. We shall also introduce the ideas presented by a small sample of teachers who were interviewed in developing the chapter.
At the core of what we shall present is the idea that e-learning, in its many forms and definitions, has created an environment where reliability and validity need to be updated from the traditional thinking. Very few Higher Education (HE) institutions have any formal requirement that staff establish the reliability and validity of their assessments. With the advent of the electronic environment, there has been a pragmatic concern for validity through the plagiarism issue. We shall explore this.
Validity and reliability need to have a context within a technology enhanced assessment environment. We argue that this is best done through basic socio-technical thinking, for the simple reason that this emphasizes the importance of the interaction between the human and technical components in the educational system. Additionally, it reinforces the need to understand the transactions that take place between system elements. We will show that this helps us develop a future oriented approach to reliability and validity in e-learning systems.
One particularly revealing comment about the effect that e-learning has had upon assessment validity is that “conventional take away assignments bleed too much” (in the internet environment). The respondent was concerned that validity almost completely disappears once you let a student walk away with an assignment. Not only do students seek help from on-line articles (“googling the answer” is a widely understood phrase amongst e-learning students), but they also seek the services of third party “assignment sub-contractors”, who often advertise on the web, and who are almost untraceable in terms of identifying the real authors.