Article Preview
TopIntroduction
Tests are an indispensable part of our education system and are used to gauge student’s knowledge. Two basic categories of tests are Objective and Subjective tests. Objective tests require student to only select a correct response from several alternatives or utmost to supply a word or short phrase to answer a question. On the contrary, subjective tests require test taker to form an original answer based on their own understanding of topic.
Subjective tests are useful in measuring educational objectives such as analysis, and synthesis. They permit freedom of response that in turn allow students to present their ideas and knowledge they have in the subject area covered by the question. With this, subjective tests also promote development of problem solving skills and eliminate guessing.
Despite the unique benefits of subjective tests, their automation is very limited when compared with automation of objective tests. Major reason for this is that there are downfalls when it comes to marking subjective exams electronically. They have lots of complexities like, different ways of answering the same question; some question requires diagrams or equations. Some questions require answer in more than one word and length of answer may even vary from individual to individual. On the contrary, it is simple and efficient to mark objective item tests because a computer can easily process multiple choice responses.
In this paper, authors propose a unique automatic grading system that can assign grades to short subjective answers provided by student. The system does so by using a model or template answer as a reference. Proposed technique is unique in itself because none of the existing techniques leverages semantics of the text to evaluate it. This limitation significantly hampers effectiveness of technique as questions can be answered in many ways based on candidate's understanding, strength of vocabulary and expressive nature. In addition to this, considering only syntax and morphology will not be sufficient to accurately mark a subjective answer. A correct answer can take many shapes as long as its semantic and syntactic structure is correct. Therefore, it is very important to understand the way the answers are paraphrased for representing the importance of words. This can be achieved by understanding semantics of text and the authors have employed Conceptual Graphs for this.
“Conceptual Graphs (CG) are a system of logic which is based on the existential graphs, the semantic networks of artificial intelligence. Conceptual graphs are described as the logical form that state relationships between concepts and thus represents a meaning.” (Sowa, 1984) They were originally developed as a representation that could help in understanding the meaning of natural language text. They have been successful in capturing the nuances in natural language along with the ability to be implemented in computer software. Conceptual Graph is a knowledge representation technique that expresses meaning in a precise, readable and computationally tractable way and therefore is an efficient tool for information and knowledge processing.
The CGs are labelled graphs and have two types of nodes – concept and relation nodes. A concept node is a discrete unit of perception and represents entities, attributes, states, and events (Sowa, 1984). There are two types of fields in a concept node - type and referent. The type field specifies the class of entity represented by the concept and an instance of the concept type is identified by a referent. The interconnection of concept nodes is shown by relation nodes. Each conceptual relation has n arcs which are linked to some concept. Every conceptual relation has a relation type and a valence which tells the number of arcs belonging to the relation. Consider an example sentence “Employees get the bonus”. The CG representation for this sentence is shown in Figure 1.
Figure 1. Conceptual Graph Representation for Sentence “Employees Get the Bonus”
Since CGs are a good way to capture semantics of text, authors have used an NLP approach along with conceptual graphs to understand the semantics of students’ answers and then evaluate it.