Conceptual Graphs Based Approach for Subjective Answers Evaluation

Conceptual Graphs Based Approach for Subjective Answers Evaluation

Goonjan Jain (Jawaharlal Nehru University, New Dehli, India) and D.K. Lobiyal (Jawaharlal Nehru University, New Dehli, India)
DOI: 10.4018/IJCSSA.2017070101
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

Automated evaluation systems for objective type tests already exist. However, it is challenging to make an automated evaluation system for subjective type tests. Therefore, focus of this paper is on evaluation of simple text based subjective answers using Natural Language Processing techniques. A student's answer is evaluated by comparing it with a model answer of the question. Model answers cannot exactly match with the students' answers due to variability in writing. Therefore, researchers create conceptual graphs for both student as well as model answer and compute similarity between these graphs using techniques of graph similarity measures. Based on the similarity, marks are assigned to an answer. Lastly, in this manuscript authors compare the results obtained by human graders and the proposed system using Pearson correlation coefficient. Also, comparison has been drawn between the results of proposed system with other existing evaluation systems. The experimental evaluation of the proposed system shows promising results.
Article Preview

Introduction

Tests are an indispensable part of our education system and are used to gauge student’s knowledge. Two basic categories of tests are Objective and Subjective tests. Objective tests require student to only select a correct response from several alternatives or utmost to supply a word or short phrase to answer a question. On the contrary, subjective tests require test taker to form an original answer based on their own understanding of topic.

Subjective tests are useful in measuring educational objectives such as analysis, and synthesis. They permit freedom of response that in turn allow students to present their ideas and knowledge they have in the subject area covered by the question. With this, subjective tests also promote development of problem solving skills and eliminate guessing.

Despite the unique benefits of subjective tests, their automation is very limited when compared with automation of objective tests. Major reason for this is that there are downfalls when it comes to marking subjective exams electronically. They have lots of complexities like, different ways of answering the same question; some question requires diagrams or equations. Some questions require answer in more than one word and length of answer may even vary from individual to individual. On the contrary, it is simple and efficient to mark objective item tests because a computer can easily process multiple choice responses.

In this paper, authors propose a unique automatic grading system that can assign grades to short subjective answers provided by student. The system does so by using a model or template answer as a reference. Proposed technique is unique in itself because none of the existing techniques leverages semantics of the text to evaluate it. This limitation significantly hampers effectiveness of technique as questions can be answered in many ways based on candidate's understanding, strength of vocabulary and expressive nature. In addition to this, considering only syntax and morphology will not be sufficient to accurately mark a subjective answer. A correct answer can take many shapes as long as its semantic and syntactic structure is correct. Therefore, it is very important to understand the way the answers are paraphrased for representing the importance of words. This can be achieved by understanding semantics of text and the authors have employed Conceptual Graphs for this.

“Conceptual Graphs (CG) are a system of logic which is based on the existential graphs, the semantic networks of artificial intelligence. Conceptual graphs are described as the logical form that state relationships between concepts and thus represents a meaning.” (Sowa, 1984) They were originally developed as a representation that could help in understanding the meaning of natural language text. They have been successful in capturing the nuances in natural language along with the ability to be implemented in computer software. Conceptual Graph is a knowledge representation technique that expresses meaning in a precise, readable and computationally tractable way and therefore is an efficient tool for information and knowledge processing.

The CGs are labelled graphs and have two types of nodes – concept and relation nodes. A concept node is a discrete unit of perception and represents entities, attributes, states, and events (Sowa, 1984). There are two types of fields in a concept node - type and referent. The type field specifies the class of entity represented by the concept and an instance of the concept type is identified by a referent. The interconnection of concept nodes is shown by relation nodes. Each conceptual relation has n arcs which are linked to some concept. Every conceptual relation has a relation type and a valence which tells the number of arcs belonging to the relation. Consider an example sentence “Employees get the bonus”. The CG representation for this sentence is shown in Figure 1.

Figure 1.

Conceptual Graph Representation for Sentence “Employees Get the Bonus”

Since CGs are a good way to capture semantics of text, authors have used an NLP approach along with conceptual graphs to understand the semantics of students’ answers and then evaluate it.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 5: 2 Issues (2017)
Volume 4: 2 Issues (2016)
Volume 3: 2 Issues (2015)
Volume 2: 2 Issues (2014)
Volume 1: 2 Issues (2013)
View Complete Journal Contents Listing