Efficacy of Deep Neural Embeddings-Based Semantic Similarity in Automatic Essay Evaluation

Efficacy of Deep Neural Embeddings-Based Semantic Similarity in Automatic Essay Evaluation

Manik Hendre, Prasenjit Mukherjee, Raman Preet, Manish Godse
DOI: 10.4018/IJCINI.323190
Article PDF Download
Open access articles are freely available for download

Abstract

Semantic similarity is used extensively for understanding the context and meaning of the text data. In this paper, use of the semantic similarity in an automatic essay evaluation system is proposed. Different text embedding methods are used to compute the semantic similarity. Recent neural embedding methods including Google sentence encoder (GSE), embeddings for language models (ELMo), and global vectors (GloVe) are employed for computing the semantic similarity. Traditional methods of textual data representation such as TF-IDF and Jaccard index are also used in finding the semantic similarity. Experimental analysis of an intra-class and inter-class semantic similarity score distributions shows that the GSE outperforms other methods by accurately distinguishing essays from the same or different set/topic. Semantic similarity calculated using the GSE method is further used for finding the correlation with human rated essay scores, which shows high correlation with the human-rated scores on various essay traits.
Article Preview
Top

Introduction

Automatic Essay Evaluation is one of the oldest research area in the field of Natural Language Processing (NLP). Unlike multiple choice questions and short question answers, an essay is an open ended question. There is no fixed format and one can have multiple ways of writing an essay. Manually grading the essays is a very resource intensive task from the perspective of time and labour. Teachers have to spend their valuable time on grading the essays written by the students. If we have an automatic essay grading system then teachers can devote more time on the teaching part. An essay is used to assess one's understanding of the particular language. Because of which, TOEFL (2019) and GRE (2019) like exams has essay writing as one of the main component. Since last 5 decades researchers are developing solutions for automatic essay grading systems (Page, 1968; Christie, 1999; Rudner et al., 2006). In Natural Language Processing field there has been many advancements in last couple of years. We have more powerful language models which can perform various tasks as par with humans (Young et al., 2018). In tasks like sentiment Analysis, Chatbot, Question Answering, Automatic Essay Evaluation, Dialogue Systems, Parsing, Word-sense disambiguation, Named-Entity Recognition, POS Tagging and many more, we are observing good results (Young et al., 2018; Khurana et al., 2017; Cambria & White, 2014). The computing resources are more available and affordable now, as compared with couple of years back. Due to this, the research in NLP using Deep Learning Techniques is taking new leap in every field (Otter et al., 2020; Young et al., 2018; Deng & Liu, 2018).

Complete Article List

Search this Journal:
Reset
Volume 18: 1 Issue (2024)
Volume 17: 1 Issue (2023)
Volume 16: 1 Issue (2022)
Volume 15: 4 Issues (2021)
Volume 14: 4 Issues (2020)
Volume 13: 4 Issues (2019)
Volume 12: 4 Issues (2018)
Volume 11: 4 Issues (2017)
Volume 10: 4 Issues (2016)
Volume 9: 4 Issues (2015)
Volume 8: 4 Issues (2014)
Volume 7: 4 Issues (2013)
Volume 6: 4 Issues (2012)
Volume 5: 4 Issues (2011)
Volume 4: 4 Issues (2010)
Volume 3: 4 Issues (2009)
Volume 2: 4 Issues (2008)
Volume 1: 4 Issues (2007)
View Complete Journal Contents Listing