Smart Computerized Essay Scoring Using Deep Neural Networks for Universities and Institutions

Introduction

This work introduces a Natural Language Processing (NLP)-related project which is based on deep learning models for the problem of Automated Text Scoring (ATS). The resulting product is a web application named EssayScore that allows users to upload essays to be scored by a deep learning model with instant feedback. The approach to the deep learning model used is Long-Short-Term-Memory (LSTM) units, which is a subset of Recurrent Neural Networks (RNN) that deals with sequential dependency. There are three major aims to be completed for this work. The first aim is to investigate the application of deep learning techniques on the problem of automatic text scoring (ATS). Besides, the second aim is to develop a recurrent neural network (RNN) model that can score essays without human intervention. The final aim is to develop a web application that can accept user essays as input and grade them using the trained RNN model. To achieve the aims stated above, 10 objectives are identified that can contribute to the three aims above. The first objective is to perform a comprehensive literature review on the automated text scoring using neural networks and any existing work done in this research area. This is done to understand how recurrent neural networks and long-short-term memory (LSTM) units work such as their strengths and weaknesses. Besides, on the technical side, the researcher is required to learn how to develop a deep learning model using Keras framework and TensorFlow to train an RNN model for the work. The functional and non-functional requirements of the web application are established before the development of the system began and commentary on these requirements from a user’s perspective are made. Furthermore, a Software Requirement Specification (SRS) will also be written to document the requirements from a developer’s perspective. Next, UML diagrams of the web application are also created during the designing phase such as the use case diagram, class diagram, and sequence diagram. On the documentation side, the researcher also produced a work report on the developed automated test scoring (ATS) system which includes the analysis, evaluation, and conclusion and recommendation. Lastly, the last objective is to critically evaluate the product and process in the report and suggest some improvements in the finished product.

An ATS system aims to act as a “second-grader” for high stake tests to reduce the burden of the teachers. Besides, EssayScore also acts as an effective learning tool for students to gain instant feedback on their essays by reducing the time taken for students to get back their essay scores. For example, a feedback loop such as the one proposed by Paruchuri (2019) in Figure 1 is an effective use case of the ATS system in facilitating the learning process of students.

Figure 1.

Feedback Loop of an ATS system (Paruchuri, 2019)

EssayScore is mainly composed of two parts: the front-end of the application which handles user input and the backend which sends essays for grading. The deep learning model chosen will be a Long-Short-Term Memory model (LSTM), which is a kind of recurrent neural network (RNN) model that is useful in capturing temporal dependencies that are commonly present in essay texts. The deep learning model will be trained using Keras and TensorFlow and the deployment will be handled by TensorFlow Serving and Docker. The model will then act as the backend of the application which handles the scoring of the essay. The front-end is responsible for making REST API requests to the backend and populates the returned essay score to the user interface (UI).

Besides, EssayScore also provides spell checking feature for each uploaded essay as each word misspelled will be highlighted in red. The web application also provides recommendations for the misspelled word for users to choose from. Furthermore, uploaded essays and predicted scores will be stored in the Cloud Fire Store, a cloud-based storage and authentication platform from Google. Users of the web application will be able to view past submissions to see their learning progress.