Manipulation of a large amount of data becomes a very tedious task. Hence, the authors took the approach of memory networks for the implementation of the chatbot. Traditionally, the LSTM model was used to implement chatbots and QA systems. But the LSTM failed to store relevant information when given a longer information set. On the contrary, the memory networks have an additional memory component with it. This can help in storing long information for further use which is greatly advantageous for the QA and chatbot systems as compared to LSTM. The authors trained and tested their model over Facebook's bAbi dataset which consists of several tasks and has questions regarding each task to retrieve the accuracy of the model. On the pedestal of that dataset, they have presented the accuracy for every task in their study with memory networks.
TopIntroduction
The digital age has bombarded mankind with a large number of devices. The devices stand to retrieve information based on the data they are suspected to. Also, give the users the response they need in the large retrieval-based world of a humongous amount of data that resides in the world. The digital devices rely on information retrieval systems such as a chatbot system. The question answering chatbot system strives to retrieve information from a large repository of data with extreme precision and lesser redundancy. This retrieval system is implemented with the foundation of Artificial Intelligence (AI), (Ainouz, S. A. Ben Ahmed, Mohammed, 2020). Also, a chatbot is more like a human and requires intelligence. Artificial Intelligence techniques are essential in its implementation, (M. Lewkowitz, 2014, Feb 12). One of the techniques to be considered as a part of AI is Machine learning (ML). ML in layman terms can be defined as the ability of a machine to learn on its own from the data it is provided and create a prediction or a decision based on the algorithm that is fed into the machine. The chatbot system also requires techniques to mimic a human brain to generate an accurate response, (Bing Liu1, G. T., Hakkani-Tur, P. S. Heck). That is where Deep Learning comes into the picture showing the neural network similar to nerves in the brain of a human.
A chatbot system can be categorized as a retrieval based system and a rule based system, (Jincy Susan Thomas, Seena Thomas, 2018). The proposed system is a chatbot that uses Memory Networks based approach which makes it better than rule-based systems. This system is a Retrieval based system as the bot is trained on a set of questions and their possible outcomes. Memory networks as the name suggests possess an external memory unlike other Neural networks. Having a memory module becomes extremely necessary as the questions sometimes can tend to be so long that a common neural network would fail to backpropagate entirely. Thus, the external memory module can come into play whenever needed in order to backpropagate and process an entire question, (Sainbayar Sukhbaatar, Arthur Szlam Jason Weston Rob Fergus). The instruction set given to the bot makes it possible to get the answer from the dataset it is trained on inorder to get the most relevant answer and output the same. A good and efficiently pre processed dataset can enable the chatbot to produce new answers. Facebook engineers combined a dataset named bAbi inorder to be used as a task response system. The bAbi set consists of 20 tasks that have variable answers, (Jason Weston, Bordes, S.C., Alexander M. Rush, Bart van Merrienboer, Armand Joulin Tomas Mikolov). We have used this dataset to train our chatbot.
The structure of the chatbot that the authors propose has been illustrated in figure 1. The figure 1 shows that whenever an user asks a question, it does the analysis of the question. Further, it retrieves the necessary document that might have an answer for the question for e.g. “where” questions will have answers in “places” documents. Then it retrieves the answer and analyzes it for it’s correctness and finally displays it to the user.
Thus this chatbot has been trained on the machine learning model of memory networks. As, training them end to end requires very less supervision during training which makes it useful in regular scenarios.