In this chapter, the authors present their system, which can use natural language query to interact with heterogeneous information networks (HIN). This chapter proposes a solution combining the GraphFrames, recurrent neural network (RNN) long short-term memory (LSTM), and dependency relation of question for generating, training, understanding the question-answer pairs and selecting the best match answer for this question. The RNN-LSTM is used to generate the answer from the facts of knowledge graph. The authors need to build a training data set of question-answer pairs from a very large knowledge graph by using GraphFrames for big graph processing. To improve the performance of GraphFrames, they repartition the GraphFrames. For complicated query, they use the Stanford dependency parser to analyze the question and build the motif pattern for searching GraphFrames. They also develop a chatbot that can interact with the knowledge graph by using the natural language query. They conduct their system with question-answer generated from DBLP to prove the performance of our proposed system.
TopIntroduction
As the appearance of large knowledge graphs (KB) such as DBpedia, Freebase, Wikipedia, DBLP, many researchers are paying more attention to question answering over knowledge graph by using the natural language query. Using natural language based query is more convenient and user friendly. Natural language query is a question such as “who is the author of paper p0001?”. The answer of this question will be generated from a triples such as <subject, predicate, object> in the knowledge graph. DBLP is a bibliographic information network which is a kind on HIN. We consider DBLP as a knowledge graph. DBLP contains a large volume of knowledge triples which can support the researchers to find the knowledge in a specific research area.
Large graph is a research direction of big data processing. Large graph can be considered as a big linked data. In the context of big data there are several research works to use a distributed computing system to process big data. According to (Rodrigo Agerri, 2015) a solution was proposed to process huge amount of textual data in the Natural Language Processing (NLP) research area. A new approach for scalable distributed language processing with multiple computers was proposed. Apache Hadoop is a framework to perform large scale processing. Hadoop implements MapReduce, a programming model for developing parallel and distributed algorithms that process and generates large data sets and large graph. Moreover, Hadoop and MapReduce is not faster than Apache Spark which run mainly in the memory. Apache Spark can contain one master and several workers. The tasks in workers are processed concurrently. When all workers finish their tasks, the result will be sent back to the master machine of distributed system. Nowadays, the trend of big data processing is the distributed computing on the cloud computing environment.
According to (Paolo Nesi, 2015) a distributed computing system includes computer clusters and parallel computing paradigms have been used to process data-intensive contexts such as large volume of text. A system based on Hadoop with multiple nodes and MapReduce are used to process GATE application framework. Gate is a set of tools for natural language processing. The performance of NLP processing of this system for big data processing is improved significantly.
In our system, the Stanford Parser is used to classify the question. It analyses the question and detects the number of objects in the question and based on this number, the question will be classified simple or complicated question. The RNN-LSTM model is used to find the answer for the simple question and the motif search of GraphFrames Library is used for complicated question. The answer is built based on the facts (triples) in knowledge graph. This chapter will present how to generate the question-answer pairs of training data set for training the neural network by using GraphFramesF Library of Apache Spark. This chapter explains how to build the motif pattern of motif search of GraphFrames to access the triples of large scale knowledge graph. By combining GraphFrames Library and the RNN-LSTM model, our proposed system can improve the performance and the accuracy of query. In this research, knowledge graph is considered as a large HIN and the processing of large HIN is a difficult problem of big data analysis. In knowledge graph, each triple is expressed by two vertices and arc connecting subject vertex to object vertex. The predicate name is the label of arc connecting two vertices. There are millions of triples in a knowledge graph, so the knowledge graph is really a very large graph. Graph can be saved in Neo4j Graph Database and accessed by graph query language. In Neo4j graph database, Cypher is a graph query language to query a graph database. When user gives a query, the graph database will be accessed to provide the facts for the answer. Hadoop is a good platform for distributed computing with multiples computers. Since we work with a large scale knowledge graph, GraphFrames Library of Apache Spark for processing large scale graph on Hadoop distributed computing environment is used.