Databases and corpora are essential resources to evaluate the performance of Natural Language Interfaces to Databases (NLIDB). The Geobase database and the Geoquery corpus (Geoquery250 and Geoquery880) are among the most commonly used. In this chapter, the authors analyze both resources to offer two elaborate resources: 1) N-Geobase, which is a relational database, and 2) the corpus Geoquery270. The former follows the standard normalization procedure, then N-Geobase has a schema similar to enterprise databases. Geoquery270 consists of 270 queries selected from Geoquery880, preserving the same kind of natural language problems as Geoquery880, but with more challenging issues for an NLIDB than Geoquery250. To evaluate the new resources, they compared the performance of the NLIDB using Geoquery270 and Geoquery250. The results indicated that Geoquery270 was the harder corpus, while Geoquery250 is the easier one. Consequently, this chapter offers a broader range of resources to NLIDB designers.
TopIntroduction
An NLIDB (Natural Language Interface to Database) is a system that can be used to access information in a database by typing sentences in a natural language such as English, French, or any other (Androutsopoulos, Ritchie, & Thanisch, 1995). For example, the sentence on the left side of Figure 1 is the query “Show all the employees called Laura and the customers whose orders have been generated by these employees”; for this sentence, the NLIDB on the right side of this figure will generate an equivalent query in SQL (Structured Query Language) to extract the information demanded by the user.
Figure 1. A simple query processed with a Natural Language Interface to Database (NLIDB)
Notable success on modern techniques to translate queries in natural language to SQL has been developed. From the 1960s to the 1990s, the first applications of NLIDBS were published, examples follow:
- •
In 1961, BASEBALL was introduced for a database about baseball games (Green Jr, Wolf, Chomsky, & Laugh, 1961).
- •
In 1972, LUNAR was applied for lunar rocks and soil composition (Woods, Kaplan, & Nash-Webber, 1972).
- •
In the 1980s, CHAT-80 was implemented in Prolog, and its database integrated by facts (true sentences in Prolog) relating to 150 of the world’s countries (Warren, 1981).
- •
In 1987, TEAM used a database that comprises geographic data (Grosz, Appelt, Martin, & Pereira, 1987).
- •
In 1989, PARLANCE was applied to different versions of a large Navy database (Bates, 1989).
- •
In 1990, JANUS was used for US Navy applications (Bobrow, Resnik, & Weischedel, 1990).
In the 1990s some spoken language understanding systems appeared: CMU (Ward, 1991), MIT (Seneff et al., 1991), SRI (Murveit et al., 1991), BBN Byblos (Kubala et al., 1992), and AT&T (Tzoukermann, 1991). All these systems were evaluated in relational databases, making the performance comparisons between NLIDBs. The database used by these systems was ATIS, of which some corpora of natural language queries are available in the literature (ATIS0 Complete, ATIS0 Pilot, ATIS0 Read, ATIS0 SD Read, ATIS2, ATIS3 Training Data, ATIS3 Test Data) (Consortium, 2019). The ATIS database consists of data obtained from the Official Airline Guide, organized under a relational schema. This guide contains information about flights, fares, airlines, cities, airports, and ground services (Hemphill,n.d.). Also, in the late 1990s and at the beginning of the next decade, NLIDBs as English Wizard was used using the Northwind database (Technology, 1997), two of them are English Query (Bhootra & Parker, 2004) and ELF (ELF software, 2002). Northwind is a relational database with information about a shipping company. Unlike ATIS, there is no corpus of natural language queries in the literature for Northwind. However, in several works, we can find small sets of queries that have been used (Tan, Lane and Rijanto, 2019; Bhootra and Parker, 2004; Nihalani, Motwani & Silka, 2011). At this time, the NLIDB, named NLPQC (Stratica, Kosseim, & Desai, 2005), was explicitly designed for the Cindi Virtual Library.