Many different measures for evaluating the performance of information retrieval systems have been proposed. The evaluation of QA systemS has been carried out by the traditional evaluation measures based on the relevance (precision and MAP), and also, by other specific measures like MRR, TRR, and FHS.
Published in Chapter:
Searching Health Information in Question-Answering Systems
María-Dolores Olvera-Lobo (CSIC, Spain & Universidad de Granada, Spain) and Juncal Gutiérrez-Artacho (Universidad de Granada, Spain & Universidad Pablo de Olavide, Spain)
Copyright: © 2013
|Pages: 17
DOI: 10.4018/978-1-4666-3986-7.ch025
Abstract
Question-Answering Systems (QA Systems) can be viewed as a new alternative to the more familiar Information Retrieval Systems. These systems try to offer detailed, understandable answers to factual questions, in order to retrieve a collection of documents related to a particular search (Jackson & Schilder, 2005). The authors carry out a study to evaluate the quality and efficiency of open- and restricted-domain QA systems as sources for physicians and users in general through one monolingual evaluation and another multilingual. Their objective led them to use definition-type questions in order to evaluate QA systems and determine if they are useful to retrieve medical information. In addition, they analyze and evaluate the results obtained, and identify the source or sources used by the systems and their procedure (Olvera-Lobo & Gutiérrez-Artacho, 2010, 2011).