A Comparison of Corpus-Based and Structural Methods on Approximation of Semantic Relatedness in Ontologies

Introduction

Ontologies are the backbone of Semantic Web information systems. They are designed to provide a shared understanding of a domain and support knowledge sharing and reuse (Fensel, 2004). Recently, attention has been devoted to using ontologies to improve the performance of information retrieval (Castells et al., 2007) and extraction systems (Ruotsalo et al., 2009), and to support tasks such as query expansion (Kekäläinen & Järvelin, 2000), knowledge-based recommendation (Ruotsalo & Hyvönen, 2007), word sense disambiguation (Ide & Véronis, 1998), and text summarization (Lin & Hovy, 2000).

The ontologies used by such systems are often light-weight general purpose concept ontologies that provide conceptualizations suitable to be used in many domains and applications, but without a manual effort they can not be expected to explicate all the relations required in specific sub-domains (Chandrasekaran et al., 1999). For example, a user searching for objects annotated with the concept flu on a health portal could be offered articles indexed with the concepts respiratory infection or pneumonia. On the other hand, the user could be interested in news related an ongoing flu epidemic with related content indexed with concepts such as vaccinations, nutrition or medication.

Avoiding manually tailoring the ontologies, but still enabling such functionalities can be enabled through augmenting relations by estimating relatedness of concepts. Estimates of semantic relatedness can be obtained by making use of structural measures that approximate the relatedness based on the structure of the ontology (Budanitsky & Hirst, 2006). On the other hand, the mentioned applications deal with unannotated corpora that can be used as a source for learning the relations (Landauer et al., 1998; Blei et al., 2003).

While good results have been obtained using both of the approaches (Landauer et al., 1998; Budanitsky & Hirst, 2006), a comprehensive empirical comparison of the approaches has not been reported. To address this, we compare the performance of a widely used corpus-based method, Latent Semantic Analysis (Landauer et al., 1998), and two well-known ontological structural measures, a conceptual measure proposed by Wu & Palmer (1994), and a path-length measure by Leacock & Chodorow (1998).

We report results of a large user study comparing these approaches in semantic relatedness approximation. The focus of the study is to (1) determine the accuracy of the methods, (2) determine the difference between corpus-based methods and structural measures, and (3) identify the strengths and weaknesses of the methods in potential application scenarios.

We show that good accuracy can be achieved using both types of methods, but the methods provide clearly distinct approximations. The results suggest that the approaches are complementary. Structural measures alone can be adequate in scenarios such as information extraction, where synonymy and hyponymy relations suffice (Califf & Mooney, 1999). The combination of methods could be beneficial in scenarios such as information retrieval or word sense disambiguation, where an extensive word context is found to be important (Kekäläinen & Järvelin, 2000; Sussna, 1993). In addition, the results suggest that the performance of the methods are dependent on the correct combination of the methods and assignment of appropriate cut-off values to ensure optimal performance.

The rest of this paper is structured as follows. The following section introduces the semantic relatedness approximation methods used. Section 3 describes the empirical study. The results of the study are presented in section 4. Finally, we conclude with a summary of results, a discussion of shortcomings, and suggestions for future work.