Building the Multidimensional Semantic Index of Webpages for Facet Extraction

1. Introduction

In the big data era, faceted search has already changed the mechanism of search. For example, on Amazon, commodities are indexed from different facets such as style, colour, size, and so on. With the support of faceted index, buyers can specify one or more facets to narrow their search, which improves the efficiency of search to a great extent. Compared with the traditional search, the efficiency of faceted search contributes to its semantic index. The traditional search engine uses the inverted index, which lacks of semantics and just maps terms to webpages to provide fast access to resources. The faceted search engines index resources from different facets at the same time and each facet has its own semantics.

In most of the current faceted search systems, facets of resources are specified by hand, which is easy to do in some applications such as e-commerce system. The facets of commodities are easy to determine, and the values of each facet is also easy to specify. For example, most of the commodities, such as TV, computer, food, car, and so on, have their description parameters which have been widely used by buyers and sellers. In fact, these parameters are actually selected as facets by e-commerce system to provide faceted search.

However, there are massive unstructured webpages on the Web, such as news, papers, articles, blogs, and so on. It is difficult to choose the facets or facet values from massive webpages by hand, which leads to the lack of faceted search systems on these kinds of resources. Take news service for example. There are two kinds of news services: one is news browse service, such as Yahoo, BBC, Routers, and so on. In this kind of websites, news is simply divided into several big classes, such as entertainment, sports, policy, military and so on. The other is news search services, such as Google News, Baidu News, and so on. In this kind of websites, news is indexed by terms from its content. Although in advanced search, the news can be searched by time, classifications, origins, they are weakly related to the news content and helpless to searching news efficiently. Therefore, it is eager but difficult to provide faceted search on massive unstructured webpages, in which, how to automatically choose facets and determine the values of each facet on massive unstructured webpages are the basic issues to be solved.

This paper proposed a novel index structure of webpages, named as Multidimensional Semantic Index (MDSI), which holds rich semantic and can be used to extract facets. In this paper, MDSI is first defined and constructed by mining semantic relations among the terms from webpages. Then, the differently dimensional semantic indexes of MDSI are bridged by mining the semantic mapping between them. At last, facets are extracted by analyzing semantic mapping relations in MDSI. In the last section, the experimental results on two datasets both show that the proposed method is feasible and comparatively precise. The contributions of this paper include:

• 1.

  Propose a novel index structure of webpages, the Multidimensional Semantic Index, which holds rich semantic and can be used to extract facets from unstructured webpages;

• 2.

  Propose an automatic facet extraction method based on MDSI, which can extract facets from massive webpages automatically.

2. Related Work