1.1. Background
The recent development of the Internet and ubiquitous technologies has enabled us to generate writing contents by bots. For example, they can use the API provided by Twitter to tweet sentences (Makice, 2009), whereas some physical objects with sensors connected to the Internet post their ``activities” to a blog (Maekawa et al., 2008). These technologies allow us to maintain a diary automatically. However, contents generated automatically by bots, usually consist of stereotyped expressions produced using a simple language model such as an N-gram model and thus they are monotonous and consist of insubstantial descriptions that do not reflect dynamic aspects of real-world situations. They stand out in sharp contrast to contents posted by (human) users, which are of a wide variety and contain informative descriptions that reflect real-world dynamics.
On the other hand, by analyzing web pages, many researches try to infer situations or states of the real world. For example, Bollen et al. (2011) measured the public mood from tweets submitted to Twitter to predict the stock market. Using RFID tags, Perkowitz et al. (2004) formulated activity models by translating labeled activities, such as “cooking pasta,” into probabilistic collections of object terms, such as “pot.” Given this view of activity models as text translations, they defined activities from the web in an unsupervised manner. The fruitful results of these researches suggest that reusing web pages relevant to events and activities of daily life may provide us with an approach to automatically produce essays associated with daily events and activities.
This study aims at retrieving many web pages relevant to everyday events and activities that can be used as subject matter for the automatic generation of writing contents. In particular, this paper describes a web search method whose query is assumed to be a “verb” expressing a daily event or activity and a “noun” representing an everyday physical object.
Search result diversification has gained momentum as a way to tackle ambiguous queries. An effective approach to this problem is to explicitly model the possible aspects underlying a query to maximize the estimated relevance of the retrieved documents with respect to the different aspects. Such aspects themselves, however, may represent information needs with rather distinct intents (Agrawal et al., 2009; Carterette & Chandar, 2009; Santos, Macdonald, & Ounis, 2010). Hence, a diverse ranking method could benefit from the application of intent-aware retrieval models when estimating the relevance of documents with respect to different aspects.
Web-search methods using “specialized search engines,” which tackle ambiguous queries, have been proposed to narrows down a search. The most frequently used specialized search engines are shopbots known as shopping agents or recommendation engines (Clark, 2000). As a methodology for category-specific web search using a general purpose search engine, Glover et al. (2001) introduced a trained classifier that can recognize pages of a specified category with high precision and a query modification that is a set of extra words or phrases added to a user query.
Specialized search engines consist of two steps: page type determination and page collection and classification. The former retrieves pages of the type that match the purpose of the user, as a candidate set of pages to be searched. The latter enables a user to search for relevant pages in the candidate set of pages. Because type determination is a task involving a large amount of pages on the web, it requires much time to process hence, it is performed as a pre-process. Therefore, it is not practical to perform the type determination for each genre.
Hirokawa (2001) proposed a search method that unifies existing search engines, and can deal with various genres without the time-consuming type determination. That is, the method selects the search engine that seems to suit a given query and returns the pages that the selected search engine returns for the query. However, in this approach unless we have a specialized search engine, we cannot search pages specific to a genre.
Also, many researches use machine learning techniques for web searches for ambiguous queries. Santos, Macdonald, and Ounis (2011), for example, proposed to diversify the results retrieved for a given query, by learning the appropriateness of different retrieval models for each of the aspects underlying the query.
This study aims at developing a search method that, for a given genre that is characterized by a genre term, retrieves relevant pages without depending on a specialized search engine. That is, instead of filtering out pages associated with a given genre in the page collection phase, the method re-ranks web pages that are retrieved using a search engine on the basis of the scores calculated from an expression consisting of weighted factors such as the frequency of query words. For each of the genres, the method optimizes the weights of the expression. It also tries to improve the scores of relevant pages by using machine learning techniques.