New Formats and Interfaces for Multi-Document News Summarization and its Evaluation

Abstract

News production, delivery, and consumption are increasing in ubiquity and speed, spreading over more software and hardware platforms, in particular mobile devices. This has led to an increasing interest in automated methods for multi-document summarization. The authors start this chapter with discussing several new alternatives for automated news summarization, with a particular focus on temporal text mining, graph-based methods, and graphical interfaces. Then they present automated and user-centric frameworks for cross-evaluating summarization methods that output different summary formats and describe the challenges associated with each evaluation framework. Based on the results of the user studies, the authors argue that it is crucial for effective summarization to integrate the user into sense-making through usable, entertaining, and ultimately useful interactive summarization-plus-document-search interfaces. In particular, graph-based methods and interfaces may be a better preparation for people to concentrate on what is essential in a collection of texts, and thus may be a key to enhancing the summary evaluation process by replacing the “one gold standard fits all” approach with carefully designed user studies built upon a variety of summary representation formats.