The process of converting all the characters in a document into the same case, either all upper case or lower case, in order to speed up comparisons during the indexing process.
Published in Chapter:
A Primer on Text-Data Analysis
Imad Rahal (College of Saint Benedict and Saint John’s University, USA), Baoying Wang (Waynesburg College, USA), and James Schnepf (College of Saint Benedict and Saint John’s University, USA)
Copyright: © 2009
|Pages: 8
DOI: 10.4018/978-1-60566-026-4.ch496
Abstract
Since the invention of the printing press, text has been the predominate mode for collecting, storing and disseminating a vast, rich range of information. With the unprecedented increase of electronic storage and dissemination, document collections have grown rapidly, increasing the need to manage and analyze this form of data in spite of its unstructured or semistructured form. Text-data analysis (Hearst, 1999) has emerged as an interdisciplinary research area forming a junction of a number of older fields like machine learning, natural language processing, and information retrieval (Grobelnik, Mladenic, & Milic-Frayling, 2000). It is sometimes viewed as an adapted form of a very similar research field that has also emerged recently, namely, data mining, which focuses primarily on structured data mostly represented in relational tables or multidimensional cubes. This article provides an overview of the various research directions in text-data analysis. After the “Introduction,” the “Background” section provides a description of a ubiquitous text-data representation model along with preprocessing steps employed for achieving better text-data representations and applications. The focal section, “Text-Data Analysis,” presents a detailed treatment of various text-data analysis subprocesses such as information extraction, information retrieval and information filtering, document clustering and document categorization. The article closes with a “Future Trends” section followed by a “Conclusion” section.