Arabic Stemmer Based Big Data

Arabic Stemmer Based Big Data

Youness Madani (Sultan Moulay Slimane University, Beni Mellal, Morocco), Mohammed Erritali (Sultan Moulay Slimane University, Beni Mellal, Morocco) and Jamaa Bengourram (Sultan Moulay Slimane University, Beni Mellal, Morocco)
Copyright: © 2018 |Pages: 12
DOI: 10.4018/JECO.2018010102

Abstract

By its morphological and syntactic richness, the Arabic language is considered among the most difficult languages to deal with it in the field of information search. This is due; in particular, to the various difficulties encountered in its Stemming, which has not yet experienced a standard approach. The Stemming algorithm for Arabic words has been an important topic in Arabic information retrieval. The intention of this article is to parallelize a stemming algorithm for Arabic by proposing a distributed stemming algorithm in a big data system. This is by using the Hadoop framework, the MapReduce programming model for the development of the algorithm, and the distributed file system HDFS for the Storage of stemming result.
Article Preview

1. Introduction

The Arabic language is the language of the Holy Quran. It is one of the six official languages of the United Nations and the mother tongue of approximately 300 million people. It is a Semitic language with 28 alphabet letters. Its writing orientation is from right-to-left. It can be classified into three types: Classical Arabic, Modern Standard Arabic, and Colloquial Arabic dialects (Hanane FROUD, 2012).

Classical Arabic is fully vowelized and it is the language of the holy Quran. Modern Standard Arabic is the official language throughout the Arab world. It is used in official documents, newspapers and magazines, in educational fields and for communication between Arabs of different nationalities. Colloquial Arabic dialects, on the other hand, are the languages spoken in the different Arab countries; the spoken forms of Arabic vary widely and each Arab country has its own dialect (Hanane FROUD, 2012).

Nowadays we see an incessant development of information technologies, these technologies produce large volumes of information, this information can exist in the form of the Arabic language, so it is the problem of finding information that Is interested in this large quantity, to remedy this problem a whole domain is born it is the Information retrieval which is interested in the development of the techniques and the tools which make it possible to find a Information in order to satisfy a need for information, called relevant information. These tools are called Information Retrieval Systems (IRS).

In an IRS, each document is represented by an intermediate representation called indexation (Erritali, 2016), and to do it there are many treatments to be realized among these, there is one that makes it possible to increase the effectiveness of the research, this is the stemming.

The morphology complexity of Arabic makes it particularly difficult to develop natural language processing applications for Arabic information retrieval. In Semitic languages like Arabic, most noun, adjective, and verb stems are derived from a few thousand roots by infixing, for example, creating words like مكتب (office), كتاب (book), كتب (books), كتب (he wrote), and كتبنا (we write), from the root كتب (Larkey, 2007).

Stemming is the process of removing any affixes from words, and reducing these words to their roots. For example, stemming the English word computing produces the root compute. This is the same root produced by the word computation.

Stemming for information retrieval (IR) is a computational process by which we remove potential suffixes and prefixes from a textual word to extract its basic form. The basic form produced does not have to be the actual word itself. Instead, the stem is said to be the least common denominator for the morphological variants (Carlberger, 2001).

This process should not be confused with the process of “morphological analysis” (or word “lemmatization”, as called by linguists) which aims at reducing morphological variants to a linguistically correct root morpheme from which they were derived.

The effect of term stemming on the performance effectiveness of information retrieval has been the subject of several investigations. Most notably of these investigations are those reported by the general indication coming out of most studies is that stemming can improve retrieval Performance, but by a small factor (Kraaij, 1996).

Complete Article List

Search this Journal:
Reset
Open Access Articles
Volume 17: 4 Issues (2019): Forthcoming, Available for Pre-Order
Volume 16: 4 Issues (2018)
Volume 15: 4 Issues (2017)
Volume 14: 4 Issues (2016)
Volume 13: 4 Issues (2015)
Volume 12: 4 Issues (2014)
Volume 11: 4 Issues (2013)
Volume 10: 4 Issues (2012)
Volume 9: 4 Issues (2011)
Volume 8: 4 Issues (2010)
Volume 7: 4 Issues (2009)
Volume 6: 4 Issues (2008)
Volume 5: 4 Issues (2007)
Volume 4: 4 Issues (2006)
Volume 3: 4 Issues (2005)
Volume 2: 4 Issues (2004)
Volume 1: 4 Issues (2003)
View Complete Journal Contents Listing