A Linguistically Sortable Bengali Coding System and Its Application in Spell Checking: A Case Study of Multilingual Applications

Abstract

By developing an inter-scheme text conversion utility, we have established in (Murshed et al., 1998) that use of non-lossy transformation instead of lossy transformation for sorting Bengali texts in linguistic order has some extra benefit. In this paper we discuss another very important application of non-lossy transformation by developing an efficient spell checking application for Bengali texts based on the internal coding scheme with non-lossy transformation. As usual, the handling of compound letters remains the key area where a Bengali text speller differs from its counterparts in other languages. Here we establish that using of the internal coding scheme in designing the dictionary and developing suggestion generating search engine not only provides a spell checking solution which is independent of any specific primary coding scheme but also assists in designing layered solution for efficient modularization and maintenance of coding. This chapter is organized as follows. In the next section we present the basic properties of Bengali script. For the sake of completeness, some results and algorithms on sorting Bengali texts in linguistic order, developed in (Murshed et al., 1998), are given in the third section. In the fourth section, we discuss various issues of developing an efficient primary coding scheme independent spell checking application based on our solution to linguistically sorting Bengali texts. The final section concludes the paper.