Designing High Accuracy Statistical Machine Translation for Sign Language Using Parallel Corpus: Case Study English and American Sign Language

Introduction

We can easily exchange our ideas, collaborate, and build strategies together, if we could speak all languages. Alternatively, we should have a communication tool that allows such innovations. Communication is the essence of human interaction. One of the most effective approaches of communication between human beings is through languages, which were born from human interaction. Communication and language exhibit a relation of interdependence. However, this natural method of communication can establish an obstacle in the cases where languages are postponed. Nobody can ignore the obvious barrier of communication between languages of different modalities, worth knowing the vocal and sign languages.

In fact, Sign Languages (SLs), used by the deaf communities, which do not hear or hear badly, are visual-gesture languages. The message is transmitted by gestures and movements and received by the visual channel. The vocal languages, as for them, exhibit an audio-phonatory character. Their message is emitted through a phonatory canal and received thanks to the auditory canal. Canals used by the SLs are thus different from those used typically, and the SLs distinguish them from other languages. The linguistic knowledge acquired after several years of research and studies on diverse vocal languages are with difficulty transposable in the SL. Therefore, SL becomes a novel object of linguistic study, and it started from the 1960s. This research field is thus more recent, and it explains that the knowledge of the linguistic research on SL is constantly evolving.

This paper concerns NLP and is interested, in particular, in the case of ASL. Exotic by their implementation of gestures/movements and not sounds, these leave the traditional phonological frame and do not have a standard phonetic script similar to the vocal languages to transcribe their realizations. Moreover, exotic by their multi-linearity, they utilize their common canal to convey several elements of information simultaneously, whereas the voice device allows the production of only a sound at a time. The language models raise natural questions that are different from those suggested by the vocal languages.

Transcription is the operation that substitutes a grapheme or a group of graphemes of a writing system for every phoneme or for every sound. It thus depends on the target language, a unique phoneme that can correspond to various graphemes following the considered language. In short, it is the writing of words or pronounced sentences in a given system. The transcription also aims at being without loss, so that it should be ideally possible to reconstitute the original pronunciation from this one by knowing the rules of transcription.

From all cited perspectives, this paper concerns the transcription of SL and the implementation of a Statistical Machine Translation and concentrates more particularly on the machine translation of a text in English to ASL and conversely. The study produced articulates around four axes:

• •

  Presenting an overview about SL and a state of the art about Sign Language Processing;

• •

  Proposing a novel transcription system to write SL called XML-Gloss Annotation System;

• •

  Building an artificial parallel corpus between English and ASL using XML-Gloss Annotation System;

• •

  Implementing a Statistical Machine Translation between English and ASL based on the Artificial Parallel Corpus.