SignMT: An alternative language learning tool

doi:10.1016/j.compedu.2009.12.009

Computers & Education

Volume 55, Issue 1, August 2010, Pages 118-130

https://doi.org/10.1016/j.compedu.2009.12.009 Get rights and content

Abstract

Learning a second language is very difficult, especially, for the disabled; the disability may be a barrier to learn and to utilize information written in text form. We present the SignMT, Thai sign to Thai machine translation system, which is able to translate from Thai sign language into Thai text. In the translation process, SignMT takes into account the differences between Thai and Thai sign language in terms of both syntax and semantic to ensure the accuracy of translation. SignMT was designed to be not only an automatic interpreter but also a language learning tool. It provides meaning of each word in both text and image forms which is easy to understand by the deaf. The grammar information and the order of the sentence are presented in order to help the deaf in learning Thai, their second language. With SignMT, deaf students are less dependent on a teacher, have more freedom to experiment with their own language, and improve their knowledge and learning skill.

In our experiment, SignMT was implemented to translate sentences/phrases which were collected from different sources including textbooks, cartoons, bedtime story, and newspapers. SignMT was tested and evaluated in terms of the translation accuracy and user satisfaction. The evaluation results show that the translation accuracy is acceptable, and it satisfies the users’ needs.

Section snippets

Why’s TSL–Thai MT needed?

Deafness or hearing impairment affects not only a child who is deaf or has a hearing loss, but also the child’s family, friends, and teachers. Although deaf, hard of hearing and hearing signers can fully communicate among themselves by sign language, there is a big communication barrier between signers and hearing people without signing skills. The biggest problem with sign language is that the vast majority of non-deaf people do not understand it. Moreover, sign language does have its

MT and CALL

Rost (2002) stated that the goals of language teaching profession are (1) giving students real opportunities to learn and helping them learn more effectively; (2) increasing the enjoyment of language learning; (3) improving students’ ability to become better language learners; and (4) making the teaching more enjoyable and rewarding. To achieve such goals, using computer technologies is one solution.

Some linguistic issues of TSL

Sign language is a three-dimensional visual language that uses combinations of signs (handshapes), gesture, facial expression, and in literate communities, some fingerspelling to construct and convey meaning (William, 1960). Gestures can be characterized by both manual and non-manual parameters. Manual parameters include hand-shapes, hand-orientation, position, and motion while non-manual parameters include posture of the upper torso, head-orientation, facial expressions, lip movement and

SignMT-system architecture

We designed the architecture of SignMT as illustrated in Fig. 6. SignMT performs the translation in four steps: word transformation (WT), word constraint (WC), word addition (WA), and word ordering (WO). WT was designed to allow the users to correctly input to SignMT. WC removes any word required no translation however the original meaning is preserved still. WA adds any necessary word required into the translation process and WO arranges all the translated words into grammatical order. Each

Translation samples

SignMT was designed to be able to translate any type of sentence, e.g., affirmative, negative, interrogative and imperative. In our preliminary experiment, the developed sign picture dictionary contains only 250 entries, and expanded to 500 and 1000 entries, respectively. There has been no significantly increase in processing time when running SignMT with the larger dictionary. The sample sentences were collected from different sources including textbooks, cartoons, bedtime stories and

SignMT: an MT-CALL

Since SignMT was aimed at being a language learning tool not only for deaf people to learn Thai and TSL (in primary school), but also for hearing people who wish to learn TSL, the interface then must be designed for their convenience, and is able to display clear information. The interface of SignMT is comprised of two main windows: input and output windows.

Input window was designed to facilitate the TSL input process. For example, to input the TSL sentence “[

–chair] [

–aunt] [

–sit] (An aunt

Evaluation

The evaluation of SignMT was divided into two parts: translation accuracy and user satisfaction. The translation accuracy was examined in terms of intelligibility and fidelity by linguistic teachers (both hearing and deaf), deaf students, and TSL interpreters. All testers are from Ratchasuda College, a college for persons with disabilities in Bangkok, Thailand. The goal of translation accuracy measurement is to determine whether the system can generate a correct and reliable translation or not.

Conclusion

We present SignMT, an alternative language learning tool for deaf, which was designed to translate from Thai sign language into Thai text. Its translation process is comprised of four steps: word transformation (WT), word constraint (WC), word addition (WA), and word ordering (WO). The distinction between Thai and Thai sign language in both syntax and semantic are concerned in each processing step. The translation begins with WT to transform the sign picture into matched text. WC removes

References (21)

S. Dangsaart et al.
Intelligent Thai text – Thai sign translation for language learning
Computers & Education
(2008)
R. Atkinson
Computerized instruction and the learning process
American Psychologist
(1968)
Dailynews. (2002). Article from Dailynews on August 8, 2002. <http://www.dailynews.co.th> Retrieved on September...
F. Ehsani et al.
Speech technology in computer-aided language learning – Strengths and limitations of a new CALL paradigm
Language Learning and Technology
(1998)
Graham, D. (2002). CALL in the good practice guide at the website of the subject centre for languages, linguistics and...
Lee, J & Kunii, T. L. (1992). Visual translation – From native language to sign language. In Proc. of 1992 IEEE...
Ministry of Education. (1999). Promulgation for acceptance of Thai sign language as national language for deaf, August...
R. Mitchell et al.
Second language learning theories
(1998)
Molnar, A. (1997). Computers in education – A brief history. Technical Horizons in Education....
Naruedomkul, K., & Cercone, N. (1999). The role for word association numbers in machine translation. In Proc. of the...

There are more references available in the full text version of this article.

Cited by (0)

View full text