Intelligent Multi-Modal Recognition Interface Using Voice-XML and Embedded KSSL Recognizer

Abstract

A desktop PC and wire communications net-based traditional studies on pattern recognition and multimodal interaction have some restrictions (e.g. limitation of motion, conditionality in space and so on) and general problems according to using of the vision technologies for recognition and representation of the haptic-gesture information. In this paper, we propose and implement Multi-Modal Recognition Interface (hereinafter, MMRI) integrating speech using Voice-XML and gesture based on wireless networks, it have purposes that recognizes and represents the Korean Standard Sign Language (hereinafter, KSSL) which is a dialog system and interactive elements in the Korean deaf communities, and the need to dialogue with deaf person in their own language, sign language, is well recognized and is widely accepted as being a positive influence on communication. The advantages of our approach are as follows: 1) it improves efficiency of the MMRI input module according to the technology of wireless communication, 2) it shows higher recognition performance than uni-modal recognition system 3) it recognizes and represents continuous sign language of users with flexibility in real time and offer to user a wider range of personalized and differentiated information using the MMRI more effectively. Experimental results, the MMRI deduces an average recognition rate of 96.23% for significant, dynamic and continuous the KSSL and speech of various users.