Developing BacaBicara: An Indonesian Lipreading System as an Independent Communication Learning for the Deaf and Hard-of-Hearing

—Deaf and hard-of-hearing people have limitations in communication, especially on aspects of language, intelligence, and social adjustment. To communicate, deaf people use sign language or lipreading. For normal people, it is very difficult to use sign language. They have to memorize many hand signs. Therefore, lipreading is a necessary for communication between normal and deaf people. In Indonesia, there is still few education media for deaf people to learn lipreading. To overcome this challenge, we develop a lipreading educational media to help deaf and hard-of-hearing to learn Bahasa Indonesia, called BacaBicara. User-Centered Design (UCD) is implemented to design the application and to analyze the constraints and conceptual models for the needs of users. This conceptual model uses the picture, lipreading video, text, and sign language to help the users understand the contents. The High fidelity prototype was implemented for evaluating usability testing. Based on the evaluation of the application, the results show that the prototype matches the usability goals and the user experience.


Introduction
Persons with disabilities are the largest minority group in the world, one of which is hearing and speech disability.The person who has the imperfection is called the deaf people.There are about two millions deaf people in Indonesia [1].In the city of Semarang, the number of people with hearing impairment is increasing.The number of deaf people in the city of Semarang in 2012 was 527 children, increasing rapidly from previous years.
One of the consequences of hearing and speech limitations, people with hearing impairment have difficulties in communication, especially on aspects of language, intelligence and social adjustment [2].Although there are many technologies to help people with speech impairments such as hearing aids and cochlear implants, these technologies will not work in a noisy environment.Futhermore, not all normal people understand the sign languange so that the message is not conveyed properly and will lead to misunderstandings [3].Therefore, an interactive media is needed to bridge the communication between normal and deaf people.Lipreading is one method that can be used to train deaf people to communicate with normal people.This technique uses the lip shape reading ability to analyze the word being spoken by the interlocutor [2] [4] [5].
Indonesia is a country that has implemented various technologies into the education sector.With the advancement of this technology, it is encouraging us to develop the application that can be used to facilitate the communication learning by using lipreading techniques for both deaf and normal people.Based on these issues, we create BacaBicara, which provides language learning through lip reader using interactive website-based technology that adopts e-learning systems.BacaBicara can be used for all ages, both normal humans and deaf people.This application provides lipreading videos as learning materials, texts, images (visualization) and exercises, so people with hearing impairments can use this application as an independent teaching material that can be accessed easily anywhere and anytime and easily understood by all people.This application is equipped with a teacher's page, as a means for a teacher to know the development of his students.
The design method used in designing the user interface of BacaBicara is User-Centered Design (UCD).The development of the application is based on the user needs [6].This method serves to build a good interaction and interface so that it meets the needs of the hearing impaired blind.The paradigm in this method places the needs, desires, and limits of the user as the main focus in each stage of the design.The paradigm is applied to various well-known methods and techniques for analyzing, designing, and evaluating [7].

Literature Review
Deaf people can be grouped into two categories: prelingual and postlingual.Prellingual child included in the category of heavy deaf, while postlingual group has hearing loss from birth.Because this limitedness, deaf people tends to have speech disorders.The development of the language learning for the deaf people may not only rely on the sense of hearing, but they need to optimize the visual abilities [8].Deaf people rely heavily on visual media to understand a language such as cues, text, images, videos, and animation [9].In the process of speaking and language learning, people who are deaf need a long time and intensive coaching.There are 3 main methods of deaf people learning languages: sign language, hearing from cochlea implant, and lipreading.Nearly 80% of non-verbal communication is the media most often used by people with speech deafness to learn languages [8].Several studies [8] [9] [10] [11] [12] have applied the use of lipreading in the form of an interactive application to increase interest in speech and deaf speech learning.
Lipreading is often analogous to "third ear", because we listen to someone through lip movements, speech patterns, and expressions of the other person (lipreading.org).
Lipreading requires the development of certain skills that make the process easier and more effective [2].When we perform the language learning method using lipreading, the appearance of images or videos is the main key in understanding the letters and words that the narrator says [13].Fig. 1 is a description of the pattern of lip movements that are grouped according to pronunciation.So, we propose BacaBicara web based application.The application is complemented by non-verbal communication; lipreading videos, narrator faces, and animations of each material.The application is presented in the educational games so as to provide some of the latest innovations to attract students' interest in learning by combining fun ideas with some learning material [14].
Based on [6] [15], the interfaces design of educational software must pay attention to the way students learn and must also provide good use so as to provide synergy between student interaction and software can run naturally.BacaBicara application uses an elearning system so that it can train users' skills and gain knowledge about new domains.Not only the elearning system, but also the development of learning through educational games is also implemented in the system.This research was conducted to investigate whether website interaction is easy to understand or not.Refer to Fig. 1, the movement of lip are influenced by Indonesian phoneme which is described by the BacaBicara narator.The process of text recognition consists of preprocessing familiar words, identification of words (word boundaries, lexicons), identification of phrases and accent of positions.the example of pronunciation Indonesia phoneme will be shown in Table 1.In this study, we use the UCD interface development method.The users are given a BacaBicara application prototype, then they are asked to do some interaction in the form of classroom management for mentoring teachers, lipreading exercises for people with speech and normal people.

Methodology
This study aims to develop a web based application that meets the usability goal and user experience for BacaBicara application as an independent communication learning to assist the deaf people.We use UCD approach to develop BacaBicara because people with hearing impairments have special characteristics that is different from the normal people.This approach is also effectively used for developing elearning systems to find out what the users need [6].UCD is a development method in building user interfaces that are designed according to user information needs.In UCD, users participate in the development process [16].

Specify contect of use
At this stage, we conduct interviews and observations to the teachers and deaf people in inclusive schools to explore the needs and problems faced by them during the process of speaking and language learning.There are two types of personality catheteristics in this study, the first is the deaf person who often communicates with normal people (without using sign language).The second is deaf people who rarely communicate with normal people and use sign language media.We focus on the first personality characteristic to implement the system.Periodic training to improve the ability of users in language Users are accustomed to using Lipreading as a language learning aid

Identify User Goals and User Task
This research proposes a learning media that provides solutions for users, especially those with deaf speech to learn language independently.The goal of the user is to make users accustomed to using this application to train their speaking skills and language, especially Indonesian.Some of the tasks that must be performed to achieve this goal include: the user must have an interest in learning language using the method of lipreading learning, users can identify differences in syllabel, new words and grammar, and the last is the user can understand what the narrator is saying so that it can hone their ability to learn lipreading.The objectives and tasks of the users are presented in Table 2.

Produce Design Solution
Based on several requirements analysis that has been carried out in the previous stage, we design a user interface that contains some features.The design stages are divided into several phases.
Information architecture: Information architecture is the structure of software used by users to find information based on user position.BacaBicara have 7 features, which are: • Welcome page: This feature is the start page of BacaBicara.Display of this interface is made to provide such information as a description of the application.• Register page: This feature enables user to register their BacaBicara account.
Users can register as students or teachers.If they register as a teacher, the system will automatically give the class code that the teacher will teach.There are five parts that the users have to fill, which are: usename, email, password, a choice to register as teacher or student, and teacher code.• Login page: After registering, the user will enter BacaBicara by entering the username and password.
•  Implementation: In this paper, a high-fidelity prototype was built using an approach.This prototype represents software and has interaction between users and prototypes [7].This prototype was developed using web technology: HTML5, CSS, Javascript and PHP.High-fidelity prototypes are implemented with the UCD model.Design principles are used to create user-friendly applications.Visual design is made taking into ac-count the visual hierarchy and its components designed to achieve usefulness and user experience goals.The design of the BacaBicara system refers to the 8 golden rules coined by Ben Shneiderman.The Golder rules were first implemented in 1986, including [19]: • Strive for consistency: Consistency in BacaBicara lies in color.We use white and blue for the color in the system.• Cater to universal usability: Users in BacaBicara include students, teachers, and administrators.Each user has their own main page according to their tasks and needs.
• Offer informative feedback: A visual presentation of an interesting object provides an environment that is really good for displaying changes explicitly.The feedback that is implemented is in the login process, if the username and password are not in the database, the system will issue a notification of error • Design dialogs to yield closure: Informative feedback on the completion of a group of actions gives users satisfaction of achievement, a sense of relief, a signal to stop contingency plans from their minds, and indicators to prepare for the next action.For example, the system gives appreciation if the students answer questions correctly and give notice if the students answer the questions incorrectly.• Prevent errors: As much as possible, design a system so that users cannot make serious errors, for example, marking the right menu items and not allowing alphabetical characters in numeric field entries.For example, giving placeholders (writing in the input) and information under the input.• Permit easy reversal of actions: As much as possible, the action must be reversible.This feature reduces anxiety, because users know that mistakes can be canceled, and encourage exploration of foreign choices.For example, in the data deletion process there is a confirmation dialog.Users can choose to cancel so that the data is not deleted.• Support internal locus control: Users want what is in the interface and the interface responds to user actions.For example, if the user presses the register button, the register process will appear, not another process.• Reduce short-term memory load: Humans have limited capacity for processing information in short-term memory.Therefore, designers avoid interfaces where users must remember information from one screen and then use that information on another screen.Examples for student list pages do not display student numbers, but student names and photos because student numbers are difficult to memorize.
The pop quiz metaphor is applied to learning media.Users will be given a short question with multiple-choice answers, this aims to develop new vocabulary or new sentences.This quiz has a maximum score of 100 points.Every user who can answer the question correctly, there will be an additional score for each question.Conversely, if each trial answer is wrong, 20 points will be deducted.At the end of each topic, there is a review page to repeat the content.The review page is designed similar to "looking for some items in the room" game.

4
Results and Evaluation BacaBicara is an independent learning media that can be used by users (deaf speakers and normal people) to learn the language and talk using the lipreading method.This application utilizes the visual user ability to pay attention to the lips of the other person so that we can understand the speaker's intentions and conduct brief communication.This application uses Indonesian as a learning material so that users can implement their abilities in their daily lives.Pronounce learning is one of the most important components to make users accustomed to seeing Indonesian pronunciation from lip reading.This learning uses videos to help visualize lipreading.Fig. 4 displays an interesting view of the BacaBicara application.
The BacaBicara presents the material in the form of a simple example, examples of placement of letters that have been divided by categories referring to the lesson page in the chapter 3 and the combination of words based on previous syllables.This material is used as a teaching material for students before starting the lipreadig training so that they can adapt to the narrator in the Bacabicara application.
BacaBicara shows the exercises that are packaged through pop quizzes as educaional media.This quiz shows lipreading videos and students must read the video footage in the multiple choice provided by the system.Students can also see a review of the exercises they have done through a review page.Fig. 5 shows the type of pop quiz provided by the system.Users will be given several new vocabulary words or sentences in the video, then the user is required to answer each of these videos by selecting several answer choices in the quiz.BacaBicara will repeat the questions that cannot be answered by the user so that the user can better understand and learn from the error.In addition, BacaBicara always provides notifications sent by the user's cellphone to remind the training schedule that was previously agreed upon between the user and BacaBicara.This innovation is used so that users consistently practice lipreading to improve lip reading skills and also to improve scores.Teachers as instructors can monitor students' abilities through the teacher's dashboard.Teachers can also provide special assistance to students who have difficulty learning this lipreading.

Interview
Testing the research using five respondents was enough if the results show more than 75% [16].Interviews and observations of this study were conducted on 5 students and 5 teachers.Interview begins with user identity questions including name, age, gender, contact, and hobbies.Followed by questions about how to communicate both to the deaf and normal people.Examples of questions include "How is your hearing condition?(Range 1 to 7, from no interference to severe hearing loss)", "How do you communicate with deaf people?", "How do you communicate with normal people?", and "What are the obstacles when communicating with deaf people and normal people?" The interview results show that the deaf in the Inclusive School communicate using sign language and writing.When dealing with deaf people, they only use sign language.When dealing with normal people, they use sign language coupled with lip movements that reflect the sentence or by writing.While the communication of normal people with hearing impairments uses direct sign language.For normal people who do not understand sign language, they can use lip movements or writing.

Exploration
After completing the interview, the user perform some explorations on the system for a few minutes.These explorations aim to find out how understanding users understand the system without instructions.The results show that the user is able to explore the system without difficulty.

Task completion
Users get tasks that must be completed from the system.Task completion aims to evaluate whether the user understands how the system works.There are 5 tasks that must be done, including: user register, user login, view profile, change profile, and complete one material.The result show that all users can assign the given tasks without difficulty.Users also understand the flow of the system.

Question and answer
Question and answer aims to get user understanding about the system.Answers are written on paper.These questions include: • What do you think about this system?• What is your expectation of this system?• What experience did you get when using this system?• Mention 3 things you like about this system!• Mention 3 things you don't like about this system!• What do you think about the whole system?(Interface, convenience, interaction, etc.) • If you can contribute to the development.What do you want to add and change?
The result of question and answer shows that the application is simple, easy to use and easy to understand.The expectations of the user have been met from the application and the user feels comfortable when using the application.

Questionnaire
Questionnaire aims to get quantitative data from users.Questionnaire uses a rating of 1 to 5. A rating of 1 represents poor and a rating of 5 represents excelent.Questions are based on the usability goal and the user experience (UX).
Based on the questionare, the results show that 100% of users agree that the prototype is useful, easy to learn, easy to remember, help, and interesting, while 90% of users agree that prototype is effective, and 70% of users agree that the prototype has the good UX.

Fig. 1 .
Fig. 1.The pattern of Indonesian lip movement Lesson page: This feature allows novice users to try the offered lesson before entering the main question page.Lesson pages are divided into 11 lessons: high vocal (i, u), moderate vocal (e pepet, e taling, o), low vocal (a), bilabial consonant (p, b, m), laiodental sonan (f), dental consonants (t, d), alveolar consonants (s.l, r, n), plato-alveolar consonants (c), palatal consonants (y), velar (k) consonants, and glottal consonants (h) [17] [18].• Guideline page: This feature allows users to get detailed user guide information in accessing the system.The information is provided in the text and video tutorials that are easily understood by the user.• Teacher page: This feature allows the teacher to maintain the progress of his students.The teacher can see the list of students who enter the class through the student page list and see the progress of each student through a statistics page.• Student page: This feature allows students to choose the topic of lipreading exercises which are divided into 15 topics.Each topic has 3 levels, namely easy, medium, and difficult.After selecting the topic presented by the system, the user will enter the question page.Question page is presented in the interactive quiz that enhances user learning interest.Workflow: Workflow is used to find out what processes users can do in software.Workflows are designed to represent information architecture, interface design, visual design, and navigation design.Each design is analyzed with the right design elements.Fig.2shows the workflow diagram of the BacaBicara application.From the welcome page, users can register as teacher or student.The login page is used for users who have already done in the previous registration and page guidelines.

Table 2 .
The purpose and tasks of the users