Autism and tablet computers in Turkey: Teaching picture sequencing skills via a web-based iPad application

https://doi.org/10.1016/j.ijcci.2014.04.002Get rights and content

Abstract

We conducted the first study on Turkish children with autism and tablet computers, with a web-based iPad application designed especially for them. We performed a pilot study on three Turkish boys of different ages with autism to observe their reactions to the tablet application and its effectiveness in teaching the sequencing skill, which is part of their educational curriculum. Our application had a testing session with no prompts or rewards and a teaching session with prompts, rewards, and demonstration of correct responses. First, our participants played the testing session to determine their baseline sequencing abilities. Next, they played the teaching session. Finally, they played the testing session again to see if they were now able to sequence the cards on their own. Through this application, the 11-year-old boy’s sequencing skills improved without external help, via only the prompts and reinforcements of the iPad application. The application was not enough to teach sequencing to the 4-year-old, who required external help, and it was too simple for the 15-year-old, who did not use any prompts and quickly became bored. Based on our findings, we discuss how to improve similar sequencing applications and offer suggestions for designing iPad applications for individuals with autism.

Introduction

Individuals with autism are impaired in social interaction and communication, and have repetitive, restricted, and stereotyped behavior patterns and interests  [1]. Early diagnosis and effective interventions through structured programs and educational methods result in significant progress in autism  [2]. Autism is universal. Individuals with autism show similar behavioral impairments and share similar learning characteristics all around the world. Information about new methods in the education of individuals with autism from one country can help those in another and is invaluable for international generalization of new technologies that are available in today’s world.

The Turkish government organized a project named Fatih Projesi to deliver tablet computers to public schools and purchased the tablets in November of 2013 (relevant press release can be found in http://fatihprojesi.meb.gov.tr/tr/haberincele.php?id=108). For students with autism who attend public schools to benefit from this project as well, there should be Turkish tablet applications designed specially for the needs of these individuals. Moreover, the United States Embassy in Ankara is conducting a project to develop a Turkish iPad application to be used as a voice output communication aid, i.e. voice what the children with autism who have problems with speaking wish to say. Yet, there is no information currently available in Turkey on how Turkish children with autism will interact with tablet applications designed for them. So, we have performed a pilot study to code and test a tablet application in Turkish for a skill that is in the educational curriculum of children with autism. The purpose of this study was to observe the reaction and interest of Turkish children with autism to educational iPad applications and see whether they will be able to learn from such an application.

This is the first study that has designed a Turkish application for children with autism and has tested it on Turkish children with autism. We carried out this pilot, data-based study since in Turkey, we currently do not have available educators for a period of time that is required for a single-subject experimental design and the time constraints on the curriculum of the participating children with autism do not allow for such an extensive design. Our purpose was to open the way for the coding of such applications in Turkey and provide information to help design controlled experimental studies in the future when the resources will be available.

This study is important in two ways: 1. to provide useful information about how Turkish children with autism react to the tablet application, if they are able to learn from it, and how the future applications should be developed, and 2. to lead the way for future studies that will allow cross-cultural comparison of the interaction of individuals with autism and tablet computers in Turkey and other countries.

Why iPads? There are several iPad applications that can be used by both individuals with and without autism, some flashcards and images that could be used within Applied Behavioral Analysis (ABA) therapy (http://www.autismspeaks.org/autism-apps), and several iPad applications that are intended for individuals with autism. Because the use of mobile technology by individuals with autism is still quite new, experts call for more in depth research in this field (Yee 2012) and the purpose of this paper is to show how Turkish children with autism interact with tablet computers and whether they are able to learn from a tablet application. We selected a skill that our participants needed to learn and aimed to teach it through the tablet computer technology. We have conducted our tests in Tohum School for Autism in Istanbul, the only school for individuals with autism in Turkey that applies ABA principles.

Previous experiments show that teaching concepts to children with autism through technological devices has many advantages over the traditional methods. Children with autism were more attentive, more motivated, and learned more vocabulary with educational software programs of a computer than with the behavioral program  [3]. Adding computerized games to the regular language therapy sessions produced the following results: children were more attentive, had more communicative initiatives and more eye contact, were more interactive, used more verbalizations, and made more action requests  [4]. Given these advantages of technology, the portable and engaging nature of iPads and their easy-to-use touchscreens that provide instant gratification for children  [5], we used iPads as our teaching medium.

Why sequencing? The reason we chose to design an application for the skill of sequencing is because sequencing events in a story is an important expressive language skill that is frequently absent in children with autism. It is one of the basic skills underlying communication, reading, and speaking, but there is no available evidence showing how the development of sequencing skills in children with autism can be assisted by an iPad application.

A lack of sequencing story events skills limits the use of language that promotes interactions, which results in a further decrease in opportunities to engage in a meaningful communication experience. For example, when an adult initiates a conversation with a child (e.g., tell me what you did today), the child may not respond if he/she does not know how to sequence in his mind and explain the events he/she was engaged in during the day in the order they happened.

Studies focusing on teaching sequencing story events skills to children with autism have reported favorable language development gains. Increase in sequencing story events skills has resulted in increases in social interactions, positive affect, and spontaneous speech  [6], as well as improvements in expressive language and social-communicative behaviors  [7]. Since children with autism are already at a communication disadvantage, these findings have led to the belief that sequencing story events should be a core component of any intervention program for young children with autism  [8].

Learning how to sequence story events helps children form connections between different stages of an event. For individuals with autism, it is hard to assemble the pieces from each stage and see the event as a whole. Sequencing game cards are used in the education of individuals with autism since they help children analyze pictures, put the events in the correct order, and predict the consequences of actions. If the individual does not understand the sequence of events, then he/she cannot narrate the event.

Sequencing activities also help children with autism acquire the prerequisite skills for reading and writing. In order to sort the images, children should be able to think of a story in pieces and distinguish the differences between its stages. The ability to notice the differences and understand what the images represent is one of the prerequisite skills for reading. In order to read, children should be able to differentiate the letters, i.e. realize that a is different from b. The skill of telling apart different photographs depicting distinct stages of an event prepares the student to differentiate letters and words later on, which are more abstract than pictures. Thus, the skills of reading and distinguishing different words that are written or spoken build on the skill of assigning meaning and distinguishing different stages of an event, which is acquired by sequencing story event photographs  [8].

Applied behavioral analysis (ABA). Applied Behavioral Analysis (ABA) is a scientific method that investigates environmental variables influencing socially important behaviors and uses those findings to implement interventions that will improve such behaviors  [9]. ABA principles posit that immediate consequences have the largest effects, and that reinforcement is key in behavioral change programs  [9]. These ideas are employed in ABA-based behavioral modification programs for individuals with autism.

The first phase of ABA starts with evaluating the individual’s behaviors and skills. In this stage, target behaviors need to be analyzed in detail. Skills are broken down into simple steps and every session consists of discrete trials that aim to teach the individual that specific skill. In the teaching session, first an instruction is presented to the individual in a clear and consistent tone. After waiting for a response for a while, if the correct response is not given, the individual is directed to the desired response by providing prompts. ABA, being a faultless teaching approach, aims to prevent individuals from giving erroneous answers, so prompting the individual to perform the desired behavior is an important part of this method. These prompts can be provided as visual cues, such as pointing out the correct answer or fading out the wrong options. Prompts need to be repeated until the desired behavior is performed and reinforcement should immediately follow the correct response to make the child understand the connection between the correct answer and the reward. These steps are repeated by fading the prompts and the reinforcers gradually as the child begins to perform better  [10]. In our application, we have used the prompt and prompt-fading procedures of ABA and we have designed the testing and teaching versions of the application according to the ABA steps described above.

Relevant iPad applications. Below is a list and short description of the existing iPad applications that are in some way related to the sequencing game we set out to design for Turkish children with autism. After analyzing what is missing in those applications with regard to the ABA principles, we designed our application specifically for the education of children with autism.

  • “Speech with Milo - Sequencing” intends to build sequencing and narrative skills for individuals who need special education. In the application, the player gets three different sequencing cards and needs to put them in the correct places. The application has a rich settings page that allows users to toggle on and off the success and unsuccessful sounds, background music, and written and spoken words. If the written and spoken words are turned on, the application gives a sentence that explains the event, such as “driving a car”. In the beginning, the application gives the prompt “put the cards in order” in both written and spoken form. The figures on the cards are cartoon characters and the background space of the cards is very busy, which makes it hard to focus on and distinguish the actions in the frames. If the user chooses to get a hint, then each card is given with a phrasal explanation written below, but since there is no sound option for the card explanations, this prompting strategy is not the best one to help the children who cannot read. If the child tries to drag the card to the wrong place, he is not allowed to place it there, which prevents him from making a mistake, as is recommended in ABA. Yet, there are no hints indicating the card’s correct location to help the child get to the correct response. When the card is placed in the correct place, there is a clapping sound, which can serve as positive reinforcement for the correct behavior. Lastly, the application does not record the data of the user, which would have been beneficial to keep track of the improvements.

  • “Tell a Story with Tommy: Community Sequences” is mainly focused on the development of story telling and sequencing skills for all children. It has seven story options: go home, the car, library, pool, school, visit, and grocery. At the beginning of the game, the story is told with images. There are four images with cartoon characters that appear on the screen one at a time, accompanied by a few sentences explaining each image, which are written on the screen and are also narrated. The sentences and the narration can be turned on and off. After that, four different cards are given to the child in a random order and the child is expected to sequence them. The child is not allowed to perform a wrong move and the narrator’s voice encourages the child to try again (“try again”, “why don’t you try again”, “okay, not quite”, “give it another shot”, “one more try”). The voice appreciates the child when she performs the correct move (“nice job, “right on”, “way to go”, “you’ve got it”). When the child is able to complete the sequencing of all four cards, there appears a dancing kid on the screen as a reward. There are verbal reinforcers in the game, which have a lot of variation, but there is a lack of visual prompts that guide the player towards the correct move. This application does not track the user’s performance either.

  • “Making Sequences” allows the selection of the game that the user wants to play (going to the beach, sending a letter, blowing up a balloon etc.) as well as the creation of a new personal one by adding the user’s own images and sounds. Since actual photography with simple background and real people is used in this application, frames are suitable for children with autism. There is a “watch” option to listen to the stories before starting the “place in order” section. The main consideration is that, during the game, the user needs to slide the cards over each other in order to put them into the correct places. Numbers are placed above the photographs as a cue for the sequencing order. As a result, a player who knows the order of the numbers can sequence the actions correctly without paying attention to what is depicted in the cards. In addition, until the “Done” button is hit, there are no clues as to whether the performed sequence is right or not. These features can be confusing for children with autism. During the game, when the child selects the “listen to story” option, the cards are narrated in the order that the child places them in, not necessarily in the correct order, which can cause the children with autism to learn an incorrect sequence as a correct one.

  • “iSequences” is especially designed for children with autism or children with other special educational needs. It aims to teach autonomous habits (such as brushing teeth, washing hands or getting dressed), leisure activities (such as going to the movies or doing sports), daily activities (such as doing laundry or going to school) and the emotions following the sequenced events (such as happiness or surprise). There are a hundred different sequencing games and each game includes three or four cards. The application has many setting options to customize the game, including five different languages. Cartoon characters are used in this application. When the child completes the sequencing successfully, the game rewards her with an applause and a reward picture. There is a different reward picture at the end of each sequence, a variety that engages the player. This application is well designed in terms of the diversity of images and activities, which is likely to prevent boredom and enable interested interaction with the application for a long time. There are no prompts to guide the child to the right action. The application waits until the last card is placed to indicate whether the sequencing is correct or not, and lets the child place the cards in incorrect locations, so there is a lack of immediate feedback to the user.

  • “Sequencing Tasks - Life Skills” is another sequencing application that uses simple, actual photographs, which helps the child to focus on the actions in the frames. Before the sequencing, the whole activity is described with a video. Then, each frame that is used in the sequencing is narrated individually. The player is not allowed to put the card in an incorrect place but there is no cue to lead him to the card’s correct location. When the card is placed in the correct location, the application rewards the player with a ring sound and by saying “wonderful”, “perfect”, “excellent”, and “that’s right”. When the player tries to put a card to an incorrect location, the application says “this doesn’t go here, try again” and moves the card back. When each card is touched, the activity depicted in it is narrated, which may be a helpful cue. There is a “Statistics” option, which shows the activity history that includes the date and time the application was played, name of the activity, and the total number of incorrect placements during each session. This is useful to track the improvement of the player.

Evaluation of the existing sequencing applications. After analyzing the properties of the existing applications listed above, we decided on the following guidelines for our application:

  • It seems that most of these applications and other educational applications for children with special needs do not have a real-time cue system that teaches the child how to play the game. Rather, the cue is given beforehand and a correct move is expected from the child or if no cue is given, the child is not allowed to make an incorrect move. Prompting is very important for learning and it is advised that prompting should be increased or decreased depending on the child’s performance  [11]. Thus, we decided to implement a real-time cue system in our game, which adjusts the level of the cues according to the performance of the child.

  • Some applications use cartoon characters and some use real photographs. A decade of experience with Turkish children with autism and a total of 23 years of experience with individuals with autism show that they learn more easily and better from real characters compared to cartoon characters, which seem to be too complicated for the children to make connections with real life. This is also remarked by autism educators on online sources, such as The Autism Helper, who say that cartoon depictions can sometimes be too abstract for children with autism and advise the use of real photographs (http://theautismhelper.com/real-photo-pecs). This is the reason behind our choice to use pictures of human models in our application. We used two different characters instead of the same person performing all the actions in order to better enable generalization across different people. Yet, we are aware that further tests should be performed on the comparative effectiveness of teaching from real vs. cartoon characters, which will be the focus of a future study on Turkish children with autism.

  • We did not put any numbers or letters on the pictures to prevent the children from paying attention solely to those cues and ignoring the event depicted in the photographs that are supposed to be sequenced.

  • Moreover, all of the applications described above allow the player to start sequencing from any card they want. Based on long years of experience with children with autism in Tohum School, the educators and the principal recommended that our application should only allow the player to start sequencing from the first card. This is important because when typically developing individuals are engaged in a dialog in real life, they start explaining an event from its beginning, not from its middle or end. This makes the story easier to understand and it helps the storyteller remember the continuation of the story, since the beginning of a story serves as a cue to what comes next. We are designing this application to improve real life communication skills of individuals with autism and teaching them to begin with the first event in the sequence is an important step in helping them acquire the skill of talking about their daily life events to others.

  • In addition, none of these games except one keeps track of the player’s performance. In schools for children with autism, for every task in the curriculum, the instructors collect data prior to, during, and after learning to see whether the student was able to learn successfully. In order to see the development of children’s abilities, we implemented a performance tracking and data recording system to collect the results of this sequencing game. Different from the data collection in “Sequencing Tasks—Life Skills” application, our application tracks the performance for each card and the overall performance.

  • We have decided to code our application as an educational game, defined as having a balance of educational and entertainment value (Prensky, 2001, as cited in  [12]). Educational video games comprise rewards and interactive cues that offer feedback and induce learning  [13], qualities we have included in our application. We decided that our application should have a testing session to measure the skill level prior to and after learning, and a teaching session to teach the sequencing skill. The testing and teaching sessions are parts of our application, which is an educational game.

Choosing the platform of HTML5. We could have coded our application for IOS, an operating system that runs on Apple products, or for Android, a mobile operating system that runs on devices other than Apple products. Or, we could have chosen to code a Web-Based application, HTML5, which is described in detail by Vaughan-Nichols  [14]. After weighing the advantages and the disadvantages of these options, we decided to use HTML5. The most important reason for our choice was the flexibility that allowed the application to be available on all platforms, such as PCs, different brands of mobile phones, and tablets. In Table 1, the advantages and disadvantages of HTML5 and the other platforms are listed.

Section snippets

Study design

This was a pilot study to observe the suitability of an Applied Behavioral Analysis based iPad application for different aged Turkish children with autism. We also aimed to see whether the prompts and cues of the application would be enough to teach the skill of sequencing without any external prompting from the teachers. We used an ABA design over three participants; A was our baseline condition where we used the testing session of the iPad application that included no prompts, B was our

Results

The performance of each child is depicted in the following graphic illustrations. In the testing session, if the child is able to place a card in its correct location, he gets 5 points, if he is not able to do so, he gets 1 point. In the teaching session, depending on how many cues the child requires before being able to place the card in its correct location, he gets the following points: 5 points for success without requiring any cues, 4 points for success after 1 cue, 3 points for success

Discussion

Although this is a quasi-experimental study, it is the first ever study on the interaction of Turkish children with autism and tablet computers. We believe that the observations and the following suggestions offer valuable guidance for future applications developed for Turkish children with autism, including those by the US Embassy, and for future experimental studies that will investigate the effectiveness of such applications in teaching basic skills to Turkish children with autism. The

Acknowledgments

We would like to thank the following educators in Private Tohum Autism Foundation Special Needs School and Tohum Autism Foundation in Istanbul for their invaluable support and help during our experiment: Zeynep Kad, Nurdan Durmaz, Deniz Yılmaz, and Sandra Pasensya.

References (19)

  • S. Venkatesh et al.

    Pervasive multimedia for autism intervention

    Pervasive and Mobile Computing

    (2012)
  • Diagnostic and Statistical Manual of Mental Disorders

    (2000)
  • R.L. Simpson

    ABA and students with autism spectrum disorders: issues and considerations for effective practice

    Focus Autism Other Dev. Disabil.

    (2001)
  • M. Moore et al.

    Brief report: vocabulary acquisition for children with autism: teacher or computer instruction

    J. Autism Dev. Disord.

    (2000)
  • F.D. Fernandes et al.

    Computerized resources in language therapy with children of the autistic spectrum

    Pro. Fono.

    (2010)
  • N. Shah

    Special education pupils find learning tool in iPad applications

    Education Week

    (2011)
  • C. Whalen et al.

    Joint attention training for children with autism using behavior modification procedures

    J. Child Psychol. Psychiatry

    (2003)
  • E.A. Jones et al.

    Multiple effects of joint attention intervention

    Behav Modif.

    (2006)
  • L.E. McClannahan et al.

    Teaching Conversation to Children with Autism: Scripts and Script Fading

    (2005)
There are more references available in the full text version of this article.

Cited by (27)

  • Tensions in designing tablet-based musical instruments for students with disabilities

    2022, International Journal of Child-Computer Interaction
    Citation Excerpt :

    In the field of Child–Computer Interaction, researchers have examined the development of music-based software (Nouwen et al., 2016) and interactive reflexive musical systems (Addessi, Anelli, Benghi, & Friberg, 2017) while more work is needed on non-traditional musical interfaces such as iPads. Additionally, tablet computers have been studied as interfaces for interactions with individuals with Autism Spectrum Disorder extensively (Doenyas, Şimdi, Özcan, Çataltepe, & Birkan, 2014; Hourcade, Bullock-Rest, & Hansen, 2012). With this increasing body of research that incorporates children with disabilities into technology design (Benton & Johnson, 2015) and the promise of tablet computers in these spaces, the use of individually designed musical interfaces has the potential to reduce barriers to learning and creativity for children of all ability levels.

  • Dynamic difficulty adjustment technique-based mobile vocabulary learning game for children with autism spectrum disorder

    2022, Entertainment Computing
    Citation Excerpt :

    Therefore, a preference for both visual animations and audio stimuli of children with ASD emerges with the reports provided by individuals with ASD. These types of reinforcements have been previously implemented in educational tablet games for ASD [20] and follows the principles of Applied Behavior Analysis (ABA), which is the most commonly used educational principle for teaching individuals with ASD in special education centers and schools. ABA refers to using data about behaviors to develop hypotheses about why a certain behavior happens in a particular context and creating interventions to change the identified behaviors.

  • Wearables and mobile technologies in Autism Spectrum Disorder interventions: A systematic literature review

    2019, Research in Autism Spectrum Disorders
    Citation Excerpt :

    The rest of the articles were published in technology related conferences and journals. Computers & Education (Fernández-López, Rodríguez-Fórtiz, Rodríguez-Almendros, & Martínez-Segura, 2013; Mintz, Branch, March, & Lerman, 2012; Mintz, 2013)⁠, International Journal of Child-Computer Interaction (Doenyas, Şimdi, Özcan, Çataltepe, & Birkan, 2014; Fletcher-Watson, Pain, Hammond, Humphry, & McConachie, 2016, 2016b; Holt & Yuill, 2017)⁠⁠, and Journal of Medical Systems (De Leo, Gonzales, Battagiri, & Leroy, 2011; Quezada et al., 2017)⁠ appeared three times each, while Research in Autism Spectrum Disorders (Neely, Rispoli, Camargo, Davis, & Boles, 2013; Smith, Spooner, & Wood, 2013) ⁠and the Journal of Autism and Developmental Disorders (Gentry, Kriner, Sima, McDonough, & Wehman, 2015)⁠ appeared once. Finally, Procedia Computer Science (Aziz, Abdullah, Adnan, & Mazalan, 2014; da Silva, Gonçalves, Guerreiro, & Silva, 2012; Silva, Gonçalves, & Silva, 2014; Tsiopela & Jimoyiannis, 2014)⁠ was the most frequent one, with four articles, followed by Research in Developmental Disabilities (Burke, Andersen, Bowen, Howard, & Allen, 2010; Cihak, Kessler, & Alberto, 2007; Vélez-Coto et al., 2017) with three articles.

  • EmotionBlock: A Tangible Toolkit for Social-emotional Learning Through Storytelling

    2023, Proceedings of IDC 2023 - 22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood
View all citing articles on Scopus
View full text