The Relationship between Narrative Proficiency and Syntactic Complexity of Story Retells Elicited from Children with ASD Spectrum Disorders (ASD)

Purpose: The purpose of this study was to examine the syntactic complexity of story retells of 5 children ranging in age from 8-12 with ASD Spectrum Disorder before, during and after narrative intervention. Method: Children participated in narrative intervention for 45 minutes, twice weekly for a period of time ranging from 19 to 33 sessions. Results: Results indicated that during baseline when children were not receiving instruction, their story retells contained more simple sentences than complex sentences. The use of complex sentences was observed to increase as children became more proficient in their narrative production skills. Discussion: Students generally improved on narrative discourse skills as a result of participating in the narrative intervention. The implications for clinicians working with students with ASD are compelling and suggest that narrative intervention may be associated with the additional benefit of improved complex sentence use.


Introduction
Complex syntax is important for the development of complex, coherent and logically constructed narratives. Children with ASD Spectrum Disorder (ASD) vary widely in terms of their mastery of complex syntax and often demonstrate difficulty with narrative comprehension and production [1]. Studies conducted in the 1970 have indicated specific deficits in the acquisition of syntax in children with ASD. For example, Bartolucci and Albers tested three groups of children (normal, autistic and mentally retarded) on their use of past tense and showed that children developing typically produced grammatically accurate sentences using past tense about 80% of the time, children with mental retardation did so 60% of the time, and children with ASD only 8% of the time [2].
Another study conducted by Pierce and Bartolucci [3] compared the syntax used by 10 children with ASD and 10 children diagnosed with mental retardation to a group of children developing typically. The authors indicated that the children with ASD were shown to demonstrate significant difficulties in their use of syntax as compared to the students with mental retardation and those developing typically.
There have been a number of studies that have concluded that syntax may not be specifically impaired in children with ASD. For example, a study by Cantwell et al., [4] examined the language of 19 ten-year-old boys with ASD, 23 boys with dysphasia and 5 with mixed diagnoses. Hour long language samples were collected while children played in their homes with caregivers. The examiners analyzed all of the utterances produced in the sample that were not direct imitations of caregiver utterances (e.g., echolalic. The researchers concluded that the syntax used by the children with ASD was as complex as utterances produced by their chronological-age-matched peers. More recent studies have reported that many but not all students with ASD may experience syntactic delays. For example, Rogers, Rice and Tager-Flusberg identified a large subgroup of children with ASD whose language profiles were similar to those of children with specific language impairment (SLI). Students were asked to respond to and probes designed to elicit third person singular and past tense morphological endings. Findings revealed that children developing typically responded correctly approximately 80% of the time, children with specific language impairment about 60% of the time, and children with children with ASD performing slightly better than children with SLI but somewhat worse than children developing typically. These findings suggest two subtypes of students with ASD; one that shares the grammatical deficits of specific language impairment, and one that does not [5].
In a similar study, Rapin and Allen showed that children with ASD used a more reduced set of syntactic structures compared to their typically developing peers [6]. Thus, many studies have shown that many students with ASD demonstrate delays in complex syntax use but the prevalence varies [1].

Narrative proficiency in ASD
Children with ASD who are functioning at higher language levels often continue to have documented difficulties in narrative proficiency [7]. The length, structure and syntactic complexity of the narratives elicited from children with ASD and in those of children developing typically are often similar. However, children with ASD tend to incorporate fewer story components (e.g., events, actions, characters) and include more irrelevant details in their narratives than typically  [7] evaluated the narratives of 15 children with ASD who were identified as demonstrating optimal outcomes, 15 children identified with high functioning ASD, and 15 typically developing children using the Child Language Data Exchange System, in order to determine if there were differences in pragmatic (narrative) impairments within these populations. The study showed that of the three groups, children with high functioning ASD demonstrated the most significant difficulty in their use of narrative elements (character, setting, events) and unambiguous pronoun references. These findings demonstrated that although children may be functioning at a seemingly high level, they may continue to demonstrate difficulties understanding and producing narratives. The purpose of the current study was to examine syntactic complexity in story retells produced by students with ASD as they participated in a narrative intervention program. It was hypothesized that syntactic complexity would increase as narrative proficiency increased.

Research questions
1. How syntactically complex is the narrative retells of children with ASD at baseline? 2. What is the relationship between narrative proficiency and narrative complexity at baseline? 3. What is the relationship between narrative proficiency and syntactic complexity during Phase I of intervention when students are first learning about story structure? 4. What is the relationship between narrative proficiency and syntactic complexity during Phases II and III of intervention when students become more proficient in narrative comprehension and production?

Methodology
Five students with ASD (two girls and three boys) between the ages of 8 and 12 were recruited from an ASD clinic [8]. All five participants were monolingual English speakers with an educational diagnosis of ASD. On the screening portion of the Universal Nonverbal Intelligence Test (UNIT) [9] each participant received a standard score of 70 or above. On the ASD Diagnostic Observation Schedule (ADOS-2) [10] the participants were characterized as verbally fluent. The students are considered verbally fluent when they form and express words compatible with the required criteria. Students were given the Clinical Evaluation of Language Fundamentals-Fourth Edition (CELF-4) [11], a general language proficiency measure, and four of the five earned standard scores at or below 85 suggesting they had low-average to low language skills.
We collected story-retell samples before intervention. Three students produced stories that included basic story elements (Participant 001, 003, 004) however they lacked internal responses (the emotions of the characters in regards to the action in the story) or plans. Though their stories were sometimes lengthy and detailed, they lacked organization and contained deficits in the establishment of causal and temporal coherence. Whereas Participant 001 and Participant 003 named their characters and settings the others did not. When evaluating the linguistic structure of the narrative samples, it was found that neither Participant 001 nor Participant 003 used subordinated clauses containing causal language, and they rarely used mental and linguistic verbs. Participant 004 inconsistently utilized subordinated and unsubordinated clauses, adverbs and elaborated noun phrases but failed to use causal terms to connect the elements in his stories. The other participants did not produce narratives with basic episodes.

Outcome Measures
Story retells were elicited individually by trained research assistants who read a story to the student and asking him or her to retell it. This occurred during each baseline session, after every other instructional session and during two follow-up sessions. Each narrative was recorded digitally and uploaded to a secure server for later transcription. To preface the students, the examiner said, "I am going to tell you a story. I want you to listen carefully. Tell me everything that you remember. You can think about it for a minute. Start when you're ready. " During this testing no prompting or visual materials were used. The examiners' only prompting was to ask if the child had finished their story.
The stories told by the students were transcribed according to the Systematic Analysis of Language Transcripts conventions [12] by research assistants who were blind to the purpose of the study. Both child and examiner utterances were transcribed verbatim. These utterances were segmented into units consisting of an independent main clause and phrases or clauses able to create a story withsubordinated to it, also known as communication units [13]. A second research assistant checked each transcript for spelling, mazing, morpheme segmentation and utterance segmentation. The two transcribers then listened to the digital recording together to resolve all transcription disagreements.

Sentence Complexity
Each sample was read and analyzed by two separate research assistants blind to the purpose of the study and the first author. As they read through each sample, sentences were categorized as simple or complex. Simple sentences were noted as sentences containing one independent clause. Each independent clause consisted of a noun phrase and a verb phrase. Complex sentences were determined to be any sentence that contained either two independent clauses joined by a conjunction or an independent clause and a dependent clause joined by a conjunction. A 'total number of utterances' was calculated for each student in every lesson. Complex sentence usage was determined by dividing the number of complex sentences by the total number of utterances.

Narrative Proficiency
The Monitoring Indicators of Scholarly Language (MISL) rubric was used to measure narrative proficiency [14]. The MISL rubric contains seven items to measure macrostructure and six items to measure microstructure in narratives. Each item for macrostructure (character, setting, initiating event, internal response, plan, attempt and consequence) was weighted to a score of 2. These scores reflected whether an element was absent (score of 0), emerging (score of 1), present (score of 2), or elaborated (score of 3). The total possible score for macrostructure scale was 21.
The MISL microstructure items were coordinating conjunctions, adverbs, mental verbs, linguistic verbs and elaborated noun phrase. Just as the macrostructure items were weighted to a score of 2, so were the microstructure items. The six items measuring microstructure were assigned a score of 0 if no exemplars were present, a score of 1 if the story contained one example, a score of 2 if two different examples were present, and a score of 3 if there were three or more different examples within the story. The total possible score for microstructure was 18. The MISL scores (macrostructure and microstructure) were combined and used as an index of overall narrative complexity.

Scoring Reliability
Twenty per cent of the original transcripts were re-transcribed to check the accuracy of the initial transcription process. The reliability between primary and secondary transcribers was 96% for communication-unit segmentation and 96% for identification of mazes (false starts, revisions). The transcripts were de-identified and using the MISL rubric the research assistants independently scored 20% of those transcripts. Interrater reliability for the MISL total scores was 95%.

Narrative intervention
The intervention used in this study is manualized and was implemented in a university clinic. Parents were given the opportunity to observe the sessions but did not participate in them. The narrative intervention was separated into three phases: Phase I-Teaching Story Elements, Phase II-Connecting and Elaborating Stories and Phase III-Creating and Editing Stories. Core story elements (character, setting, initiating event, internal response, plan, attempt, consequence, and reaction) were taught in Phase I. Icons (e.g., smiley face for external response; rocket taking off for initiating event) depicted each element and were included on a story board. Wordless picture books designed specifically for this intervention were used to explain and illustrate each story element. These books were also uploaded onto iPads and used by the clinician. As the story was told it was broadcasted on the iPad and each story element was verbally defined and examples were provided. Once all the story elements had been addressed, the students went through a series of lessons where they were asked to identify each element in the model stories and use those models to create new stories. The students use the model stories as a framework for their new stories (e.g., parallel stories) but a few elements were changed (e.g., different characters, different actions). They crafted these stories on a storyboard containing the story icons by drawing stick pictures [15]. The students were asked to practice telling stories without the support of icons and storyboards after they succeeded with the support of icons and storyboards.
At the end of each phase contextualized, literature-based activities were conducted using children's trade books [16]. Each literature unit began with a pre-story presentation. In the presentation the students were shown the book title, followed by a word review. Once the vocabulary review was complete, the book was read to the child. The clinician highlighted the story elements verbally and by using the graphic organizers and icons as they read the story. The students were asked questions related to the story elements and asked to retell the story with or without icons, picture manipulatives and storyboards. During each phase, students engaged in mini lessons targeting knowledge and use of concepts that contribute to narrative proficiency. For example, in Phase I, the students were in engaged in lessons teaching the concepts of before and after, first within the context of the literature book and then in independent practice activities involving real-life situations (e.g., you must cook the brownies before you eat them.) Once each participant had completed the 18 lessons in Phase 1 they were tested to determine whether or not they were prepared to move on to Phase II. The criteria for the students to begin Phase II were the child had to (a) Identify all of the icons by name, (b) Give satisfactory examples and/or definitions for each icon (c) Be able to create a story with assistance about a picture that contained all of the elements, and (d) Answer comprehension questions about story elements. Specific instructions and additional materials were provided in the manual in the event that one or more of these criteria were not met. These instructions were used to reteach each skill the student struggled with. This process was followed until the exit-testing criteria were met.
Phase II (Connecting and Elaborating Stories) was focused on creating more elaborate and complex stories through teaching linguistic structures, concepts and vocabulary. Making connections between story grammar elements and using mental state and causal language were highlighted in the instruction of Phase II. It also included instruction designed to broaden knowledge of microstructure (e.g., use of coordinative subordinating conjunctions, adverbs, elaborated noun phrases, and mental state and causal language). Dialogue was introduced as a way to elaborate their stories. The students also participated in activities designed to include complicating events as a way to create more complicated stories. In each lesson, emphasis was placed on using mental state and causal language in order to maintain connections between story elements. For example, students were encouraged to create stories that contained words such as because and so to explain why a character felt a certain way or why he or she planned to take various actions.
At the end of Phase II, students participated in contextualized literature-based activities that were based on a different, slightly more complex children's book that contained multiple examples of elaborated noun phrases. These lessons were similar to those in Phase I; however icons (e.g., dialogue, plan again) and a more elaborate storyboard were added. The new icons were added to provide support for the students to create more elaborate stories. Activities were designed to coincide with the literature book. Additional activities were also designed for independent practice involving the use of vertical structuring. For example, the students were given a scenario ("The boy fell off his chair. ") and were asked to tell how the character in the scenario felt (Child: 'embarrassed'). Then, the students were asked to explain why the character may have felt that way (Child: 'because he fell off the chair. ') The student's responses were combined by the clinician and then modelled back to the child (Clinician: 'The boy was embarrassed because he fell off his chair. ') The child was therefore provided a model of the use of causal state language in a complex sentence.
Exit testing was also conducted after the child had completed Phase II to determine whether they were prepared to move to Phase III (Creating and Editing Stories). The criteria to move to Phase III were that the child was required to create a story about a picture including all of the story elements, the words because or so, two or more feeling words, one or more mental or linguistic verbs, one or more adverbs and one or more elaborated noun phrases. The students also had to answer comprehension questions and recall story details. As in Phase I, additional materials were provided in order to reteach any of the skills the students continued to struggle with until they met the exit-testing criteria.
In Phase III, the students were given multiple opportunities to create, tell, edit and revise their own spontaneously generated stories. The students had opportunities to create stories with or without icon and storyboard support. All of the lessons were conducted within the context of literature books that contained multiple, embedded episodes and more complex concepts, vocabulary and syntax than in Phase I and II. In the final phase the students started by completing literaturebased activities followed by activities on the cause-and-effect relationships indicated by conditional clauses containing the adverbs if and then. The students completed these activities in guided and independent practice settings. Using the content of the book, structure was taught (e.g., if the whale gets stuck, then he should ask others for help pushing off the sand). This context was then extended to real life situations (e.g., if you found a puppy, then you should try to find the owner). Additional books were used by the clinicians to teach this concept when the students did not demonstrate sufficient understanding.
Developing independence in understanding and use of narrative macrostructure and microstructure was the main focus of Phase III. In order to accomplish the purpose of this phase, lessons were designed to develop the metacognitive skills necessary in order for the students to judge the adequacy of their own stories. To facilitate their independence, the students were taught to use a self-scoring rubric containing questions targeting macrostructure and microstructure elements in their stories. For example, the rubric asked, "Does the story have at least two characters?" and, "does my story have words that relate to thinking or dialogue?" The self-scoring rubric was first used with the book Little Croc and the Whale [17] and then in the stories the students created from the single-scene and sequenced scene prompts.

Fidelity of intervention implementation
To accompany the lessons in the intervention program, an observation checklist was created and used by a member of the research team. A member of the research team observed each session to ensure that all aspects of the lesson were being taught. If the integrity of any lesson fell below 85%, the research staff held a meeting with the clinician immediately after the lesson to address what was omitted. There were sessions in which the fidelity was compromised due to omission; however, this only occurred when there was a lack of time to complete the lesson. Omitted information was always addressed in the following sessions after the clinician reviewed the material in the previous lesson. A member of the research team who did not observe the lesson in person reviewed twenty per cent of the lessons a second time. Interrater reliability was calculated point by point and was 85% or greater for implementation.

Research design
A concurrent multiple-baseline across-participants design. Two sets were created from the five participants. The students with higher language ability were the first set and those with lower language ability were the second set. Participants began at baseline within their respective sets at about the same point in time to control for external influences on participant performance [18]. The first child in each set started intervention at the same time while the beginning of intervention was lagged for other participants in each set to control for maturation and history threat to internal validity.

Data Analysis
A visual analysis of the data was conducted following the Kratochwill et al. [19] method. Predictable patterns in performance were identified during the baseline phase. After the baseline, data was collected throughout the intervention phases, and predictable patterns of performance were noted. It was then determined whether or not there was evidence that the intervention phase held a correlation with the change in participants' MISL scores. The parameters recorded and examined during and between phases were (a) the mean score for each phase, (b) the slope of the fitted regression line for the data, (c) the range of variability in the data, (d) immediacy of trend change at phase initiation, (e) significance of score change between each phase, and (f) consistency of data patterns across phases [19].
While there is some dispute over the validity of visual and statistical inspection for interpretation of single-subject studies [20], as single cases (especially from low-incidence populations such as ASD, who experience different levels of variability from the expected norm) do not meet many of the basic assumptions for accepted statistical analyses, the American Psychological Association recommends that all studies submitted for publication include certain quantifiable and visual data for ease of comparison across studies [21].

Participant 001
Baseline Visual data for Participant 001's complex sentence percentages are presented in Figure 1 and Table 1. These figures, for each participant, represent the percentage of syntactically complex sentences that were used by the students on average over each session. The variation lines represent the standard deviations from these averages and show how consistent each participant was in grammaticality and syntax throughout the sessions, with narrower variation lines indicating higher consistency. Participant 001 attended five baseline sessions prior to beginning intervention. Figure 2 compares the narrative ability of Participant 001 scores according to MISL and her use of syntactic complexity throughout the study. Participant 001 had moderate ability to produce narratives at baseline. Participant 001 earned high scores (2 or higher) during baseline on the MISL for use of character (gave characters' names), but other macrostructure items (e.g., setting, initiating event, action, consequence) varied widely, with her highest scores earned during Baseline Session 2 (a story about a scene depicting a crowd of people at the beach). Participant 001 utilized complex sentences only a small portion of the time in the five baseline sessions with consistency.

Intervention
Participant 001 attended 23 intervention sessions spread across 11 weeks. She spent nine sessions in Phase I, eight in Phase II and six in Phase III. She also participated in two follow-up sessions. MISL scores were obtained once weekly (after every other intervention session). In Phase I of intervention, her narrative abilities increased significantly. Her use of complex sentences increased slightly but there was marked variation.
In Phase II, there was a slight drop in her MISL scores but her syntactical skill improved significantly. During Phase III there was a small increase in her narrative proficiency skills and her MISL scores become consistent. Her syntax complexity scores were maintained throughout intervention.

Participant 002
Baseline Visual data for Participant 002's complex sentence percentages are presented in Figure 3 and Table 1. Participant 002 attended four baseline sessions prior to beginning intervention. Figure 4 compares the narrative ability of Participant 002 scores according to MISL and his use of syntactic complexity throughout the study.

Intervention
Participant 002 participated in 33 intervention sessions across 17 weeks. He participated in 17 sessions during Phase I, nine in Phase II, and seven in Phase III, and he attended two follow-up sessions. During Phase I there was an increase in his narrative proficiency while there was a very large increase in the use of complex syntax.
In Phase II his narrative skills continued to increase, as did his use of complex syntax. However, there was a lot of variation.
Phase III shows an increase in narrative proficiency and he consistently received high MISL scores. However, there was a slight decrease in the complexity of his syntactical forms. During this phase, his complex sentences remained consistent and had little to no

Participant 003
Baseline Visual data for Participant 003's complex sentence percentages are presented in Figure 5 and Table 1. Figure 6 compares the narrative ability of Participant 003 scored according to MISL and his use of syntactic complexity throughout the study.
Participant 003 attended 12 baseline sessions prior to beginning intervention. There were downward trends in her MISL scores, which controlled for history and maturation effects. She earned a score of 2 or higher for internal response in one story she told during baseline. During baseline, she demonstrated moderate narrative skills, though they were not entirely complete. Her complex sentences were produced inconsistently and very few were used in baseline sessions.  In Phase I, Participant 003's narrative skills increased slightly, however, as this occurred her complex sentence usage dropped significantly. As intervention continued, her use of complex sentences increased along with her MISL scores. In Phase III, her syntactic complexity scores were lower than sentences she produced at baseline.

Participant 004
Baseline Visual data for Participant 004's complex sentence percentages are presented in Figure 7 and Table 1. Figure 8 compares the narrative ability of Participant 004 scores according to MISL and his use of syntactic complexity throughout the study.
Participant 004 remained in baseline for nine sessions prior to beginning intervention. His MISL scores were highly variable and followed an upward trend during baseline. Participant 004's parents were anxious for him to begin treatment, so treatment was initiated even though his baseline data were trending upward. Participant 004 demonstrated knowledge of the crucial elements of stories (initiating event, action, and consequence scores of 2 or higher) in six of the nine baseline sessions. Participant 004 included an internal response in one story during baseline.  intervention proceeded, his MISL and syntactical scores increased at an almost equal rate. His consistency also became more stable as the intervention went on.

Participant 005 Baseline
Visual data for Participant 005's complex sentence percentages are presented in Figure 9 and Table 1. Figure 10

Intervention
Participant 005 participated in 27 intervention sessions across 14 weeks. He attended 17 sessions spent in Phase I, three in Phase II, and seven in Phase III, and he participated in two follow-up sessions. Based on the results seen from Participant 5, we can see that he was not cooperative in his intervention. In Phase I his syntactic complexity was variable but he seemed to be improving. He showed improvement from baseline to Phase I in his MISL scores, but was noncompliant and often refused to tell stories. Therefore, his performance was highly variable.

Discussion
The overarching research goal of this project was to determine whether the introduction of a three-phase approach to narrative intervention was associated with reliable changes in overall story complexity and an increase in the use of complex sentences in verbal children with ASD. Results indicated that during baseline when students were not receiving instruction, 3 out of 5 students' story retells contained some complex sentences.
The first research question asked, "How syntactically complex are the narrative retells of children with ASD at baseline?" The findings suggested that the utterances used by children with ASD who participated in this study were not sufficiently complex during baseline. None of the stories told by students contained 30% or more utterances that were complex.
The second research question asked, "What is the relationship between narrative proficiency and narrative complexity at baseline?" The findings suggested that the more proficient the students were at telling stories, the more complex their stories were, implementing more of the macro and micro structures of narratives and therefore creating a higher syntactic complexity in their narratives from a baseline level.
The third research question asked, "What is the relationship between narrative proficiency and syntactic complexity during Phase I of intervention?" Findings for this study suggested that as narrative abilities improved in Phase I of intervention, the amount of syntactically complex sentences increased slightly for all participants as their MISL scores increased.
The fourth research question asked, "What is the relationship between narrative proficiency and syntactic complexity during Phases II and III of intervention?" Our findings suggested that the progress that was observed in Phase I for the students' syntax continued to rise during Phases II and III.
Complex syntax is important for the development of complex, coherent and logically constructed narratives. Children with ASD vary widely in terms of their mastery of complex syntax [1]. This study suggests that a narrative intervention may prove beneficial not only for improving the content and coherence of stories but also for improving the use of complex sentence structures for children with ASD. Though the mastery of syntax among the participants did vary, the narrative intervention proved beneficial for all of the students involved. Their complex syntax use was judged to improve in a parallel fashion along with their narrative skills.

Clinical Implications
The instruction that was provided in this study resulted in positive outcomes for narrative comprehension and production and complex sentence use for children with ASD. Three out of five children produced some complex sentences before beginning instruction; however, Participant 002 was shown to use complex sentences only after beginning instruction. After instruction, his use of complex sentences and his ability to recall and retell stories improved dramatically. For most of the students in the study, their use of complex sentences gradually increased over the course of the study. As they told more complex stories, they incorporated more complex linguistic structures. These abilities increased gradually for three of the five students throughout intervention.
It is possible that the use of narrative intervention can be useful not only in helping these students understand and tell stories, but to improve their comprehension and use of complex syntax. Clinicians may not have to focus on syntax outside of the context of narrative instruction. This would be an efficient way to implement therapy for children with ASD so that they can learn to implement new skills while improving those that they already possess. This type of intervention mirrors the authentic framework that is being used in classrooms, and addresses the language needs of children with ASD.

Summary
The results of this single-case study demonstrated that a three-phase narrative intervention program resulted in changes in overall syntactic complexity in five children with ASD. The intervention effects were demonstrated at different points during intervention. This intervention was manipulated systematically and outcome variables were measured by multiple assessors with high interrater reliability. There were gradual changes in level of syntactic complexity between baseline and intervention phases for all five participants and changes in trend for four of the five participants. These findings show that students with ASD may increase their syntactic abilities by participating in an intervention containing external organizational scaffolds that were integrated with particular linguistic structures. The students were given the opportunity to practice telling coherent and cohesive stories in phases starting with learning about story grammar elements with an emphasis on how they related to each other. In Phases II and III, the students gradually took on more responsibility for integrating macrostructure and microstructure elements.