Signposting for diagnosis of Autism Spectrum Disorder using the Diagnostic Interview for Social and Communication Disorders (DISCO)

Recent research has investigated the capability of the Diagnostic and Statistical Manual for Mental Disorders ( DSM-5 ) descriptions to identify individuals who should receive a diagnosis of Autism Spectrum Disorder (ASD) using standardised diagnostic instruments. Building on previous research investigating behaviours essential for the diagnosis of DSM-5 ASD, the current study investigated the sensitivity and speciﬁcity of a set of 14 items derived from the Diagnostic Interview for Social and Communication Disorders ( DISCO Signposting set) that have potential for signposting the diagnosis of autism according to both the new DSM-5 criteria for ASD and ICD-10 criteria for Childhood Autism. An algorithm threshold for the Signposting set was calculated in Sample 1 ( n = 67), tested in an


Pl e a s e n o t e:
C h a n g e s m a d e a s a r e s ul t of p u blis hi n g p r o c e s s e s s u c h a s c o py-e di ti n g, fo r m a t ti n g a n d p a g e n u m b e r s m a y n o t b e r efl e c t e d in t his ve r sio n.Fo r t h e d efi nitiv e ve r sio n of t hi s p u blic a tio n, pl e a s e r ef e r t o t h e p u blis h e d s o u r c e.You a r e a d vis e d t o c o n s ul t t h e p u blis h e r's v e r sio n if yo u wi s h t o cit e t hi s p a p er.
Thi s v e r sio n is b ei n g m a d e a v ail a bl e in a c c o r d a n c e wit h p u blis h e r p olici e s. S e e h t t p://o r c a .cf. a c. u k/ p olici e s. h t ml fo r u s a g e p olici e s.Co py ri g h t a n d m o r al ri g h t s fo r p u blic a tio n s m a d e a v ail a bl e in ORCA a r e r e t ai n e d by t h e c o py ri g h t h ol d e r s .

Introduction
Although autism has long been described as a spectrum (Wing, 1996), the condition has only recently been given the name Autism Spectrum Disorder (ASD) in the fifth edition of the Diagnostic and Statistical Manual for Mental Disorders (DSM-5;American Psychiatric Association, 2013).Initially, the validity of the DSM-5 description was questioned (Barton, Robins, Jashar, Brennan, & Fein, 2013;Gibbs, Aldridge, Chandler, Witzlsperger, & Smith, 2012;Matson, Belva, Horovitz, Recent research has investigated the capability of the Diagnostic and Statistical Manual for Mental Disorders (DSM-5) descriptions to identify individuals who should receive a diagnosis of Autism Spectrum Disorder (ASD) using standardised diagnostic instruments.Building on previous research investigating behaviours essential for the diagnosis of DSM-5 ASD, the current study investigated the sensitivity and specificity of a set of 14 items derived from the Diagnostic Interview for Social and Communication Disorders (DISCO Signposting set) that have potential for signposting the diagnosis of autism according to both the new DSM-5 criteria for ASD and ICD-10 criteria for Childhood Autism.An algorithm threshold for the Signposting set was calculated in Sample 1 (n = 67), tested in an independent validation sample (Sample 2; n =78), and applied across age and ability sub-groups in Sample 3 (n = 190).The algorithm had excellent predictive validity according to best estimate clinical diagnosis (Samples 1 and 2) and excellent agreement with established algorithms for both .The signposting set has potential to inform our understanding of the profile of ASD in relation to other neurodevelopmental disorders and to form the basis of a Signposting Interview for use in clinical practice.ß 2014 The Authors.Published by Elsevier Ltd.This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/3.0/).Kozlowski, & Bamburg, 2012;Matson, Hattier, & Williams, 2012;Matson, Kozlowski, Hattier, Horovitz, & Sipes, 2012;Mattila et al., 2011;Mayes, Black, & Tierney, 2013;McPartland, Reichow, & Volkmar, 2012;Wilson et al., 2013;Worley & Matson, 2012;Young & Rodi, 2013).However, several recent studies using different instruments show evidence for good sensitivity and specificity of the criteria (Frazier et al., 2012;Huerta, Bishop, Duncan, Hus, & Lord, 2012;Kent, Carrington et al., 2013).The use of one method, the Diagnostic Interview for Social Communication Disorders (DISCO; Leekam, Libby, Wing, Gould, & Taylor, 2002;Wing, Leekam, Libby, Gould, & Larcombe, 2002) indicated that the description of DSM-5 Autism Spectrum Disorder could be captured effectively using a set of items within a single diagnostic interview and without any modification to the DSM-5 rules (Kent, Carrington et al., 2013).The diagnostic algorithm for DSM-5 developed using items from the DISCO had excellent predictive validity in two samples of children, and excellent sensitivity in a third sample of children, adolescents, and adults (Kent, Carrington et al., 2013).
In a recent study exploring an abbreviated algorithm for DSM-5, a small set of 14 highly discriminating items were identified from the DISCO based on their predictive validity for individuals with clinical diagnoses of autism compared with individuals with confirmed diagnoses of intellectual disability or language impairment (Carrington, Kent et al., 2014).Eleven of these items related to social-communication behaviours -with seven items specifically related to the 'socio-emotional reciprocity' sub-domain -and the remaining three items related to the sub-domains of 'stereotyped or repetitive motor movements, use of objects or speech', or 'insistence on sameness, inflexible adherence to routines, or ritualised patterns of verbal or non-verbal behaviour'.Given the relative lack of items measuring restricted, repetitive patterns of behaviours, this set of items did not represent the full diagnostic profile specified by DSM-5; therefore, the item set itself could not be considered as a candidate set for an abbreviated DSM-5 algorithm.However, in the current study, we examine these 14 items further to examine whether a minimum threshold applied to this set of items might be sufficient to indicate a diagnostic outcome of either DSM-5 ASD or ICD-10 Childhood Autism.A DISCO algorithm based on this highly reduced set of 14 items (hereafter referred to as the Signposting set) has utility for future research, addressing the question of which behaviours are truly distinct to the behavioural profile of ASD.Moreover, if found to have good predictive validity, an interview based on these items (DISCO Signposting Interview) may also have potential for use by clinicians to signpost the need for a fuller diagnostic assessment using either DSM-5 criteria for ASD or using ICD-10 criteria for Childhood Autism.
The term 'Signposting' is used in the current study to differentiate from the more commonly used term 'screening'.This differentiation was introduced as the intended use of the item set and algorithm in the current study was not as a general screening tool; rather, the aim was to determine whether the item set and algorithm could guide clinicians in selecting an appropriate diagnostic pathway when first assessing cases where a concern has been raised.Consistent with this aim, brief, age-specific ten-item 'red flag' questionnaires have been developed from the Autism Spectrum Quotient (AQ; Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, 2001) to help guide the referral of cases for full diagnostic assessment (AQ-10; Allison, Auyeung, & Baron-Cohen, 2012).When a threshold of six items was set (the value that best balanced sensitivity and specificity), the AQ-10 for each age-group had high levels of both sensitivity and specificity in comparison with a non-clinical control sample (Allison et al., 2012;Booth et al., 2013).However, questionnaire measures are commonly thought to have two major limitations in clinical practice.First, parent-and particularly self-report measures may be vulnerable to under-reporting or over-reporting of symptoms due to a lack of insight into the presence or impact of certain behaviours.Second, individuals may misunderstand written questions, and therefore provide responses that are not a true representation of the behaviour.An interview conducted by a trained administrator would provide the opportunity for more detailed questioning, thus ensuring that an individual has understood the question, and allowing the opportunity for identifying and exploring areas where an individual may not fully appreciate the impact of a behaviour.This measure could therefore provide an accurate and objective measure of behaviours associated with ASD, thus assisting clinicians when referring individuals for further diagnostic assessment.
In order to investigate the potential utility of the 14-item DISCO Signposting set, a new Signposting algorithm threshold was designed in the current study, and tested against three diagnostic outputs.The threshold was calculated using Receiver Operating Characteristic (ROC) curve statistics in a single development sample (Sample 1).The predictive validity of the algorithm applied to the Signposting set was then tested relative to best estimate clinical diagnosis made according to ICD-10 criteria in Sample 1 and in a second, independent validation sample (Sample 2).In an additional step, outcome on the algorithm for the Signposting set was compared with outcome on both the full DISCO DSM-5 (Kent, Carrington et al., 2013)a n dICD-10 ( Leekam et al., 2002) algorithms in both Samples 1 and 2, and a third sample of children, adolescents, and adults (Sample 3).Finally, the sensitivity of the Signposting algorithm across age and ability level was investigated in Sample 3. Good agreement between outcome on the algorithm for the Signposting set and algorithms for both DSM-5 and ICD-10 criteria would support the potential for this item set to form the basis of a DISCO Signposting Interview1 to guide further diagnostic assessment according to international diagnostic criteria beyond DSM-5.
Sample 1 (Development) comprised 82 children (34-140 months), 36 (31 males) of whom had a clinical diagnosis of ICD-10 Childhood Autism or DSM-IV-TR Autistic Disorder (18 higher ability, 18 lower ability).The non-ASD clinical control group included 31 children (19 males) with either language impairment (LI; n = 14) or intellectual disability (ID; n = 17) who comprised the higher and lower ability clinical control groups respectively.Fifteen typically developing children comprised a non-clinical control group (nine males).Children in the two clinical groups were recruited through clinical services and special schools.Diagnoses were made by qualified clinicians who were not connected to the research study and diagnosed using other methods (e.g. the Autism Diagnostic Interview-Revised (ADI-R); Lord, Rutter, & Le Couteur, 1994), without reference to the DISCO.All children in the ASD group subsequently qualified for a DISCO ICD-10 algorithm diagnosis, with excellent inter-rater reliability at both the item level and for diagnostic outcome (Leekam et al., 2002).The higher and lower ability sub-groups were defined at the time of recruitment based around an IQ of above or below 70 respectively.These groupings were confirmed using either the Leiter International Performance Scale (Leiter, 1979) or the Bayley Scale for Infant Development (Bayley, 1993; composite performance mental age scores were converted to IQ scores).The ASD and control groups were matched on IQ and chronological age; however, there were significantly more males in the ASD group than the control group (x2 ð1Þ ¼ 6:38, p < .05).Data from this sample were used to calculate the threshold for the item set.Sample 2 (Independent Validation) included 52 children with ASD (DSM-IV-TR Pervasive Developmental Disorder; 43 males, 34-137 months, 85% with co-occurring ID) and a non-ASD clinical control group of 26 children with ID (16 males, 48-134 months).Children were recruited through clinical services and special schools in the Netherlands (Maljaars et al., 2012), and diagnoses were made by an independent clinician without reference to the DISCO as above.The diagnostic reports of children in the ASD group were reviewed, and children for whom the diagnosis was not clear were not included in the sample.Moreover, substantial agreement between diagnostic outcome on the DISCO and the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2000) was reported (Maljaars et al., 2012).The ASD and clinical control group were matched for chronological age.The sample also included 37 typically developing children (15 males, 24-49 months).The ASD group and both control groups were matched for non-verbal mental age, measured with a Dutch test for non-verbal intelligence (SON-R 2.57; Tellegen, Winkel, Wijnberg-Williams, & Laros, 1998).Data from this sample were used to test the algorithm and threshold for the Signposting set developed based on the data from Sample 1.
Sample 3 comprised 190 children (n = 112), adolescents (n = 33) and adults (n = 45) assessed using the DISCO in a specialist tertiary clinic by the clinicians who designed and developed the interview.All individuals received DISCO algorithm diagnoses of Childhood (n = 180) or Atypical (n = 10) Autism.IQ was primarily measured using age-appropriate Wechsler Intelligence Scales and participants were divided into high and low ability groups (IQ above and below 70 respectively; Leekam et al., 2007).The sensitivity of the algorithm was assessed across both age and ability level using data from this sample.

Measures and item selection
The DISCO is a semi-structured, standardised developmental history interview that guides clinicians in collecting a detailed profile of an individual's strengths and difficulties.The interview is typically conducted with a parent or carer, but for adults, can be conducted with the individual themselves.Questions focus on seven broad areas, covering ''Family and medical background'', ''Infancy'', ''Developmental skills'', ''Repetitive, stereotyped activities'', ''Emotions'', ''Maladaptive behaviour'', ''Interviewers' Judgement of quality'', with additional questions considering other psychiatric disorders and forensic problems. 2The DISCO is widely used in clinical practice internationally, and has been validated relative to other, well-established diagnostic instruments, including both the ADI-R (Nygren et al., 2009) and the ADOS (Maljaars et al., 2012).In the DISCO, each item is typically rated according to the level of impairment both for lifetime (ever) and current scores.Codes for behaviours typically indicate marked (0), minor (1), or no problem (2), with some items including an additional rating to indicate that a skill or behaviour is not yet achieved or is not present.In the full DISCO DSM-5 algorithm, codes for each item were selected that best fit the DSM-5 descriptions; although both lifetime (ever) and current scores were available, only ever scores were used for these analyses; this is common practice for the development of lifetime diagnostic algorithms (Kent, Carrington et al., 2013).Details of the item selection are described elsewhere (Carrington, Kent et al., 2014).In brief, predictive validity of each of the items included in the full DSM-5 algorithm was calculated using data from Sample 1.Using a stringent threshold of p < .001, the most highly discriminating items were identified.These items are listed in Table 1 reported by Carrington, Kent et al. (2014), and are reproduced in Appendix 1.

Threshold calculation
The threshold for the item set was calculated using receiver operating characteristic (ROC) curve statistics, calculated using data from Sample 1 only.In the development of the original DISCO DSM-5 algorithm (Kent, Carrington et al., 2013), a threshold was established for each sub-domain specified by DSM-5 (three sub-domains of social communication behaviours and four sub-domains of restricted and repetitive patterns of behaviours) and rules were set that governed the combination of these sub-domains as specified by DSM-5. 3 The current item set, however, required a different approach; the Signposting set consisted only of a single set of items, without sub-domains and a single threshold was therefore required.Consequently, a more stringent approach was adopted, and the threshold was calculated using the Youden J statistic (Youden, 1950).The Youden J statistic is a standardised statistic that has been used in the development of diagnostic assessments for ASD (e.g.Cohen et al., 2010) and in other areas of medicine (e.g.Chiu et al., 2011;Portalez et al., 2012).This statistic identifies the value that provides the optimal balance between both sensitivity and specificity ((sensitivity + specificity) À 1) and is therefore the most stringent statistical method to identify a cut-off or threshold in diagnostic measures.

Data analysis
In Sample 1, the internal consistency of the item set was first assessed using Cronbach's alpha and inter-item correlations were calculated to assess redundancy.Then the predictive validity of the thresholded item set was tested relative to participants' clinical diagnosis using ROC curve analyses in both Sample 1 (Development Sample) and Sample 2 (Independent Validation Sample).Finally, in addition to comparison with diagnostic outcome based on participants' original clinical diagnosis, outcome on the Signposting algorithm was also compared with outcome on previously published full DISCO algorithms for both DSM-5 (Kent, Carrington et al., 2013)a n dICD-10 ( Leekam et al., 2002) using McNemar's statistic in all three samples.Finally, the sensitivity of the algorithm for the Signposting set (relative to outcome on the DISCO ICD-10 algorithm) was investigated in different age and ability groups in Sample 3 using Chi-square analyses.

Results
The internal consistency of the 14-item Signposting set as calculated in Sample 1 was excellent (alpha = .92)with very little redundancy; 'does not give comfort to others' was highly correlated with both 'no emotional response to age peers' (r = .80)and 'lack of awareness of others' feelings' (r = .76).Given that removal of any of these three items decreased the internal consistency of the set as a whole, all items were retained.The results from ROC curve analyses in Sample 1 are presented in Table 1; the maximum Youden J statistic indicating the optimal balance between sensitivity and specificity was achieved with a threshold of six. a The maximum value for Youden J, indicating the threshold at which the ROC curve maximally deviates from the chance line, and, therefore, providing the best balance between sensitivity and specificity.
Sensitivity, specificity, and AUC for the algorithm applied to the Signposting set in both Sample 1 and 2 are reported in Table 2.The sensitivity of the item set was excellent in Sample 1 (.97), although slightly lower in Sample 2 (.89).In the most stringent analyses, when clinical controls were included only (omitting typically developing individuals), the specificity of the item set was high (Sample 1 = .81;Sample 2 = .89);however, when typically developing individuals were included, resulting in a mixed control group, specificity was improved (Sample 1 = .87;Sample 2 = .95).Moreover, as is clear from Table 2, analyses comparing the ASD group with typically developing controls only would result in perfect specificity.
Comparison with outcome on previously published DISCO algorithms revealed that the sensitivity of the Signposting algorithm was comparable to the original DISCO DSM-5 algorithm in all three samples, and specificity was comparable to the full DISCO DSM-5 algorithm in both Samples 1 and 2 (p > .05).Importantly, the outcome on the Signposting algorithm was also statistically comparable to outcome on the DISCO ICD-10 algorithm in all three samples, supporting the use of the Signposting set in guiding diagnosis according to ICD-10 as well as DSM-5.Finally, the item set had excellent sensitivity (above .90) in all age and ability sub-groups in Sample 3 (Table 3).Chi-square analyses revealed no significant variation according to either age or ability level (p > .05).

Discussion
This study represents the first step in developing an algorithm for a set of 14 highly discriminating items previously identified from the DISCO (Carrington, Kent et al., 2014) and referred to as the DISCO Signposting set.This algorithm has the potential to guide diagnosis of both DSM-5 Autism Spectrum Disorder (ASD) and ICD-10 Childhood Autism.It builds on previous research using the DISCO, by demonstrating excellent levels of predictive validity relative to clinical diagnosis according to ICD-10, in addition to excellent agreement with outcome on previously established DISCO algorithms for both  DSM-5 and ICD-10.Finally, the algorithm applied to this Signposting set had excellent sensitivity across age and ability sub-groups in a sample of children, adolescents and adults.Overall, the results from this study demonstrate the potential of this small set of items, considered essential to the DSM-5 descriptions of ASD, to identify autism according to two international classification systems across a broad age range.This work is of significance for future research examining features that may distinguish ASD from other neurodevelopmental disorders such as ADHD and speech and language impairments, as well as for the investigation of common features across these disorders.This study found that an optimal threshold level of just six items could be applied to a very small item set in order to discriminate between individuals with a confirmed ICD-10 clinical diagnosis of Childhood Autism and those in a clinical comparison group who had an intellectual disability or language impairment.Moreover, further investigation of the predictive validity of the algorithm applied to the Signposting set in a large sample of individuals with a broader range of clinical diagnoses, developmental levels and chronological age could further inform our understanding of the overlapping profiles of neurodevelopmental disorder, including the newly defined Social (Pragmatic) Communication Disorder (DSM-5), and thus contribute to the development of new diagnostic criteria .
Even though the ASD groups included in these samples had both social-communication symptoms and repetitive behaviours by virtue of their clinical diagnosis, the discrimination reported in this study was predominantly based on social and communication features.The predominance of social-communication items within the Signposting set is consistent with evidence that social-communication symptoms are among the most common early signs of ASD captured by screening tools (Charman & Gotham, 2013).Indeed, with a threshold of just six items, it would theoretically be possible to score on the Signposting algorithm entirely on the basis of social-communication behaviours.Despite the bias within the Signposting set toward social-communication behaviours, comparable sensitivity for the DISCO Signposting algorithm when applied to the item set was found across the age and ability sub-groups of Sample 3 and across the broad, heterogeneous range of symptom patterns within the groups.This suggests that the items measured behaviours that are relevant across the autism spectrum.
Analysis of the items included in the original DISCO DSM-5 algorithm identified three items that were present in over 90% of cases in both ability groups and the child and adult groups included in Sample 3 (Kent, Carrington et al., 2013).These three 'global' items were all included in the Signposting set identified in the current study.Moreover, the remaining items included in the Signposting set all had comparable frequency at different ages and level of ability (Kent, Carrington et al., 2013).As noted by Carrington, Kent et al. (2014), items specifically associated with one particular sub-group of individuals with ASD, such as higher ability individuals, would be endorsed less frequently within the sample as a whole than items with a more global relevance, and would therefore be less likely to differentiate between an ASD and clinical control sample.Consequently, the selection criteria for the DISCO Signposting set was effectively biased toward the inclusion of items measuring behaviours common across developmental and ability levels.The relative paucity of items from the DSM-5 domains measuring restricted and repetitive patterns of behaviour may, therefore, reflect a greater range of potential manifestations of these behaviours contributing to the heterogeneous profile of ASD.
The results from this study also have potential implications for clinical practice.There is potential to develop the Signposting algorithm further into a Signposting Interview for use by clinicians to signpost the need for more comprehensive diagnostic assessment using either DSM-5 criteria for ASD or using ICD-10 criteria for Childhood Autism.As outlined in Section 1, the age-specific red-flag measures derived from the AQ were also intended to guide the referral of cases for diagnostic assessment (Allison et al., 2012).Each of these questionnaires had excellent sensitivity (children = .95;adolescents = .93;adults = .88)and specificity (children = .97;adolescents = .95;adults = .91);however, specificity was calculated relative to a non-clinical control group.In the current study, the specificity of the Signposting algorithm applied to the 14-item Signposting set was calculated relative to a clinical control sample, thus providing a more stringent test of predictive validity.Moreover, the DISCO Signposting set and algorithm had excellent sensitivity for children, adolescents and adults (Sample 3), suggesting that this single instrument could be used across age-groups, unlike the age-specific 'red flags' measures.Finally, the use of a clinician-led interview could circumvent potential limitations of questionnaires which are reliant on an individual's interpretation of questions and insight into their child's or their own behaviour.
Despite their clear potential, until the Signposting item set and algorithm are fully tested with a clinically referred sample, the clinical utility of the Signposting Interview as a guide for referral for more comprehensive diagnostic assessment cannot be known.The analysis for this study was based on secondary data and the Signposting Interview first needs to be used and tested as a stand-alone interview method.Despite these limitations, this study does demonstrate excellent predictive validity of an algorithm threshold applied to the Signposting set, indicating a strong relation to clinical outcome in this limited context.This work, therefore, highlights the opportunity to develop a 'family' of nested interviews in which the identification of 'signs' for autism and more detailed follow-up through more comprehensive assessment could be completed using the same concepts and range of interview items.

Table 1
ROC curve analyses applied to Signposting set in Sample 1 (Development Sample).

Table 2
Sensitivity and specificity of the DISCO Signposting algorithm compared with clinical diagnosis in Sample 1 and Sample 2 (Independent Validation Sample).

Table 3
Sensitivity of the DISCO Signposting algorithm across age and ability (low and high) in Sample 3 compared with ICD-10 algorithm output.