ARTICLES
Reliability of the Services for Children and Adolescents–Parent Interview

https://doi.org/10.1097/01.chi.0000139558.54948.1fGet rights and content

ABSTRACT

Objective:

To describe the psychometric properties and test the reliability of a new instrument designed to measure mental health services use within pediatric clinical samples, the Services for Children and Adolescents–Parent Interview (SCAPI), which was developed by the National Institute of Mental Health Multimodal Treatment Study of Children With Attention Deficit Hyperactivity Disorder (MTA).

Method:

Similarities and differences with other measures of services use are described. Ten types of services are measured by the SCAPI. Formal test-retest reliability testing was carried out in 104 subjects with a mean time between tests of 18 days.

Results:

Test-retest κ values ranged from 0.49 to 1.00, with an overall κ value for all services of 0.97. Seven of the 10 service types had κ values of 0.75 or higher, indicating excellent reliability. In addition, matched responses on specific questions about reasons for seeking services, starting and ending dates, number and length of visits, and type of provider seen were more than 75% for most service categories, consistently so for reporting of medications and school services.

Conclusions:

The SCAPI is a reliable instrument for assessing mental health and related services use and may be an especially valuable adjunct in studies involving clinical samples, especially clinical trials.

Section snippets

METHOD

A test-retest reliability study was conducted to determine whether consistent responses could be provided by respondents to the questions that comprise the instrument. Power analysis and statistical consultation indicated that 90 paired test-retest interviews would adequately test stability of responses. The six sites involved in the MTA (University of Pittsburgh, University of California at Irvine, Columbia University, University of California at Berkeley, Duke University, New York

Use and Number of Services

For the 10 service types, test-retest κ values ranged from 0.49 to 0.98, with an overall use of services κ value of 0.97 (Table 1).

Seven of the 10 κ values reflected excellent agreement, using Fleiss guidelines (>0.75), and three reflected fair to good agreement (>0.40 to <0.75). Reports of child medication, overnight stays, and parent medication achieved κ values in the 0.90 range (0.98, 1.00, and 0.93, respectively). School services, which typically have the lowest reliability, achieved a

DISCUSSION

This is the first report of the reliability of an instrument designed specifically to measure mental health and related services use within pediatric clinical trials. It suggests that parents or caregivers can reliably report services use, even to the level of specificity of dose, provider type, and dates of services use. The significance of having a reliable instrument with which to obtain these reports centers on three major issues. First, the public health significance of examining both the

REFERENCES (33)

  • L Bickman

    The evaluation of a children's mental health managed care demonstration

    J Ment Health Admin

    (1996)
  • BJ Burns

    A call for a mental health services research agenda for youth with serious emotional disturbance

    Ment Health Serv Res

    (1999)
  • BJ Burns et al.

    Children's mental health service use across service sectors

    Health Aff (Millwood)

    (1995)
  • G Canino et al.

    Assessing functional impairment and social adaptation for child mental health services research

    Ment Health Serv Res

    (1999)
  • EJ Costello et al.

    The Great Smoky Mountains Study of Youth: functional impairment and serious emotional disturbance

    Arch Gen Psychiatry

    (1996)
  • EMZ Farmer et al.

    Reliability of self-reported service use: test-retest consistency of children's responses to the Child and Adolescent Services Assessment (CASA)

    J Child Fam Studies

    (1994)
  • Cited by (54)

    • Reasons Why Children and Adolescents With Attention-Deficit/Hyperactivity Disorder Stop and Restart Taking Medicine

      2018, Academic Pediatrics
      Citation Excerpt :

      For the 2-, 3-, 6-, 8-, and 10-year assessment time points, medication use was based on parent responses to the Services for Children and Adolescents—Interview. This is a reliable measure of ADHD service use, including medication use.26 For the 12-year assessment time point, medication use was based on participant responses to a single question: “Are you currently taking ADHD medication?”

    • Adolescent substance use in the multimodal treatment study of attention-deficit/hyperactivity disorder (ADHD) (MTA) as a function of childhood ADHD, random assignment to childhood treatments, and subsequent medication

      2013, Journal of the American Academy of Child and Adolescent Psychiatry
      Citation Excerpt :

      Secondary analyses explored alcohol and marijuana/other drug use disorders separately. In addition to original randomized assignment to treatment, two medication treatment variables were calculated from responses to the Services for Children and Adolescents–Parent Interview,57,58 administered at all assessments to collect a cumulative history of medication treatment since enrollment into the study. Detailed information about the children’s psychoactive medication over the follow-up period has been previously reported.47

    View all citing articles on Scopus

    The MTA (Multimodal Treatment Study of Children With Attention Deficit Hyperactivity Disorder) is a cooperative treatment study performed by six independent research teams in collaboration with the National Institute of Mental Health, Rockville, MD, and the Office of Special Education Programs of the U.S. Department of Education, and the Office of Juvenile Justice and Delinquency Prevention, Department of Justice.

    The SCAPI was developed through a cooperative agreement funded by the National Institute of Mental Health. It cannot be copyrighted and is in the public domain. It can be obtained online at www.jaacap.com via the Journal's Article Plus feature, where the SCAPI interview and supplementary materials are available.

    Article Plus (online only) materials for this article appear on the Journal's Web site: www.jaacap.com.

    Disclosure: Dr. Jensen has research grants from Janssen Pharmaceutical and McNeil Pharmaceuticals; is a consultant/advisor to Shire, Celltech, Janssen, McNeil, Pfizer, Novartis, and Lilly; and is on the Speakers’ Bureau of Phase V, CME Outfitters, and REACH. Dr. Arnold has research funding from Celgene, Shire, Lilly, Noven, Sigma Tau, Targacept, and Novartis; is a consultant/advisor to Shire, Sigma Tau, and Noven; and is on the Speakers’ Bureau of Shire, McNeil, Lilly, and Novartis.

    View full text