Measuring child functioning: assessing correlation and agreement between caregiver and child responses at the Iganga-Mayuge Health and Demographic Surveillance Site in Uganda

Introduction: Empirical data is scare on assessment of concordance between caregiver-child responses on child functioning. Objective: To assess correlation and agreement between children (11–17 years old) and their caregivers’ responses to the UNICEF/Washington Group Child Functioning Module (CFM) at the Iganga-Mayuge Health and Demographic Surveillance Site (IM-HDSS) in Uganda. Methods: CFM with 24-questions corresponding to 13 domains of functioning was administered to children between 11–17 years of age and their caregivers. Descriptive analyses of the child/ caregiver responses were conducted. Correlation and agreement between caregiver and child responses were assessed. Results: Of the 217 caregiver/child pairs eligible for this study, 181 pairs agreed to participate (83.4%). The mean age of children was 13.9 ± 1.9 years, and 56.4% were males. Cronbach’s alpha was 0.892 and 0.886 for the caregiver and child versions of CFM respectively, showing good internal consistency in both. There was a significant overall agreement between mean score of caregiver (5.36±5.63 out of 39) and child (5.45±5.34) pairs. Spearman’s rank correlation between the pairs was 0.806 (strong positive correlation). Bland-Altman plots for CFM scores showed greater agreement between caregiver and child at lower scores. Percentage agreement between the pairs for overall disability was greater for mild (83.53%) and moderate (79.37%) categories as compared to the severe (66.67%) category. There was substantial agreement (kappa 0.623) for overall disability between the pairs. Conclusion: This study indicates that there is significant correlation and agreement between self-reported caregiver-child pair responses, opening the way for considering children as CFM respondents, when possible.


INTRODUCTION
The United Nations Children's Fund (UNICEF) and the Washington Group on Disability Statistics (WG) have collaboratively developed a set of questions to assess disability in children based on the International Classification of Functioning, Disability and Health for Children and Youth (ICF-CY) framework [1,2]. They developed two tools -one for children between 2-4 years and another for those between 5-17 years. These tools cover 16 questions over 8 core domains of functioning for children 2-4 years of age and 24 questions over 13 core domains of functioning for children 5-17 [3][4][5].
The UNICEF/Washington Group Child Functioning Module (CFM) is designed to be administered to primary caregivers of children, and this is considered an appropriate approach for younger children. However, in cases of older children, it is important to get their own input on perceptions of functioning. This is because the perceptions of children and their parents/caregivers may differ with respect to the child's abilities and disability [6]. In order to reduce any bias introduced through interviewing parents/caregivers alone, it may also be possible to interview children, when appropriate. Not only does this approach allow children to self-report their limitation, but it also validates caregiver responses as has been shown in various studies on pediatric mental health, non-communicable diseases like asthma, and mobility [4,[7][8][9] Although there is no official child version of the CFM, UNICEF and WG have conducted cognitive testing in the US to assess differences in caregiver and child responses. It showed that the level of agreement between caregiver and child was highest for physical domains, like vision and hearing, probably because these are easy to observe by parents/caregivers. The agreement was lowest for learning, concentration, accepting change, making friends, anxiety, and depression, probably because these cannot be observed directly [6].
Since the pilot study by UNICEF/WG was done in the US only, there is lack of empirical data on how the child version of the CFM would work in other settings, especially LMICs such as Uganda. To address this gap, the child version of the CFM was implemented at the Iganga-Mayuge Health and Demographic Surveillance Site (IM-HDSS). The aim of this study was to determine correlation and agreement of CFM between responses of children (11-17 years old) and their caregivers at the IM-HDSS in Uganda. More specifically, this study explores overall and domain-level agreement for caregiver-child pairs. It further assesses caregiver-child pair agreement based on specific characteristics: child sex, child age group, child school enrollment status and household wealth quintile.

Study site
IM-HDSS is located in eastern Uganda [10], and follows over 89,000 individuals living in about 18,000 households of whom 18,000 are between 11-17 years of age. IM-HDSS conducts census-level data collection two times a year on births, deaths, pregnancies and their outcomes, and in-and out-migrations [10]. In addition, it also periodically collects data on access to health services, causes of death, relevant socioeconomic and education data, non-communicable diseases and injuries [11]. Between 2005 and June 2019, 21 rounds of data collection were completed.
This study was nested within an ongoing parent study to pilot electronic data collection for injuries and disability in IM-HDSS which were implemented in paper format during a previous study conducted at IM-HDSS between 2008 -2009 and subsequently integrated in three rounds [11,12]. A pilot using tablet-based data collection was conducted in round 19 (April -June 2017); and this was used as sampling frame (see sampling frame section below) for the current study on child disability.

Study design and respondents
This was a cross-sectional study conducted between September 2018 -January 2019.
Respondents were children between 11 to 17 years of age and their caregivers; an effort was made to administer CFM separately to the child and caregiver during the same visit. However, caregivers did not sit with the child during the interviews. In instances when the child was not available, three attempts were made to interview the child. All child interviews were conducted after completion of caregiver interviews, and after receiving oral parental permission and child assent, which was recorded during tablet-based data collection.

Sampling frame and sample size
Modified Washington Group short set (mWG-SS) had been previously administered in three rounds at IM-HDSS to collect data on disabilities on all individuals five year an older. This data was collected from head of the household. mWG-SS included a total of six questions on vision, hearing, walking, upper body mobility, self-care and communication. Responses were recorded on a 4-level Likert scale (0= no difficulty, 1=some difficulty, 2=a lot of difficulty and 3=cannot do at all). Scores range from 0 -18 such that the higher the score, the greater the difficulty [11,13]. Data available from mWG-SS on children between 5-17 was not previously analyzed, however, it provided basis for sampling frame for this study and helped to draw household and individual listings available from the latest IM-HDSS rounds (rounds 19 and 20). A total of 377 children with disabilities between the ages of 5-17 were identified in round 19 (April -June 2019) ( Figure 1).
The IM-HDSS data management team confirmed active status of their IDs in round 20, which had been completed four months (May 2018) before the beginning of this study (September 2018). Based on the data check, 342 children out of 377 from round 19 were found to have active IDs in round 20; 29 children were more than 17 years, one had died, four had moved to another location within IM-HDSS, and one had moved out of IM-HDSS. Active IDs mean that these children were present at the IM-HDSS site as of round 20. Caregivers of these 342 children with disabilities were approached for participation in this study; and caregivers of 308 children agreed to participate in the study. However, only 217 of the 308 children were between the ages of 11-17 years and were eligible for this study; and only one child per household was selected ( Figure. 1). Thus, of the 217 caregiver-child pairs that were eligible for participation in this study, 181 pairs agreed to participate (83.4%).

Study tool: UNICEF/Washington Group Child Functioning Module (CFM)
This study utilizes the CFM with the primary respondent being caregivers of children [1]. For 5-17-year olds it comprised 24-questions with responses on 4-level Likert scale (0=no difficulty, 1=some difficulty, 2=a lot of difficulty and 3=cannot do at all). These questions covered 13 domains: vision, hearing, walking, self-care, communication, learning, remembering, concentrating, accepting change, controlling behavior, making friends, anxiety, and depression. The total score range was 0 -39, and the higher the score, the greater the disability. It takes about 20-25 minutes to complete CFM [2]. Caregiver was defined as an individual who spends most time with the identified child and is the main person responsible for taking care of their needs (food, care) within the house.
The child response version of CFM caters to children between 11-17 years of age [6]. The CFM tool was translated and back-translated to Lusoga using a standardized approach [14]. The language was kept at the level of understanding of children (grade 5) and was revised based on input from a pre-test and field staff. Details of CFM adaptation and validation are available elsewhere [15].

Data collection and management
Parental or caregiver oral consent and child's assent were taken before face-to-face interviews were conducted using two separate tablet-based versions (caregiver and child) of the CFM. The tablets included English and Lusoga versions of questions, which were developed in Microsoft Excel .xls format and uploaded to KoBoToolbox (https:// www.kobotoolbox.org/) for data collection. Data from the tablets was submitted daily to a secure, encrypted cloud server with no copy left on the tablet. This ensured data confidentiality and security. Data were downloaded daily from the server in MS Excel (.xls and .cvs format). Using unique study IDs, caregiver and child data were merged at the end of data collection.

Data analysis
Descriptive analyses were conducted to depict the demographic characteristics of children and their caregivers. Binary and categorical variables are reported in percentages and mean with standard deviation as well as median and interquartile range (IQR) are reported for continuous variables. The internal consistency of caregiver and child versions of the CFM was calculated using Cronbach's alpha. Cronbach's alpha of ≥0.9 is excellent, ≥0.8 is good, ≥0.7 is acceptable, ≥0.6 is questionable, ≥0.5 is poor and <0.5 is unacceptable [16].
Disability has a skewed distribution with less people with sever disability [17], this was true for data from this study, thus non-parametric test were used for assessing concordance between caregiver and child pairs.
Tests performed using CFM scores-Total raw scores based on Likert scale responses of CFM were calculated to assess individual scores and their distribution. The differences between paired caregiver and child scores were also calculated. Total disability score was categorized into four mutually exclusive categories: no disability, mild disability, moderate disability and severe disability. No disability was defined if "none" was marked for all 13 domains. Mild disability was defined if "some difficulty" was marked as the highest response for any one of the 13 domains, moderate disability for "a lot of difficulty" as the highest response for any one of the 13 domains, and severe disability for "cannot do at all" as the highest response on any one of the 13 domains.
The Wilcoxon signed-rank test is a non-parametric test and was conducted to test the null hypothesis that there is no difference in the distribution between paired caregiver and child scores [18]. It does not assess relation between responses of child-caregiver pairs; therefore, association between caregiver and child scores was assessed using Spearman's rank correlation coefficient. Its value ranges between −1 to +1, and values are interpreted as "very weak" for values between 0.00-.19, "weak" for 0.20-0.39, "moderate" for 0.40-0.59, "strong" for 0.60-0.79, and "very strong" for 0.80-1.0 [19]. The Bland-Altman plot for difference between caregiver and child scores, and average of caregiver and child scores was developed to evaluate agreement between caregiver and child scores and to assess any overreporting or under-reporting of functional disability by either caregivers or children [20]. This test was performed because correlation between scores of child-caregiver pairs does not specify agreement.
Tests performed using Likert-scale responses-In addition to assessing correlation and agreement based on caregiver and child scores, overall and domain-specific percent agreement and kappa statistic between caregiver and child responses were also calculated. Percent agreement was calculated as the number of pairs with caregiver and child pair agreement divided by the total number of caregiver and child pairs [21]. Cohen's Kappa was also calculated and is reported when the outcome variable is categorical and is preferred when the respondent pair is selected purposively (as was the case in this study) [21,22]. The advantage of Cohen's Kappa over percentage agreement is that it accounts for chance agreement between raters. It varies from −1 to 1, where 0 reflects chance agreement and 1 reflects perfect agreement. Values between 0.1 -0.20 show slight agreement, 0.21 -0.40 fair agreement, 0.41 -0.60 moderate agreement, 0.61 -0.80 substantial agreement, and 0.81-0.99 is near perfect agreement.
All analyses were conducted in STATA 14.

Descriptive analysis
The mean age of children was 13.9 ± 1. 9 years; 56.4% were males. Over 80% had a primary caregiver, and about half of the children lived in a nuclear family system. Mothers were the primary caregivers for 60% of the children. (Supplementary table 1 and 2). Comparison of disability categories based on respondents showed that children reported higher percentage of mild (child: 51.4% and caregiver: 47.0%) and moderate (child: 38.1% and caregiver: 34.8%) disability, whereas the percentage reported for severe disability was 8.3% by children and 11.6% by caregivers (Supplementary figure 1).
Cronbach's alpha was 0.892 and 0.886 for the total disability scores for the caregiver and child versions of CFM respectively, showing good internal consistency.

Tests performed using CFM scores
The overall mean score was reported to be 5.36 ± 5.63 out of 39 for caregivers and 5.45 ± 5.34 for children, showing significant overall agreement between scores within caregiver and child pairs. Interestingly, median scores reported by children (Median: 14; IQR: 7-23) were lower for severe disability compared to those reported by caregivers (Median: 18; IQR: 8-23). (Table 1).
Caregiver and child scores were also compared based on child sex, age group, school enrollment and household wealth quintile. Means and medians of caregiver and child scores did not vary based on child sex, age group, school enrollment and household wealth quintile and showed similar trend. Higher scores were observed for 15-17-year olds compared to 11-14-year olds. Scores were also higher for children not enrolled in schools. Caregiver and child pairs in the least poor quintile had higher scores compared to other quintiles. (Supplementary table 3). The overlap between caregiver and child scores showed that there was no statistical difference between the child and caregiver pair scores.
Wilcoxon signed-rank test was not statistically significant and failed to reject the null hypothesis that the median of difference equals zero (p-value 0.969). Spearman's rank correlation between caregiver and child scores was 0.806 (p-value <0.005), showing very strong correlation between scores. (Figure 2). Bland-Altman plot for CFM scores showed greater agreement between caregiver and child at lower scores; in addition, the majority of the agreement was within the 95% limits ( Figure 3).

Tests performed using Likert-scale responses
Overall percentage agreement between caregivers and children was 76.80%. Percentage agreement between caregiver and child for overall disability severity categories showed that the percentage of agreement was greater for mild (83.53%) and moderate (79.37%) categories compared to the severe (66.67%) category. (Table 2).
Domain-specific percentage agreement showed that agreement was greater than 90% for severe disability in the domains of hearing, self-care, learning, and making friends, and over 80% for walking, communication, and depression. However, percentage agreement was less than 70% for anxiety. This was observed for overall agreement for females, for 15-17-year olds, for children in school and for children belonging to the "less poor" and "least poor" wealth quintile (Supplementary table 4).
Likert scale disability severity categories for caregivers and children were also assessed based on child sex, child age group, school enrollment and wealth quintiles and showed that caregiver and child responses were each statistically associated with school enrollment, with a greater percentage of severe disability reported for children not enrolled in school (Supplementary figure 2).
There was substantial agreement (kappa 0.623) between caregiver and child responses for the overall disability scores. The kappa statistic was substantial for all domains except for controlling behavior (0.533), anxiety (0.542) and depression (0.593). The kappa statistic based on child sex, age group, school enrollment and wealth quintile were moderate to substantial except for the poor quintile (kappa 0.833), which was near perfect. For specific domains, the kappa statistic for child sex, age group, school enrollment and wealth quintile were in the same range as the overall agreement. However, there were variations based on wealth quintiles. For example, for hearing, the agreement was substantial across all characteristics except for the poor, which was near perfect (kappa 0.826), and less poor, which was moderate (kappa 0.525). For concentration, the agreement was substantial across all characteristics except for the less poor, which was near perfect (kappa 0.919), and least poor, which was moderate (kappa 0.402). For anxiety, the agreement was moderate to substantial across all characteristics except for the less poor (kappa 0.396) and least poor (kappa 0.333) quintiles, which were fair. (Supplementary table 4).

DISCUSSION
To the best of our knowledge, this is the first disability study conducted at the IM-HDSS in which children between 11-17 years of age were interviewed in addition to their caregivers. It assessed responses of caregiver-child pairs for child's level of functioning. This study explored both total scores and Likert categories for caregiver and child responses using multiple statistical tests to assess correlation and agreement between caregiver-child pair responses. Correlation was assessed to determine the association between child and caregiver responses --if child score goes up, caregiver score goes up as well. However, this does not assess level of agreement between responses. There is a possibility that child and caregiver scores are correlated but may also have a large difference in their level of agreement. In such instances, it is recommended to assess both correlation and agreement between scores [9]. Based on these approaches, it was found that the responses did not differ significantly between children and their caregivers. This means that community-based assessment of child functioning and disability could be done by interviewing either caregivers or children with disability. However, disability assessment needs to be contextualized, and it is important to understand relevant caregiver (education, occupation), household (wealth quintile, income) and social (family structure) factors that may impact a child's disability. Given that caregivers are more informed about such factors and the impact these have on their child, family and household, it is probably important to interview caregivers in addition to child interviews to obtain a better understanding of contextual factors [23].
The agreement between caregiver and children was more for mild and moderate disability compared to severe disability. This could reflect that children with severe disability adapt to their limitation and are able to cope with functional limitation. Hence, their perception of disability might be less severe compared to the perception of their caregivers. This may also imply the underlying issue related to how caregivers perceive extent of disability in their children. Therefore, it is important to consider both child and caregiver perspectives to get a better understanding of level of disability among children [24].
In addition, it was also interesting to note that agreement for domains like anxiety was less compared to other physical domains like vision and hearing. This was also observed by Massey et al [6]. First it could be related to understanding of words anxiety and depression. In most Western countries, these words are colloquially used for clinical diagnosis of these conditions, however, in many LMICs, these words are more often used to express underlying mood of an individual for example feeling anxious before starting school [25]. This area needs further exploration through administration of clinical assessments for anxiety and depression and qualitative assessment to understand mental health well-being of these children. Second, the expression of being anxious or depressed could also be associated with other types of disabilities that a child may have, or it could be an expression of parental anxiety and depression associated with child's disability and its impact on the parents. These are important areas to explore in pediatric mental health and well-being and need further exploration in this population.
The distribution of disability scores based on selected child characteristics showed that scores did not vary based on the child's sex. This observation is specific to IM-HDSS, and it may differ if the same study is conducted in other parts of Uganda or other LMICs. Reporting any physical limitation is considered a taboo in many parts of the world, especially when the child is a girl [26]. Social norms and culture have a major influence on how "disability" is perceived and acknowledged in many parts of the world, especially LMICs where individuals with disability are isolated from society [26]. The CFM is thus designed in a way to avoid using the word "disability" in its questions; it focuses more on assessing limitation rather than labeling children as disabled [5].
Higher scores were also reported for children not enrolled in school. This was consistent across caregiver and child responses; and possibly indicates that the severity of the level of disability hampers enrollment of these children into school. Potential reasons may include lack of accommodations (trained teachers, books in braille, ramps etc.) in schools [27], or attitude of teachers and peers, which act as a barrier to integration of students with disabilities into the school environment [27]. In addition, this study did not collect data from teachers and school administration to assess the factors associated with lack of enrollment among children with high disability scores. This is an important area of future work and needs to be considered to ensure a holistic approach to addressing both disability in children and social inclusion. Children with physical disability can receive education in mainstream schools, and the CFM has been shown to provide useful information regarding disability severity [28].
Disability is considered a taboo, and until recently, many caregivers had not registered their disabled children with the IM-HDSS staff. However, over the past few years, IM-HDSS has worked to collect routine data on these children. Combined with the CFM tool, the core IM-HDSS data has the potential to identify needs of these children and guide the local district health office to develop and implement strategies to improve the health and well-being of children with disabilities. The administration of CFM did not require field staff to have medical and clinical knowledge and background. This supports the broad application of CFM in communities, especially in LMICs where standardized methods to assess disability are lacking and thus lead to data scarcity. Translation of the CFM (child version) did not require any changes in the questions and was at the level of understanding of children 11-17 years of age. This is reassuring as it helped in implementation of CFM without major changes in questions and will help in making comparisons with different contexts in future studies on disabilities.
One of the limitations of the study is that the results may not generalize to the whole of Uganda and other LMICs. Disability needs to be contextualized based on individual, family, household and social factors. These findings may vary even if the study is conducted again at the IM-HDSS because disability is a dynamic process and perception of disability among caregiver-child pairs may change as the child learns to cope with their disability or recovers from a disabling event. Second, the major issue with child interviews was the nonavailability of children during daytime and weekdays. The team approached these children during evenings and weekends and made up to three attempts to complete child interviews. Third, interviews with children have their own ethical issues since the issue of disability may cause anxiety among children. This potential limitation was countered by the fact that the CFM questions do not mention the word "disability"; instead, the questions assess limitations for the 13 domains, which are less likely to cause anxiety in children. Fourth, this study did not have a follow-up using clinical assessment method to verify the information shared by caregiver-child pairs. However, significant agreement on most domains show that the self-reported limitations by caregivers and children were similar. Fifth, selected characteristics were explored in this study; however, due to time constraints, lived experiences of caregiver-child pairs were not recorded to explore various aspects of disability and their impact on their daily living.
It is important to understand multiple perspectives to assess child functioning One is the insider perspective, which means how a child perceives their level of functioning. This is based on their own experience of living with disability. Second is the outsider perspective, which means how caregivers perceive their child's disability and level of functioning and is based on household and social factors [24]. It is important to give children the opportunity to report their functioning; this helps to empower children as reporters of their health and wellbeing. Their understanding of their health can form the basis of community-based interventions, giving these children the voice to help them integrate into their society.

CONCLUSION
This study shows significant agreement in self-reported caregiver-child pair disability score using the CFM tool. This means that children can report their limitations using a standardized tool like CFM. However, in order to contextualize disability within a child's environment, it is recommended to also interview caregivers, as they can provide more information on family and household-related factors of which a child may not be aware. This will provide a better and deeper understanding of disability in children.