Evaluating Cross-Cultural Applicability of Patient Reported Outcome Measures of Integrated Care

Introduction: Patient Reported Outcome (PRO) measures of integrated care have been recognized as a pivotal requirement for understanding how to promote integrated care from the patient perspective. Cross-cultural applicability of these measures is salient to evaluate efforts to improve integrated care across different health care systems, draw comparisons and establish best practices. Yet, PRO measures suitable for this purpose remain scarce. Obtaining measures applicable across contexts requires (1) a measure that is validated for a specific context, (2) a valid translation and adaption for the new context, and (3) an evaluation of the psychometric characteristics of the translated measure following standardized evidence based procedures. Only then can investigators interpret the way cross-cultural influences impact construct formation and perceptions. We present lessons learned from evaluating the psychometric characteristics of the Patient Perception of Integrated Care (PPIC) survey, developed by Singer and colleagues at Harvard University, which was successfully transferred from the US to measure integrated care in The Netherlands.

Theory/Methods: The PPIC is a survey tool that measures integrated care from the patient perspective across six domains: Provider knowledge of patient, Staff knowledge of patient’s medical history, Specialist knowledge of patient’s medical history, Test result communication, Support for medication and home health management, and Support for self-directed care. The PPIC was translated and adapted for administration in The Netherlands, using the WHO guidelines for instrument translations. The survey was then distributed to patients of five primary care centers in the region of South Limburg, The Netherlands. We hypothesized that comparability of a translated survey and its source requires not only substantive equivalence of survey items but also similar psychometric properties. A manual for analysis of the PPIC survey composed in partnership with the US-based PPIC development team provided a detailed overview of the validation process used with the original PPIC survey and guided our data preparation (e.g. identifying and excluding ineligible cases, scoring and entering PPIC data and recoding) and psychometric analysis (e.g. analyzing demographics, non-response and testing the survey latent scale structure by means of factor analysis).

Results: A total of 5,991 surveys were distributed by mail, of which 62 were returned undeliverable because the respondent had changed address or was deceased. We received back a total of 3,734 surveys for an overall response rate of 63%. Of respondents, 85% were 55 years or older, 52.4% were female, 34% had general secondary education or primary vocational education. Respondents were predominantly Dutch (93.6%). Most (81.8%) of the respondents had moderate to good health, and 81.7% completed the survey without help. An exploratory factor analysis produced a six-factor model that was largely consistent with the factor structure of the US PPIC. Of 21 items that were included in the analysis (others were omitted due to low covariance coverage), 19 of the items in the Dutch survey combined under the same factors as the US survey. Reliability estimates met or were close to the acceptable threshold of α > 0.70. Only one factor had a lower alpha of 0.438. Goodness-of-fit measures were supportive for of the six-factor model (CFI .924, TLI .908, RMSEA 0.038).

Discussion: By standardizing the evaluation of the psychometric properties of the Dutch and US PPIC surveys, we demonstrated the validity and reliability of the PPIC survey for assessing integrated patient care across US and Dutch contexts and cultures. The manual for analysis of the PPIC survey proved an important tool for achieving equivalence in our evaluation methods and establishing reliably that the integrated care domains of the PPIC survey are as applicable for Dutch patients as they are for patients in the US. The extremely well matched factor structures across settings support this interpretation.

Conclusion: Our results highlight the importance of standardizing evaluation procedures for patient reported outcome measures that assess integrated patient care. Only by eliminating deviation in methods can we reliably assess comparability between the translated survey and the original survey. Based on our analysis, the PPIC survey appears to be a promising instrument for assessing integrated care from the patient perspective not only within the US but also across systems and cultures like The Netherlands. The generalizability of our findings may, however, be limited by the relative similarity between the US and Dutch cultures. Future research should investigate the impact of standardization on PROs-evaluation in countries with greater culture differences to establish cross-cultural transferability of integrated care measures and comparability of outcomes.