Linking process indicators and clinical/safety outcomes to assess the effectiveness of abatacept (ORENCIA) patient alert cards in patients with rheumatoid arthritis

Patient alert cards (PACs) for abatacept (ORENCIA) inform patients and healthcare professionals (HCPs) about the risk of infections and allergic reactions. The study evaluates the effectiveness of the PACs in rheumatoid arthritis patients and HCPs, using process indicators (awareness, receipt, utility, knowledge, behaviour) and outcomes.


| Study design
The study consisted of three sub-studies: • Two cross-sectional surveys of HCPs and patients to assess process indicators (awareness, receipt, utility, utilisation, knowledge and behaviour related to the PACs) and • A retrospective chart review of patient data to assess clinical and safety outcomes.
The study was conducted in France, Germany, Spain, Sweden and the UK.
The details of the study design are shown in Figure 1. Figure 1A shows the scenarios considered valid for inclusion in the patients substudies. In the retrospective chart review sub-study, follow-up data were collected for 2 years before the date of informed consent, or less if abatacept was commenced less than 2 years from the date of informed consent ( Figure 1B).
Key messages contained in the PACs are listed in Table 1.
The study, classified as a Post-Authorisation Safety Study (PASS), was conducted according to GVP Modules XVI 1 and VIII 2 and best practices based on guidelines 3-5 and publications. 6-14

| Study population and sampling
Only patients treated with abatacept for RA in the 3 months preceding the date of completing the questionnaire were included. After informed consent patients received the survey questionnaire. Data were extracted from the clinical medical records of patients who had completed the survey questionnaire.
Physicians were selected using random sampling applied to lists of rheumatologists provided by the MAH and panels ( Figure 1C).
Nurses were identified through physicians. Physicians and nurses who recruited patients were not eligible for the HCP survey. 'Fair market' fees were paid to HCPs for completed questionnaires.
Patients and HCPs were identified by a unique code in the OXON validated electronic data capture (EDC) platform, allowing linkage of patients between the survey and the retrospective chart review.

| Data collection tools
HCP and patient questionnaires were translated into local languages, and cognitively pretested by HCPs and patients. The questionnaires consisted of multiple-choice questions including conditional branching based on responses to previous questions to improve user friendliness and reduce missing data. Patient questionnaires were completed online or on paper. HCP questionnaires were only available online.
For the retrospective chart review, HCPs or data managers were trained to extract information from patient's clinical medical records and enter it into the EDC platform. The EDC platform guided data extraction and contained edit checks and management of queries to improve data quality.

| Sample size
The study aimed to recruit 400 patients to allow precision of less than ±5% around plausible estimates (ranging from 50% to 90%) of correct responses for process indicators related to the PACs. This number would also permit the detection of a moderate decrease in the risk of infection-related hospitalisations (odds ratio < 0.23) between patients with ≥80% correct responses vs <80% correct response. A target of 80 HCPs would allow precision ±6.6% to 11.0% around plausible estimates (70%-90%) of correct responses for HCP process indicators.

KEY POINTS
• Surveys (patients; healthcare professionals) and a retrospective chart review assessed the effectiveness of abatacept patient alert cards (PACs).
• Knowledge of risk of infection was higher among patients and healthcare professionals who had received the PAC versus those who had not.
• The safety outcome of infections leading to hospitalisation increased numerically with decreasing patient survey results.
• Results supported the effectiveness of abatacept PACs.
• This hybrid design linked process indicators with outcomes within the samepatients.
F I G U R E 1 Design features of the three related sub-studies. A, Patient scenarios valid for inclusion in the study. Horizontal arrows represent abatacept treatment periods for specific patients. Patients who received abatacept within the 3 months prior to the date of questionnaire completion were eligible for participation in the study. B, Retrospective data collection. Follow-up data were collected for the 2 years prior to the date of informed consent, or less if abatacept was commenced less than 2 years from the date of informed consent. C, Sampling and recruitment strategy 2.4 | Study endpoints and data analysis

| HCP and patient surveys
HCPs and patients with questionnaires received before database closure were considered as 'completers'. Participation rates were defined as the number of participants with completed questionnaires among the number invited in each category or among the number eligible. 3,4 The definition of study endpoints and scores is provided in In the patient survey, the utility score was computed as 100 times the sum score of items assessing clarity, conciseness, completeness and brievity (recoded as 0 = worst value to 4 = best value) and understandability (ranging from 0 to 4), divided by the maximum possible score of 20. In the HCP survey, the utility score was computed as 100 times the sum score of items assessing helpfulness, clarity, conciseness, completeness and brevity (recoded as 0 = worst value to 4 = best value), divided by the maximum possible score of 20.

Knowledge
• Proportion of HCPs/patients with correct responses to questions related to the important identified risks of infections and allergic reactions associated with abatacept treatment • Overall knowledge score, computed as the percentage of correct responses to variables that assessed knowledge, that is, 100 times the sum of all variables, divided by the number of variables.

Behaviour
• Proportion of HCPs/patients with correct responses to behavioural questions • Overall behavioural score, computed as the percentage of correct responses to variables that assessed behaviour, that is, 100 times the sum of all variables, divided by the number of variables.
Global score An average score that summarises the overall correct utilisation of the PAC, correct knowledge, correct behaviour and utility. This score is presented in three categories by tertiles: high level (scores ≥67%), medium level (scores 34%-67%) and low level (scores 0%-33%).
Correlation of global score of the patient survey with: Percentage of patients with: • Results of any test to screen for TB prior to administration of abatacept therapy

| Retrospective chart review
The extracted outcomes included (  Figure 2 and patient characteristics in Table 3.
From those with information, 59.7% ( Levels of correct knowledge in patients are shown in Figure 3. The mean knowledge score (across all knowledge questions) was higher among those who recalled receiving the PAC compared with those who did not (64.5% vs 36.9%; P < .001). As shown in Figure 4, knowledge about the risk of infections was higher among

| HCP survey
Only 320 HCPs were evaluable among 2385 HCPs invited to participate. From these 320 HCPs, only 107 were eligible and 79 (50 physicians and 29 nurses) completed the questionnaire. The percentage of completers/evaluable invitees was 24.7% (79/320) and that of completers/eligible was 73.8% (79/107). The disposition of HCPs in the study is shown in Figure 5 and characteristics of the 79 HCPs in Table 5. Knowledge about the risk of infections was 84.8% (67/79) ( Figure 6). The recommendations of pre-screening for TB and VH were known by 84.8% and 73.4% of HCPs, respectively ( Figure 6). The mean global score was higher among HCPs who reported having received the PAC compared with those who did not (68.8% vs 33.1%; P < .001).

| DISCUSSION
The study employed a hybrid design to evaluate the effectiveness of the abatacept PACs using both process indicators and clinical/safety outcomes in the same patients, and is the first published study of its type of which we are aware.
Patients were recruited via physicians and the lower than anticipated recruitment of patients appeared to be due low use of abatacept in the study centres. The study has some limitations. While the planned number of HCPs was recruited, it involved a significant recruitment effort requiring invitations to 2385 potential HCPs. How many of the 2029 nonresponders, despite several reminders, were not eligible is unknown but we do know that some two-thirds of all responders were not eligible. Thus, the generalisability of the results has some uncertainty.
Nonetheless, low response rates in surveys involving HCPs is a wellknown limitation and has been previously acknowledged in other studies of this kind. [17][18][19] While differences in results in those who receive and do not receive the PAC is the strongest evidence from the surveys alone, they may not be unconfounded comparisons. Thus, results need to be cautiously interpreted. There is potential bias in extrapolating current understanding and implementation in patient surveys with past events in this retrospective study. The scores were created to assess the impact of the PAC in the target populations, including those who reported being aware and having received the PAC vs never having received the PAC. Furthermore, it is possible that there may have been some information bias, particularly since behaviour was self-

| CONCLUSION
The study assessed the effectiveness of the abatacept PACs among patients and HCPs in five European countries, using process indicators and outcomes. This novel study design bridges the gap of linking process indicators with outcomes at the individual-patient level and strengthens the clinical relevance of results from surveys.
The results support the effectiveness of the abatacept PACs.
The study results suggest that the distribution of the PAC to patients could be improved, despite already being included inside the abatacept product packaging and that nurses should be the main target of any strengthened distribution efforts, where feasible.

ETHICS STATEMENT
The protocol was approved by Ethics Committees in all five participating countries.