Validation of a screening test for alcohol use, the Russian Federation

Abstract Objective To validate a Russian-language version of the World Health Organization’s Alcohol Use Disorders Identification Test (AUDIT). Methods We invited 2173 patients from 21 rural and urban primary health-care centres in nine Russian regions to participate in the study (143 declined and eight were excluded). In a standardized interview, patients who had consumed alcohol in the past 12 months provided information on their sociodemographic characteristics and completed the Russian AUDIT, the Kessler Psychological Distress Scale and the Composite International Diagnostic Interview to identify problem drinking and alcohol use disorders. We assessed the feasibility of administering the test, its internal consistency and its ability to predict hazardous drinking and alcohol use disorders in primary health care in the Russian Federation. Findings Of the 2022 patients included in the study, 1497 were current drinkers with Russian AUDIT scores. The test was internally consistent with good psychometric properties (Cronbach’s α : 0.842) and accurately predicted alcohol use disorders and other outcomes (area under the curve > 75%). A three-item short form of the test correlated well with the full instrument and had similar predictive power (area under the curve > 80%). We determined sex-specific thresholds for all outcomes, as non-specific thresholds resulted in few women being identified. Conclusion With the validated Russian AUDIT, there is no longer a barrier to introducing screening and brief interventions into primary health care in the Russian Federation to supplement successful alcohol control policies.


Introduction
In September 2018, the World Health Organization (WHO) released SAFER, a new initiative and technical package outlining five high-impact strategies that can help governments to reduce the harmful use of alcohol and related health, social and economic consequences. 1 One of these strategies is to "facilitate access to screening, brief interventions and treatment," which is enabled by the Alcohol Use Disorders Identification Test (AUDIT), a simple 10-item test. 2, 3 The test is the most successful screening instrument for assessing an individual's hazardous and harmful use of alcohol and for alcohol use disorders worldwide. The term harmful use is used as a diagnostic code (F10.1) in the International classification of diseases and related health problems, 10th revision (ICD-10) in the section: mental and behavioural disorders due to psychoactive substance use. 4 The term hazardous use is a non-diagnostic term which denotes a pattern of alcohol consumption carrying a risk of harmful consequences to the drinker. 2 Developed by WHO, the AUDIT was primarily intended for screening purposes in primary health care to identify individuals with the above-mentioned drinking patterns and potential Publication: Bulletin of the World Health Organization; Type: Research Article ID: BLT.20.273227 Page 3 of 21 alcohol dependence. Today, the AUDIT, or its short form, the AUDIT-C (three-item form), 5 is being used as the main screening instrument in these settings globally. Screening and brief interventions have become a standard part of any comprehensive alcohol policy, especially since WHO launched the SAFER initiative. 1 To implement the AUDIT in a national context, questions on how to best place it in the national treatment system -including primary and specialized care for alcohol use -need to be answered. Determining the best cut-off scores for different risk levels and the subsequent management of alcohol use disorders is crucial, and validation studies are a standard and required step in such an implementation process. 6,7 Although international studies report a wide range of AUDIT cut-off scores for different settings, most of them are not based on validation efforts, which reduces the instrument's efficiency. 8 In the Russian Federation in 2016, the WHO Regional Office for Europe and the Russian health ministry sought to include the AUDIT as part of an initiative to systematically implement screening and brief interventions. 9 However, experts involved in the initiative expressed concerns that the AUDIT might not adequately assess the drinking patterns specific to the Russian Federation and its neighbouring countries, mainly because none of the Russian versions of AUDIT had ever been validated. 10 In addition, issues were found with two terms: the so-called standard drink and single occasion of drinking (intended to assess heavy episodic drinking). [10][11][12] As a result, a Russian AUDIT needed to be revised and validated. Accordingly, we established a protocol for the process and developed a modified AUDIT for use in the Russian Federation, hereafter called the RUS-AUDIT ( Fig. 1), details of which are given elsewhere. 13  The objectives of our study were to (i) validate the RUS-AUDIT; (ii) empirically examine additional alcohol use patterns (item 3 of the AUDIT) to allow for the best identification of hazardous use patterns in the Russian context; (iii) determine the best cut-off scores for providing brief advice and interventions in primary health-care settings; and (iv) determine the best cut-off scores for potential alcohol use disorders.

Design
We chose concurrent validation as part of a cross-sectional study design to determine how well the Russian AUDIT values correlate with values of other diagnostic tests that have been validated before (see below), and which combinations could serve as measures relevant for future interventions (brief interventions or referral to the specialist treatment system).

Sample
A total of 21 primary health-care facilities from eight regions, covering seven out of eight Federal Districts of the Russian Federation, participated in the data collection for validation of our RUS-AUDIT (data repository). 14 We recruited a probability sample of 2173 participants from rural and urban facilities with at least 200 participants from each region. We collected data between August 2019 and February 2020. We established the following quotas for the subsample from each region to secure representation of all important sociodemographic groups: 3 50% males, 3 50% 40 years and older and not more than 50% recruited from a so-called dispanserization setting. Dispanserization is a term used in the Russian Federation to denote preventive activities undertaken at the population level and organized within primary health-care facilities. These activities include measures such as specialized medical examinations for the early detection of diseases and risk factors, including alcohol use. 15 The sampling frame was all patients who visited a participating primary health-care facility on the day of the interviews. After providing the patients with medical services, the treating doctors or nurses referred the patients to trained interviewers for the assessment in a separate room.

Interviewers
We trained five interview trainers, using modules we had developed and a special training manual for interviewers. These interview trainers carried out nine training sessions for interviewers between August and October 2019 in the participating regions The training sessions lasted 6-7 hours and covered the basics of screening and brief interventions for alcohol, the structure of the AUDIT, an overview of the RUS-AUDIT validation project, the basics of interviewing techniques, and a thorough overview of the instruments used (data repository). 14 The trainers used role play to simulate the interview process. At the end of the training sessions, the trainers assessed each trainee individually using a role play of specific interview situations.
Only trainees who could demonstrate an ability to administer the interviews correctly were selected as interviewers by the trainers.

Interview procedure
Participants provided verbal consent before being interviewed using a standardized form. The form included questions on demographic characteristics (age, sex, type of housing and disposable income) and drinking and smoking status. Participants who had consumed alcohol within the past 12 months were further interviewed using the modified 13-item version of the RUS-AUDIT. All participants with a RUS-AUDIT score of five and higher and thus indicating a certain risk level were then asked to complete the Kessler Psychological Distress Scale (K10) 16,17 and the alcohol use disorder module of the Composite International Diagnostic Interview (CIDI). 18 Both instruments were translated into Russian according to WHO guidelines. 19 We also asked a subsample of people with AUDIT scores lower than five (every third person) to complete these instruments (Fig. 2). 14 We also gave the Kessler instrument to every third person who had not consumed alcohol in the past 12 months to have an additional control variable, we did not give it to all such participants to save on assessment time as the Kessler scale applies to mental distress and not just alcohol use disorders. The institutions participating in the study undertook quality control procedures for the interviews in their institutions by checking each form directly after completion and checking all forms before submitting.

Statistical procedures
The first step was a descriptive statistical analysis of the study sample, defining the final analytic sample size for the RUS-AUDIT validation study and visually examining the score distribution.
Next, we examined one-dimensionality and internal consistency of the RUS-AUDIT scale by calculating the Cronbach α and other internal consistency measures. We did additional analyses to determine whether any of the three additional items on heavy drinking (11.1, 11.2 and 11.3; Fig. 1), would improve the psychometric properties. We did receiver operating characteristic analyses to test whether the RUS-AUDIT had acceptable statistical properties to predict four main outcomes: hazardous drinking, problem drinking, harmful use and alcohol dependence.
Hazardous drinking was defined as an alcohol intake of > 20 g/day for women and > 40 g/day for men, based on the drinking categories identified by WHO and the European Medicines Agency in the absence of low drinking guidelines in the Russian Federation. 20 We introduced problem drinking as an operational definition to denote the next relevant risk level as per the RUS-AUDIT scale and based on scoring on any of the following test items of the Composite International Diagnostic Interview: health problems related to drinking; objections by family or friends to drinking; collapse of relationship with loved ones due to drinking; financial difficulties due to drinking; attacked or injured someone while intoxicated; problems with police (drunk driving, accident); reduced time for important activities (work or leisure); and have had a disease (e.g. liver disease, stomach problems) or psychological problems (depression, anxiety) due to drinking. 18 These questions correspond with broader problems associated with alcohol use and some symptoms of alcohol use disorder, partly meeting the ICD-10 criteria. 4 We defined harmful alcohol use and alcohol dependence according to the relevant items of the Composite International Diagnostic Interview, following the ICD-10 classification, which is the currently used manual to diagnose alcohol use disorders in the Russian Federation.
As for the cut-offs, we selected the two outcomes linked to secondary prevention (hazardous and problem drinking) based on the Youden index, which is the difference between the proportion of true-positive and false-positive results. 21 We selected the criteria for requiring a treatment intervention (alcohol dependence alone and alcohol use disorders, which includes both harmful use and alcohol dependence) based on accuracy and specificity and in case of equal values on accuracy to avoid unnecessary costs to the health-care system and potential registration in a treatment programme.
To identify the best short version, we tested all possible combinations of three (a total of 286 combinations are possible) of the 13 RUS-AUDIT items for prediction of the full RUS-

Sample
Of 2173 people approached to join the study, 143 (6.6%) declined to participate. Of the remaining 2030 people, two did not fall within the required age range and six did not identify The distribution of the RUS-AUDIT among drinkers is shown in Fig. 3. As with level of alcohol use, 22,23 we found a typical gamma distribution with a peak on the left-hand side and a fairly long tail to the right-hand side. The overall mean score in the RUS-AUDIT was 5.76 (standard deviation, SD: 5.50), the mode was 1 and the median 4. Women had significantly lower scores (mean: 3.40; SD: 3.54) than men (mean: 7.78; SD: 6.05; P < 0.001).
Publication: Bulletin of the World Health Organization; Type: Research Article ID: BLT.20.273227 Prediction of main outcomes Table 1 shows the best thresholds and prediction characteristics of the RUS-AUDIT to predict the main outcomes (hazardous drinking, problem drinking, alcohol use disorders and alcohol dependence) necessary to set up treatment.
Overall, the RUS-AUDIT predicted all outcomes well, and was slightly more accurate for women than for men based on receiver operating characteristic analyses. The area under the curve was greater than 0.83 for all outcomes, and between 74% and 94% of the patient outcome scores were correctly predicted using the proposed algorithms of classification. Sex-specific thresholds were necessary for all outcomes: using non-specific thresholds resulted in very few women being identified with any outcome.

Best short versions
For predicting the RUS-AUDIT, the combination of items 3, 9 and 10 was the strongest. We will  25 For all outcomes, the area under the curve was greater than 0.80: for point estimates (data repository). 14 This finding means that some outcomes were even better predicted by the three-item RUS-AUDIT-S than by the full 10-item RUS-AUDIT.
Publication: Bulletin of the World Health Organization; Type: Research Article ID: BLT.20.273227 Another combination (items 1, 10 and 11) was an even better average predictor of the outcomes tested. However, as this combination included item 11 which was not selected for inclusion in the final version of the RUS-AUDIT, the combination of items 3, 9 and 10 was kept as the standard short version of the RUS-AUDIT.
As there had previously been an AUDIT-4 in use in the Russian Federation (a short version consisting of the first three and the last AUDIT test items), 26 we also tested all 715 combinations of four items for the best correlation with the outcomes (data repository). 14 The AUDIT-4 combination scored worse on all of the outcomes than the combination selected as the best average combination of the RUS-AUDIT-S (data repository). 14 Thus, if a four-item combination is to be used, the most suitable combination of RUS-AUDIT items is items 1, 3, 9 and 10. The three-item combination of the RUS-AUDIT-S, however, produced similar statistical properties for most outcomes and, because it is shorter, is preferable for use in primary health care.

Discussion
The RUS-AUDIT was internally consistent, capable of predicting hazardous drinking, problem drinking and alcohol use disorders, and feasible for use in primary care settings in the Russian Federation. Nonetheless, some potential limitations exist. Our results clearly show distinct patterns of drinking in the Russian context. The first three consumption items were less important to the total score than in other countries. 3,6 In fact, the average level of drinking a day was relatively low, and a lot of the alcohol was consumed on heavy drinking occasions, which had higher volumes of alcohol consumed per occasion than in countries with the same overall level of consumption, e.g. Germany. 29 This finding has two implications that need to be considered when implementing the RUS-AUDIT and when comparing it to other versions of the AUDIT. First, this finding means that many concepts of high-income countries in western Europe and North America that centre around average heavy drinking over time 30  The challenge of implementing the RUS-AUDIT within the health-care system will be to properly define what is possible in which parts of the primary health-care system, and under what circumstances people should be transferred to the specialized alcohol and drug treatment system. According to Russian legislation, specialized treatment is provided free of charge, but individuals are officially registered as so-called narcological patients once diagnosed and a prolonged monitoring regimen is imposed. These patients are automatically excluded from certain professions, can lose their driving licence and may experience social stigma and discrimination because of their status. Therefore, official registration is often a punitive experience and can be an important barrier to help-seeking behaviour. 33,35 Thus, mandatory registration should be re-examined in the interests of establishing a safe and high-quality screening and care system in the country. The main goal will be to create a system of health care that recognizes the importance of early intervention and prevents hazardous consumption and severe alcohol use disorders from becoming more chronic, 36 and thereby avoids the associated attributable harm. 37 A suggestion on how the system could be designed using the RUS-AUDIT and the RUS-AUDIT-S is discussed in the data repository. 14 The RUS-AUDIT is the first rigorously translated and adapted Russian version of the AUDIT and it potentially provides an additional high-impact strategy for Russian alcohol control policies, complementing the existing successful general population strategies. 33,34 Furthermore, its potential for use for Russian-speaking populations with similar drinking patterns outside of the Russian Federation should be explored.
The experience of developing and validating the RUS-AUDIT shows that translating standardized instruments such as the AUDIT into other languages may not provide a reliable tool without the introduction of modifications and prior rigorous and culture-specific research and evaluation. We hope that the documented experiences and materials developed [11][12][13] can be useful for other countries as models for validation procedures.