Accuracy of Alcohol Use Disorders Identification Test (AUDIT) for detecting problem drinking in 18-35 year-olds in England

The accuracy of screening tests for identifying problem drinkers in primary care settings has recently been evaluated in an International Cochrane Collaboration systematic review and meta-analysis (Smith et al., 2014). This review found that AUDIT (Babor et al., 2001) had good accuracy for identifying alcohol abuse or dependence, and that the short-form AUDIT-C had good accuracy for identification of hazardous drinking. But this Cochrane review did not include any studies for a UK primary care population that focused on young adults aged 18-35 years or that included women.

In this accuracy study we:

1. Assessed the accuracy of the AUDIT and AUDIT-C for the detection of hazardous drinking, alcohol abuse (DSM-IV), alcohol dependence (DSM-IV) and alcohol use disorders (DSM-5) with young adults in a UK primary care population using ROC analysis and suggested optimal cut-point threshold scores
2. Calculated probability estimates for each discrete test score using Bayes’ Theorem (Foxcroft et al., 2009).

This was a method comparison study, with 14 primary care practices in the Thames Valley area of England. Using a systematic sampling protocol, a selection of 14,480 adults aged 18-35 years were sent a 30-item General Lifestyle Questionnaire (GLQ) that included the 10-item AUDIT, to complete and return. All patients, regardless of AUDIT score, were also invited to participate in a telephone interview following return of the questionnaire.

Telephone interviews were conducted within 14 days by trained researchers who were blind to AUDIT responses and score, using (a) Time-Line Follow-Back (TLFB) to ascertain quantity and frequency of alcohol consumption in the previous 90 days (Sobell et al., 1988; Sobell et al., 1992), and (b) World Mental Health Composite International Diagnostic Interview (WMH-CIDI) to assess alcohol abuse, alcohol dependence and alcohol use disorders (Robins et al., 1988).

Quantity of alcohol ascertained via TLFB, was standardized into UK units. Hazardous drinking was defined as exceeding recommended drinking levels: 14 (women) or 21 (men) units of alcohol in any one week; or 2 (women) or 3 (men) units a day for five days in any one week (Royal College of Physicians, 2011). Alcohol abuse and alcohol dependence variables using DSM-IV criteria (American Psychiatric Association, 1994) were created from WMH-CIDI data using algorithms provided by the WHO WMH-CIDI Centre at Harvard University. The new DSM-5 (American Psychiatric Association, 2013) alcohol use disorders (AUD) variable was created from WMH-CIDI data by the authors (code available on request).

ROC curves, sensitivity, specificity, positive and negative predictive values and positive likelihood ratios were calculated, along with 95% confidence intervals. An unweighted and two weighted Youden J Index scores were also calculated to indicate potential optimal threshold (cut-point) test scores. We used Bayes Theorem to calculate post-test probability by reference test, index test and gender.

Of the 14,480 patients invited to participate in the study, 1,022 (7.1%) patients consented by returning the GLQ. Of these, 626 (61.3%) also consented to be interviewed. We completed 420 (138 men and 282 women) telephone interviews within our target timeframe of two-weeks following return of the GLQ. Comparing our achieved sample (N=420) with Lower Layer Super Output Area Index of Multiple Deprivation (IMD) quintiles for England (2007), most respondents (53%) came from the lowest deprivation quintile; only 10% were from the two highest deprivation quintiles. The majority were white (86%), and 25% were aged 18-24, 32% aged 25-29 and 43% aged 30-35.

Using TLFB reference standard data, 49% (67) men and 51% (144) women were classified as hazardous drinkers. Using WMH-CIDI reference standard data, 36% (49) men and 19% (53) women were classified positive for DSM-IV alcohol abuse, 13% (18) men and 8.5% (24) women were classified positive for DSM-IV alcohol dependence, and 52% (72) men and 40% (112) women were classified positive for DSM-5 AUD (none vs. mild/moderate/severe).

Table 1 presents the area under the curve (AUC) for the AUDIT and AUDIT-C tests for hazardous drinking in men and women. The AUCs with respective 95% CIs indicate that both tests have good or very good accuracy for the respective reference standard with no evidence of a difference between the two tests for hazardous drinking. The AUC for alcohol abuse, alcohol dependence and AUD, with 95% CIs, is also shown in the Table, and indicates that AUDIT is a good or very good accuracy test for dependence and AUD, but less so for abuse (sufficient accuracy).

Optimal cut-points for identification of hazardous drinking using AUDIT were nine and four for men and women, respectively. The optimum cut-point decreased to five and two when weighting favoured sensitivity, and increased to eleven and seven when weighting favoured specificity, for men and women, respectively.

Optimal cut-points for identification of DSM-IV alcohol abuse using AUDIT were ten and five for men and women, respectively. The optimum cut-point decreased to five and two when weighting favoured sensitivity, and increased to fifteen and ten when weighting favoured specificity, for men and women, respectively.

Optimal cut-points for identification of DSM-IV alcohol dependence using AUDIT were twelve and seven for men and women, respectively. The optimum cut-point decreased to nine and two when weighting favoured sensitivity, and increased to twelve and eleven when weighting favoured specificity, for men and women, respectively.

Optimal cut-points for identification of DSM-5 AUD (none vs. mild/moderate/severe) using AUDIT were ten and six for men and women, respectively. The optimum cut-point decreased to five and two when weighting favoured sensitivity, and increased to thirteen and eleven when weighting favoured specificity, for men and women, respectively.

Post-test probability curves, for hazardous drinking, DSM-IV alcohol abuse, DSM-IV alcohol dependence, and DSM-5 AUD, along the range of AUDIT scores, are shown in Figure 1:

Figure 1: Bayes’ Theorem post-test probability estimates for drinking problems in English males and females aged 18-35, according to AUDIT score.

For a male with an AUDIT score of 25, he would have a 0.92 probability of being a hazardous drinker, a 0.35 probability of being an alcohol abuser, a 0.68 probability of being alcohol dependent, and a 0.81 probability of having an alcohol use disorder. For a female with an AUDIT score of 25, she would have a 1.00 probability of being a hazardous drinker, a 0.31 probability of being an alcohol abuser, a 0.85 probability of being alcohol dependent, and a 0.88 probability of having an alcohol use disorder.

In this study we found that the AUDIT screening test was accurate for the assessment of reference standard classifications of hazardous drinking, DSM-IV alcohol abuse, DSM-IV alcohol dependence, and DSM-5 AUD in a sample of 18-35 year-old adults from UK primary care. The short-form AUDIT-C had a similar accuracy profile to the full AUDIT for the detection of hazardous drinking. The response rate was low, and the sample skewed to low deprivation postcodes.

Optimal test thresholds depended on the value attached to minimizing the cost associated with false test results. From a public health paradigm it may be more appropriate to be more tolerant of false positives and use a threshold with higher sensitivity for detecting hazardous drinking, when coupled with brief, low cost and effective interventions.

An alternative approach, using Bayes Theorem, is to calculate the post-test probability for each test score and to use this in feedback and dialogue with screened patients. This approach has the advantage of using all available information rather than collapsing test scores above and below a selected threshold score, and can also take account of varying pre-test probabilities based on known prevalence rates for specific age, gender and other population parameters, and clinical judgment.

Such an approach is entirely feasible using computer- or web-based assessment and feedback technology, and could also address some of the implementation problems that have been identified with alcohol screening and brief intervention in general practice (van Beurden et al., 2012). The feasibility and effectiveness of this approach should be examined in further research.

DF & LS jointly conceived and designed the project and analysis. HT and SH collected data and undertook initial analyses. DF undertook the analyses and led the writing for this report.

A peer-reviewed paper based on this project is available in Alcohol and Alcoholism here: http://alcalc.oxfordjournals.org/content/50/2/244

American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders (4th ed.) (DSM-IV). Washington, D.C.: APA.

American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders (Fifth ed.). Arlington, VA: American Psychiatric Publishing

Babor TF, Biddle-Higgins JC, Saunders JB et al. AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care. Geneva, Switzerland: World Health Organization 2001.

van Beurden, I, Anderson, P, Akkermans, RP, Grol, RP, Wensing, M, Laurant, MG (2012) Involvement of general practitioners in managing alcohol problems: a randomized controlled trial of a tailored improvement programme. Addiction 107: 1601-11.

Foxcroft DR, Kypri K, Simonite V et al. Bayes’ Theorem-estimate population prevalence from Alcohol Use Disorders Identification Test (AUDIT) scores. Addiction 2009;104:1132–7.

Robins, LN, Wing, J, Wittchen, HU et al. (1988) The Composite International Diagnostic Interview: An epidemiologic instrument suitable for use in conjunction with different diagnostic systems and in different cultures. Archives of General Psychiatry 45: 1069-77.

Sobell LC, Sobell MB, Riley DM et al. The reliability of alcohol abusers’ self-reports of drinking and life events that occurred in the distant past. Journal of Studies on Alcohol 1988;49:225–32.

Sobell LC, Sobell NB, Litten RZ et al. Timeline Follow Back. A technique for assessing self-reported alcohol consumption In: Measuring alcohol use. New Jersey: Humana Press Inc 1992. 41–72.

Smith LA, Foxcroft DR, Holloway A, Minozzi S, Casazza, G. Brief acohol questionnaires for identifying hazardous, harmful and dependent alcohol use in primary care. Cochrane review, 2014.