Increasing Alcoholism: A Follow-Up

I posted an article a week or so ago on a new study claiming a rise in alcoholism. The study has been met with some major criticism. From Vox:

some researchers are pushing back. They argue that the data used in the study is based on a federal survey [NESARC] that underwent major methodological changes between 2001-’02 and 2012-’13 — meaning the increase in alcoholism rates could be entirely explained just by differences in how the survey was carried out between the two time periods. And they point out that the study’s conclusions are sharply contradicted by another major federal survey…That survey has actually found a decrease in alcohol use disorder from 2002 to 2013: In 2002, the percent of Americans 12 and older who qualified as having alcohol use disorder was 7.7 percent. In 2013, that dropped to 6.6 percent.

One key difference is the NESARC used data of people 18 years and older, while NSDUH used data of people 12 years and older. But even if you isolate older groups in NSDUH, the rates of alcoholism still dropped or remained relatively flat — certainly not the big rise the NESARC reported.

Now, the NSDUH isn’t perfect. For one, it surveys households — so it misses imprisoned and homeless populations, which are fairly big segments of the population and likely to have higher rates of drug use. But NESARC also shares these limitations, so it doesn’t explain the difference seen in the surveys.

Here are some of the major changes to the NESARC:

• The NESARC changed some questions from wave to wave, which could lead survey takers to respond differently.
• In the 2001-’02 wave, NESARC respondents were not given monetary rewards. In the 2012-’13 wave, they were. That could have incentivized different people to respond.
• No biological samples were collected in the first wave, while saliva samples were collected in the second. What’s more, respondents were notified of this at the start of the survey — which could have led them to respond differently, since they knew they’d be tested for their drug use.
• Census Bureau workers were used for the 2001-’02 survey, but private workers were used for the 2012-’13 survey. That could lead to big differences: As Grucza told me, “Some researchers speculate that using government employees might suppress reporting of socially undesirable behaviors.”

The article continues,

Researchers from SAMHSA told me that they would caution against trying to use the different waves of NESARC to gauge trends.

“Given these points, we would strongly caution against using two points in time as an indicator in trend, especially when the data for these two points in time were collected using very different methods and do not appear to be comparable,” SAMHSA researchers wrote in an email. “We would encourage the consideration of data from multiple sources and more than two time points, in order to paint a more complete and accurate portrayal of substance use and substance use disorder in the nation.”

In short, it looks like the JAMA Psychiatry study was based on some fairly faulty data.

When I asked about these problems surrounding the study, lead author Bridget Grant, with NIAAA, shot back by email: “There were no changes in NESARC methodology between waves and NSDUH folks know nothing about the NESARC. Please do not contact me again as I don’t know NSDUH methodology and would not be so presumptuous to believe I did.”

But based on SAMHSA’s and Grucza’s separate reviews of NESARC, its methodology did change.

When I pressed on this, Grant again responded, “Please do NOT contact me again.”

After this article was published, Grant confirmed NESARC went through some methodological changes between 2001-’02 and 2012-’13. But she argued that there’s no evidence such changes would have a significant impact on the results.

It concludes,

None of that means America doesn’t have an alcohol problem. Between 2001 and 2015, the number of alcohol-induced deaths (those that involve direct health complications from alcohol, like liver cirrhosis) rose from about 20,000 to more than 33,000. Before the latest increases, an analysis of data from 2006 to 2010 by the Centers for Disease Control and Prevention (CDC) already estimated that alcohol is linked to 88,000 deaths a year — more than all drug overdose deaths combined.

And another study found that rates of heavy drinking and binge drinking increased in most US counties from 2005 to 2012, even as the percentage of people who drink any alcohol has remained relatively flat.

But for now, it’s hard to say if a massive increase in alcohol use disorder is behind the negative trends — because the evidence for that just isn’t reliable.