New HIV Vaccine Less Effective Than Initially Reported

Last month, researchers made a big splash with the news that a new HIV vaccine reduced the risk of infection by 31%. But new analysis shows only a 26% reduction, which could have occurred by chance.

According to Alice Park in Time, the discrepancy occurred because in their initial announcement, researchers included all the participants who had started the study. But some got infected with HIV before they had received all six shots of the vaccine, and then had to drop out. Since the point of the research was to measure the effect of all six injections, only those who received all of them should have been included in the data. And when the numbers are run this way, the result is a 26% reduction in risk — a finding that's not statistically significant, meaning it could have occurred by chance.

It's tempting to criticize the researchers, from the National Institute of Health and the US Army, for rushing to release promising results before properly vetting them. According to Park, the researchers decided to publicize their findings in a press conference before they were peer-reviewed or replicated because the government of Thailand, where the study was conducted, "wanted to inform its citizens of the positive findings as soon as possible." However, even before the announcement, a few scientists who had seen the data were expressing concerns.

Even more scientists are speaking out critically now. One AIDS researcher, who spoke anonymously to the ScienceInsider blog, says,

The press conference was not a scholarly, rigorously honest presentation. It doesn't meet the standards that have been set for other trials, and it doesn't fully present the borderline results. It's wrong.

The Army, however, says that it didn't present both the 31% and 26% numbers because that just would've been too complicated. An online statement reads, in part,

The multiple statistical analyses are all consistent with the same conclusion: that the vaccine was modestly effective at preventing HIV. However, explaining the differences between them is complex and the appropriate venue for this technical discussion of statistics is at an open scientific conference and in the scientific publication now under review at a major journal.

If truly explaining the results was only appropriate for a scientific forum, then it does seem that the researchers should have waited for such a forum before publicizing them. The statement claims that the researchers fulfilled the Thai government's request for a public statement in order to honor "our commitment to the volunteers who participated in this trial," but the volunteers would have been better honored by an accurate representation of the findings. The Army appears to be trying to have it both ways, saying that they had to publicize results quickly, but they couldn't possibly publicize the accurate ones because no one would understand them. But there are a lot of ways out of this double bind — it's really not all that hard, for instance, to explain the difference between a 26% and 31% figures. Army researcher Col. Nelson Michael says,

We tried very carefully to make sure that message was crystal clear. There's now hope. But that said, we've tried to be very careful not to oversell this.

It may not have been intentional, but overselling is exactly what they've done.

Behind The Rising Doubts About Hailed AIDS Vaccine [Time]
The HIV Vaccine And Science By Press Release [Wall Street Journal Health Blog]
Unrevealed Analysis Weakens Claim Of AIDS Vaccine "Success" [ScienceInsider]
RV 144 Update [US Military HIV Research Program]