Early Results From Clinical Trials of Chronic Conditions Are Often Exaggerated

Early results from clinical trials in chronic medical conditions demonstrate a treatment effect that is often larger than what is found in subsequent trials, according to new research published in Mayo Clinical Proceedings. More than 1 in 3 early clinical trials may show exaggerated results.

Researchers identified 70 meta-analyses of 930 randomized controlled trials and estimated the prevalence of the first 2 published trials having the largest effect. They discovered that the first or second studies reported an effect 2.67 times larger than what subsequent trials eventually showed. This is known as the Proteus effect.

“This phenomenon of exaggerated early results was present in a whopping 37% of the studies we reviewed,” Fares Alahdab, MD, lead author of the study and a research fellow in Mayo Clinic’s Evidence-Based Practice Center, said in a statement. “Physicians and patients should be cautious about new or early clinical trial evidence. Exaggerated results could lead to false hope as well as possibly harmful effects.”

The researchers noted that clinical and policy decisions made using the early trial results could be misguided and based on incorrect estimates. They sought to evaluate the Proteus effect and explore factors associated with the phenomenon. The factors they evaluated included sample size, total number of events, study length, length of follow-up, risk of bias, whether trials were stopped early, and study settings.

The Proteus effect was more frequently seen in studies in the field of endocrinology compared with cardiology, oncology, nephrology, or pulmonary medicine. Meta-analyses evaluating a procedure were far less likely to exhibit the Proteus effect. The phenomenon was more likely to be seen in meta-analyses evaluating a medication.

The researchers found no factors were significantly associated with the Proteus effect.

“Some people may think this is an anti-innovation message,” Murad said. “To the contrary, we welcome new treatments. We just want people to know that the benefit seen in real practice, when treatments are given to people with various comorbidities and in different settings, may be smaller than what was seen in the earliest clinical trials.”

Reference

Alahdab F, Farah W, Almasri J, et al. Treatment effect in earlier trials of patients with chronic medical conditions: a meta-epidemiologic study [published online February 21, 2018]. Mayo Clin Proc. mayoclinicproceedings.org/article/S0025-6196(17)30836-4/fulltext. Accessed February 21, 2018.