Physician, Regulatory, and Payer Perspectives on the Value of Real-World Data

One of the kickoff sessions on the first day of the 2017 American Society of Clinical Oncology (ASCO) Annual Meeting in Chicago, Illinois, was a lively discussion on ensuring that the data used to inform patient care and create healthcare policies holds value. This, according to the speakers who participated on the panel, entails moving away from clinical trial data and foraging real-world data captured in health records

The goal of the session was to identify existing and developing data sources that can be used to inform comparative effectiveness research, patient care, and healthcare policy. The presenters also provided examples of questions that can be asked using these data, and described how payers consider observational data alone or in conjunction with data from clinical trial.

Ronald C. Chen, MD, MPH, associate professor, Department of Radiation Oncology, The University of North Carolina at Chapel Hill, was both presenter and chair of the session, and he tried to convince listeners that while randomized controlled trials (RCTs) are the industry’s gold standard, they come with significant limitations.

Chen explained that RCTs are not obsolete and that they are a significant source of information, "but there are gaps that exist and we need to find alternatives.” Comparative effectiveness research is unbiased comparison that yields valid results. “The goal is to estimate the truth … which RCTs help with, especially when it comes to comparing the efficacy of product A vs product B,” Chen said. He added that RCTs and observational data together can prove helpful to clinicians, policy makers, payers, and patients.

While RCTs minimize confounding, there are limitations to this gold standard, Chen added. He then cited the Prostate Cancer Intervention Versus Observation Trial (PIVOT) as a case study. The primary question that PIVOT asked was whether radical prostatectomy can save lives in men who have localized prostate cancer. Starting with a large cohort of over 13,000 men, only 5000 met the trial’s eligibility criteria. Of these, only a little over 700 patients were randomized between the 2 arms: radical prostatectomy and observation, between 1994 and 2002; the follow-up period ended in early 2010 and the results were published in 2012.

The trial found no difference in survival between the observation and the surgical intervention arms, but nearly 20 years after it was initiated.

Chen listed the following potential limitations of RCTs:

• Patients are often highly selected (younger, healthier) This begs the question: are outcomes representative of “all” patients in the real world? Generalizability is a concern. “Can decreased generalizability decrease treatment adoption?” he asked.
• Can results remain relevant, especially since RCTs usually require a long time to completion?
• Is it possible for RCTs to provide clinically relevant and timely results?
• Not every clinically important question can be addressed through an RCT

Providing the FDA’s perspective on real-world data was Sean Khozin, MD, MPH, senior medical officer at the FDA. Khozin reviewed how the FDA is using real-world evidence in the context of regulatory decision-making.

Khozin provided a timeline of human progress across over the past several decades, which included the development of steam and gas locomotives, the moon landing, and the Internet explosion, among others. However, the clinical trial model remains rudimentary, he said.

“There remains room for improvement in RCTs,” Khozin explained. “We trust these results because they have robust internal validity.” He described the structure of RCTs as a “validity imbalance,” with an overcompensation of internal validity and an external-validity deficit.

RCTs also have poor generalizability—there is no median or average patient “That is just a statistical concept.” So, treatment decisions based on the ‘median’ outcome of a trial will not help us maximize the potential of precision oncology.

Khozin went ahead to outline the characteristics of real-world data, he explained is mostly based on why the data were collected, meaning the intent of data collection: within the controlled settings of a clinical trial or in the real-world of a physician’s office.

“Real-world data helps retrospective analysis,” Khozin explained, which can be achieved using electronic health record (EHR) data that is cleaned up. EHRs have a structured (billing and lab codes, patient history and demographics) and an unstructured component (physician notes and diagnostic reports).

Speaking with The American Journal of Managed Care^® in November 2016, Khozin said that one component of the FDA’s Information Exchange and Data Transformation initiative is using real-world evidence, in the form of EHRs, to guide regulatory decisions. He explained that exclusion criteria in clinical trials can be limiting, so the data from clinical trials may not reflect the patients being treated in the real world.

“We can change the intent of data collection from research to real-world data, by providing clinicians incentives to do so,” Khozin said during the ASCO session. This is how pragmatic or prospective trials are defined. Khozin stated that frameworks exist for real-world data collection and that the FDA is not concerned with the original intent of data collection, since there are processes in place to scrutinize the submitted information.

Real-world data can be used for:

• Pharmacovigilance. Currently a passive process associated with voluntary reporting of adverse events, real-world data can power an active pharmacovigilance program (eg, FDA’s Sentinel program and direct EHR abstraction).
• Benchmarking. To develop historical control benchmarks to inform future trial designs, and to provide reliable safety and efficacy data.
• Conduct pragmatic clinical trials. To allow for point-of-care clinical decisions, EHRs are vehicles for prospective clinical research at point of care, can support randomization, are patient-centric, and may bend the cost curve.

The primary challenge with real-world data, Khozin said, is ensuring data quality and the need to provide the right incentives at the point of care to extract clinically relevant data. “This is more an organizational issue.”

Alan Rosenberg, MD, vice president of clinical pharmacy and medical policy, Anthem, spoke about how payers view nonrandomized data when making coverage determinations.

“Quality, access, efficiency, equity remain challenges with healthcare in our country, compared with the rest of the world,” said Rosenberg. “This is a real issue for us as healthcare providers.”

He believes that the variation in care stems from socioeconomic and geographic differences. “However, a lot of this variation remains unexplained,” Rosenberg added. “We as payers, are aware of the origins in knowledge gap, and also understand the cost associated with running RCTs. But we do expect well-designed trials that provide significant outcomes.”

At the same time, we know that real-world adherence is much lower than RCT adherence. So, the number-needed-to-treat in RCTs is much higher, considering adherence issues.

Payers do consider non—clinical trial data when making coverage decisions and the process involves:

• Examination of relevant peer-reviewed data, which are used with caveats
• There’s recognition that there is significant difference in quality of non-clinical trial data
• Recognition of the difference between quantum and small incremental results
• Recognition of the difference between RCT and developing treatment for ultra-rare diseases

Rosenberg also emphasized that FDA approval is necessary but may not be sufficient for making coverage decisions.