Expert Perspectives on the Role of Real-World Evidence, Patient-Reported Outcomes, and Economic Models in Oncology Formulary Decision and Regulatory Submissions for FDA Approval

This publication was supported by Pfizer, Inc.

REAL-WORLD EVIDENCE

American Journal of Managed Care^®: How would you define real-world evidence (RWE)?

Bhakti Arondekar, PhD, MBA: The FDA defines RWE as the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of real-world data (RWD). This is very consistent with how a manufacturer would define RWE. In the broadest sense, RWE is any data that is developed outside of a clinical trial setting. RWE can play a pivotal role in helping us assess the efficacy and safety of our medicines at a population level, outside of the confines of controlled clinical trial. It is also an incredibly valuable tool to communicate the benefits and risk associated with a medicine to patients, payers, regulators, and providers.

Jay Weaver, PharmD, MPH: It depends on the need. Patient-reported outcomes (PROs) can be a source of RWD. It could be useful to understand the experience of the patient as they are taking the medication, for instance, adverse effects. That data sometimes is captured in more meaningful ways in phase 4 assessments post launch as a product comes to market and is used in a larger population. The second domain can be claims data extracts; how is the product being used? What do we see within our claims systems in terms of diagnosis, population, [and] ages? After they use the product, what additional diagnosis are they experiencing? What other procedures are they experiencing while on the medication? Are the services rendered suggesting that a product is or isn’t working based on its intended use? Another domain of that is moving away from claims and into the transactional information of care, such as the electronic medical records (EMRs). From EMRs, structured data can be collected and insights can be gleaned. These data fields might indicate someone’s illness or other medications they are using. That data is most useful data when codified in some way, but [there is] also free text information within the EMRs—which can be useful through reading over notes and what’s happening post–care delivery. Machine learning has been used to mine these free-text notes for meaningful insights. We can understand unexpected consequences of therapy, and can create new data sets and instruments. Those are some main areas of RWE. Then metadata can be established, which is a combination of those pieces of information-gleaned insight. What are we understanding about a product launch as it is used in a practical setting? What is the confluence of those things telling us?

AJMC^®: How do you utilize RWE in decision-making?

Weaver: One way to understand whether or not a product is being used in a way differently than intended, either from the labeling of the product or from the policy that we create establishing the criteria for use. We can understand if those are being followed or if there is drifting of utilization. We also can learn about discontinuation of therapy. Are we investing in therapies that are not seeming to reach the clinical end points and not being continued? That’s very important information. [For example, in] clinical studies, a product may perform at, say, a 70% efficacy level, reaching some end point in a controlled, observed environment in which patients are taking their medication, and we are making sure that they stay in the trial. Outside of that trial, maybe people have a high discontinuation rate, and, therefore, the therapy is only 50% effective. If we make decisions around the coverage of that product and the value proposition of it based on it being 70% effective, then it is only 50% effective. Those assumptions tend to be faulty in terms of economic impact. It would be less cost effective than it seems. That data can be cycled in, not only from the perspective of contracting for what we pay for it, but it also could take the form of supporting new types of innovative contracting (eg, risk sharing). If someone fails the therapy or discontinues the therapy, there might be some type of remuneration back to the health plan sponsor to make them whole for the investment, or at least part of the investment, in a therapy that didn’t reach its end point.

AJMC^®: Does this differ in oncology versus other therapeutic areas? If yes, how?

Weaver: The interesting thing about oncology RWE is that it tends to be a condition for which there are very defined end points that we can observe. So, whether they are adverse events or failures of therapy, there is a good way to know that information. If someone has recurrence of cancer, we can detect that. There are other disease states [for which] the picture is not quite as clear. Let’s say someone with diabetes has a cardiovascular event; sometimes, it is hard to know if that cardiovascular event was due to their underlying diabetes or some other cardiovascular factor. As you get further away from that kind of evidence, it’s harder to know their association. Oncology, because of the end points as well as [the possible speed of recurrence], is a very attractive disease state for looking at that data.

AJMC^®: What do you believe are the best sources of RWD (eg, EMRs, claims data, registries)?

Arondekar: The research question that you want to address should drive the real-world data source that you use to generate RWE. Each of the data sources listed above [has its] own advantages and disadvantages. For example, EMRs often have rich data on clinical variables that might not be captured in claims data. So, if your research question is assessing clinical outcomes associated with a particular tumor type, you will give preference to EMR data or registry data over claims data. On the other hand, if you want to look at a large patient population, since a payer is making decisions at the population level, you would prefer claims data with a large number of patients included. With technological advances in EMRs and data curation, we now have the ability to integrate data from both data sources so you get the best of both worlds.

The interesting thing about oncology RWE is that it tends to be a condition for which there are very defined end points that we can observe. So, whether they are adverse events or failures of therapy, there is a good way to know that information.

—Jay Weaver, PharmD, MPH

AJMC^®: How is RWE utilized?

Arondekar: From a manufacturer’s perspective, historically, RWE has been used to understand unmet need, treatment patterns, cost of illness, prevalence, and incidence of different diseases in the target patient population for a drug. The availability of larger data sets have also enabled researchers to generate comparative effectiveness data once a medicine has been on the market for a duration of time or when newer competitor medicines are introduced. With the passing of the 21st Century Cures Act, there has been a push toward the use of RWE to supplement data from clinical trials, either to contextualize or to compare clinical trial evidence (ie, synthetic control–arm studies). In addition, we have also seen label expansions resulting from the use of RWD.

AJMC^®: How do you think that RWE can address knowledge gaps in oncology?

Arondekar: Oncology has its own nuances vs other therapeutic areas when it comes to RWE. Recently, a lot of new drugs that are being developed are [for treatment of] rare tumors, biomarker-driven patient populations, or patients [who have certain cancers with] a poor prognosis [for which] the only treatment option available is palliative care. Moreover, there have been products approved based on data from early-phase, single-arm clinical trials, all of which leads to uncertainty in terms of outcomes in a [real-world] setting and broad patient population. These knowledge gaps can be very important for payers to address, and RWE can play an important role in addressing these knowledge gaps. For example, findings from external control, [real-world] studies complement efficacy data from single-arm trials in successful oncology product approvals.

AJMC^®: What type of RWE do you find most beneficial?

Arondekar: With new product approvals, the treatment landscape changes from the time that the clinical trials are designed. Therefore, the standard of care evolves, and it also impacts treatment guidelines. Moreover, payers don’t have head-to-head comparative data to inform their formulary decision-making when there are multiple products approved in the same class. Under these circumstances, RWE-based comparative effectiveness studies can be very beneficial for payers and physicians in helping them with formulary or treatment decisions. In addition, with the introduction of newer therapies, RWE studies assessing adherence, adverse events, and resource utilization can also be very informative. Finally, clinical trials use restrictive inclusion and exclusion criteria in patient selection, thereby limiting the generalizability its findings. Therefore, RWE studies validating findings from trials can further provide value to payers.

AJMC^®: Please discuss findings from the recent survey conducted that assessed payer perceptions of the use of RWE in oncology decision-making.¹ Please discuss study design, results, and key takeaways.

Arondekar: This was a survey of US payers to understand their perceptions of the use of RWE in informing oncology formulary decisions.¹ A multidisciplinary steering committee developed a web-based survey and piloted the survey with 5 US payers. Based on comments received from the pilot survey, we revised the questionnaire and fielded the survey to more than 200 US payers who were part of the Academy of Managed Care Pharmacy (AMCP) Market Insight Program. This panel included pharmacy leaders involved in medication product evaluation and/or utilization management who agreed to participate in payer-based research. After the survey was completed, we convened a virtual panel discussion with 10 US payers to discuss the survey findings. Findings from this survey showed that most US payers use RWE to help inform oncology formulary decision-making. In particular, US payers thought RWE was useful to assess comparative effectiveness where head-to-head clinical trials are not available and to help inform clinical guidelines. Based on the survey, almost half of US payers reported [using RWE] to help inform off-label coverage decisions.

RWE can be very helpful in contextualizing clinical trial data under the following circumstances—[in the] niche patient population with rare tumor types or diseases or in the case of single-arm clinical trials in which it might be unfeasible or unethical to conduct a randomized trial.

—Bhakti Arondekar, PhD, MBA

AJMC^®: Do you generate your own RWE? If yes, why? How is this accomplished?

Arondekar: Yes, we generate and publish RWE across all phases of drug development. Prelaunch, we generate RWE to understand disease epidemiology, standard of care, treatment patterns, and disease burden. In the postlaunch setting, we develop RWE to understand real-world utilization outcomes beyond those studied within the clinical trial (eg, adherence, costs, resource utilization, real-world outcomes), validate clinical trial findings, and [accomplish] comparative differentiation. Manufacturers have access to several data sources (eg, claims data, EMRs, registries) along with in-house analytic capabilities to generate robust RWE. Smaller health plans often don’t have these capabilities and rely on the manufacturer to generate and disseminate these findings.

AJMC^®: Your colleagues and you published a review concerning RWE inclusion in FDA submissions.² Please discuss review methodology, findings, and key takeaways. Why do you think that RWE is so infrequently included in submissions?

Arondekar: The analysis, published in Clinical Cancer Research, is the first of its kind to systematically aggregate detailed regulatory feedback to provide practical insights to manufacturers.² Whereas existing FDA guidance documents provide a theoretical framework for conducting regulatory RWE studies, the details of how to actually design and analyze adequate RWE studies remain largely unaddressed. To meet this critical knowledge gap, a team of researchers from Pfizer, Analysis Group, and the Dana-Farber Cancer Institute analyzed 133 original and 573 supplemental oncology New Drug Application and Biologics License Application approvals to identify the attributes of a successful RWE study that contribute to an accelerated or full drug approval. The key takeaways from our review were as follows:

• Engage the FDA early to confirm appropriate data sources and whether the RWE study should be designed as a natural history study for contextualization or as an external control study for comparison with the pivotal trial. A hybrid study design to combine trial with external control data through Bayesian or frequentist methods and ambidirectional RWE data collection (both prospective and retrospective) are study designs worth considering.
• Select appropriate data sources to ensure that RWD are of high quality and fit for purpose. Although chart review was the most common source for RWE, the FDA also commented that data from such studies could have limited generalizability and [be] subject to selection bias.
• Align the RWE and pivotal trial populations by matching on-trial inclusion and exclusion criteria to the extent possible and adjusting for the remaining imbalance in baseline characteristics with propensity score weighting methodology, such as inverse probability treatment weighting. Critically, the study protocol needs to be developed a priori.
• Describe methods to minimize residual confounding and unmeasured confounding, including appropriate index date and reduction in missing values. If imputation methods are used to address missing values, then validation of the imputation algorithms is recommended. The impact of unmeasured confounding should be evaluated through quantitative bias analysis.

Whereas these findings might seem straightforward, they are often challenging to implement in practice. For example, each data source has its own strengths and limitations, so getting the right data source, understanding the limitations, [and] curating it to develop regulatory-grade data can be a challenge in itself. Identifying and mirroring a clinical trial can also be difficult, especially for newer, biomarker-driven products [for which] the biomarker might not routinely be tested and captured in RWD sources. Also, patients who are prescribed products in a real-world setting can be difficult to match with the patients who are enrolled in clinical trials using strict inclusion and exclusion criteria. For all these reasons, RWE that is generated is not frequently used in supporting regulatory submissions or, even if submitted, might not be accepted by regulatory agencies [for decision-making].

AJMC^®: In which cases do you think that RWE can be most helpful as part of an FDA submission?

Arondekar: RWE can be very helpful in contextualizing clinical trial data under the following circumstances—[in the] niche patient population with rare tumor types or diseases or in the case of single-arm clinical trials in which it might be unfeasible or unethical to conduct a randomized trial.

AJMC^®: Would you look more favorably upon an oncology drug that included RWE as part of the submission?

Weaver: That’s a difficult question to answer. Yes, sometimes. There is a baseline set of overall study information that we need, and all things being equal among, let’s say, 2 different agents, if 1 agent had RWE and the other one did not, I would look more favorably on that [first] agent. On the other hand, if you look at differences of performance within the clinical trial, and 1 product lacks RWE but has stronger efficacy in clinical trials, we would look at the magnitude of the difference to determine which has the stronger performance. It’s a difficult trade-off to consider but necessary to evaluate.

AJMC^®: Are you utilizing outcomes-based contracting at your organization? Why or why not?

Weaver: Yes, but in a very limited fashion. There wasn’t anything needed in our market for some time just based on the clinical management strategy. We have recently launched into OBCs that with a small number of contracts, and we are contemplating a much larger footprint of those contracts as we think about vendors such as pharmaceutical manufacturers sharing in the accountability of delivering good outcomes between the plan, the pharmaceutical supplier, the provider, and the member and aligning that. It’s a key strategy as we move forward and make sure that we are all pulling in the same direction in terms of our incentives to drive value overall.

AJMC^®: What educational opportunities exist to increase RWE understanding and evidence generation for payers? How would this best be accomplished? What tools are needed?

Weaver: We are at a time of an inflection point around how information is used and what we consider is part of our reviews and our policy. For instance, composite end points have been shunned for many years by people in evidence circles who are considering coverage, and now they are accepted by the FDA as a primary end point for some studies for some drugs to be approved. In a similar way, RWE is becoming a very meaningful, important, and impactful set of measurements for understanding how products perform, and we need a paradigm shift [regarding] using those end points. Traditionally, people in the payer space who are making policy decisions have only thought of using information from controlled clinical trials in terms of their policy decision. We need to bring these new data sets in to learn more than we know today and accept those as part of our evidence package for policy consideration. I think it is going to take some time for that migration of thinking ... As we begin to teach that in our colleges of medicine and colleges of pharmacy and bring that into the evidence-based medicine community (eg, some of the thought leadership in that space, such as at Oxford, and other evidence-based medicine centers), we will begin to grow in terms of our use of that data and the acceptance of those data sets.

PATIENT-REPORTED OUTCOMES

AJMC^®: How do you use PRO data in oncology decision-making, if at all? What types do you find most impactful?

Weaver: We look at several things; the most prominent are PROs around discontinuation and adverse effects. Understanding the experience folks have in terms of tolerability of therapy is key to knowing how to manage those adverse effects. How do we help a member or patient understand what they might experience? What types of treatment might they use, or what differences in their routine could help lower the instance of those adverse effects? Lastly, knowing when to discontinue the therapy. These are key opportunities for us to help improve the experience, improve quality of life, but also to know the likelihood of therapy to deliver on its promise.

AJMC^®: What do you see as the value of PRO data in informing policies and formulary decisions? What challenges have you experienced?

Weaver: Many people who do value assessments have a negative perception around patient-reported information, because it is subjective. People can report experiencing things that may or may not be related to the treatments, or they may not know how to describe [their experience] in a way that reflects what is happening with the product that is getting co-opted. It is something that is worth [being cautious about]. We think about how much credibility we give to [patient-reported data]. On the inverse of that is the opportunity of seeing through the lens of someone who is taking the therapy, and we can listen to [their] unique challenges or successes—that they were able to go to work more days than they had before. If a global assessment measures productivity and/or activity, those are very meaningful end points [that indicate] someone’s likelihood to stay engaged in their care. There are opportunities and challenges.

When making formulary decisions, US payers desire additional evidence beyond traditional safety and efficacy data. This may be more pronounced in oncology due to single-arm trial designs, niche patient populations, and accelerated regulatory approvals. Economic models help project long-term clinical and economic benefits of such therapies that payers can use to inform access decisions for medicines.

—Alexander Niyazov, PharmD, MPH

AJMC^®: In what other ways do you use PRO data or think it could be helpful?

Weaver: Other ways we use that information is to understand physician experience or physician preference for a certain therapy, so we can understand some of the information, some of the experiences, with care. We also utilize that information to appreciate member satisfaction. It’s a way to understand how satisfied someone is with their care, and, in particular, with their treatment. As we move into this world of Medicare quality, stars ratings, and [Consumer Assessment of Healthcare Providers & Systems] scores, those perceptions are needed for monetizing our plans and keeping members in our rolls. We are providing the therapies that people are feeling best on and are most likely to stay on. Those members are more likely to renew with our plan so that we can help to recoup the investment we made in those therapies.

AJMC^®: Please discuss findings from a recent survey by Oderda et al that assessed payer perceptions of the use of PROs in oncology decision-making.³ Please discuss study design, results, and key takeaways.

Alexander Niyazov, PharmD, MPH: This was a survey of US payers to better understand their perceptions of how PROs impact formulary decision-making in oncology. A multidisciplinary steering committee developed a web-based survey and piloted the survey with 5 US payers. Based on comments received from the pilot survey, we revised the questionnaire and fielded the survey to more than 200 US payers who were part of the AMCP Market Insight Program. This panel included pharmacy leaders involved in medication product evaluation and/or utilization management who agreed to participate in payer-based research. After the survey was completed, we convened a virtual panel discussion with 10 US payers to discuss the survey findings. The survey showed that [the] majority of US payers, both small payers (< 1 million lives), and large payers (≥ 1 million lives), believed that PRO evidence was useful in informing oncology formulary decision-making. This was true for PRO data coming from clinical trials as well as [for real-world] PRO data. Almost half of the payers reported that lack of PRO data would influence oncology formulary review. Overall, these findings validated manufacturers’ efforts in collecting patient-centric evidence collected through PROs to enable patient access.

AJMC^®: How do you feel that PRO data could influence a value-based agreement, if at all? Do you see the use of PRO data increasing for your organization in the future?

Weaver: It could have a big influence on those agreements with information [on] potential problems in therapy, such as how likely [it is for] someone to discontinue because of adverse effects or how likely is it to improve quality-of-life aspects. Being able to frame that with a likelihood of the event in value-based agreements is a very powerful tool.

ECONOMIC MODELS

AJMC^®: How do you utilize economic models in oncology decision-making? What types of models do you find most and least beneficial? When do you find them most beneficial (eg, when evaluating products with similar safety/efficacy)?

Weaver: Models are at inflection points where [they] are becoming used, but folks still misunderstand them and misuse them. The models most commonly used for oncology in our area are budget-impact modeling and cost-effectiveness modeling. I find both helpful. I was training at the time, and I had a postdoctoral training that taught me the value of those tools, so I’ve employed those throughout my career. There is a way to contextualize and compare the value of treatments with other known interventions. It helps us understand things that are hard to wrap our heads around in terms of investments in high-cost therapies that have very high-cost morbidities associated with the disease.

AJMC^®: Please discuss findings from a recent survey by Biskupiak et al that assessed payer use of economic models in oncology decision-making.⁴ Please discuss study design, results, and key takeaways.

Niyazov: This was a survey of US payers to better understand their perceptions of how economic models impact formulary decision-making in oncology.⁴ A multidisciplinary steering committee developed a web-based survey and piloted the survey with 5 US payers. Based on comments received from the pilot survey, we revised the questionnaire and fielded the survey to more than 200 US payers who were part of the AMCP Market Insight Program. This panel included pharmacy leaders involved in medication product evaluation and/or utilization management who agreed to participate in payer-based research. After the survey was completed, we convened a virtual panel discussion with 10 US payers to discuss the survey findings. The survey showed that [a] majority of US payers indicated moderate/most interest in cost-effectiveness and budget-impact models. Large plans (≥ 1 million lives) were more likely to have expertise in reviewing oncology models than [were] small plans (< 1 million lives). Common reasons for not reviewing economic models were that the models weren’t available at the time of review and [that there was a] perception of bias. More than two-thirds of payers reported that economic models were useful when comparing therapies with similar efficacy/safety. Overall, this study highlights the relevance of economic models in informing oncology formulary decision-making. Additionally, [the study results shed] light [on] opportunities that exist to improve the utility of oncology economic models.

AJMC^®: What do you see as the benefits and drawbacks of economic models? How could the drawbacks be overcome?

Niyazov: First, the benefits. When making formulary decisions, US payers desire additional evidence beyond traditional safety and efficacy data. This may be more pronounced in oncology due to single-arm trial designs, niche patient populations, and accelerated regulatory approvals. Economic models help project long-term clinical and economic benefits of such therapies that payers can use to inform access decisions for medicines.

Economic models may have some drawbacks, as well:

1. Black box. Payers often criticize models that are perceived to be too complicated and not transparent enough.
2. Concerns about lack of external validity. When clinical trial [results are published], payers may still have some unanswered questions. As such, assumptions based on RWD or expert opinion are made. Whereas efforts are made to use assumptions based on national estimates, those assumptions may be subject to change over time and may not be reflective of a particular health plan.
3. Some economic models are cumbersome to use. Most models have many tabs that may not be easy to navigate. As such, end users may get frustrated when reviewing models.

These limitations can be overcome in a few ways. First, to improve transparency, developers should reference all assumptions included in building the model. Additionally, highlight all assumptions that were included in the model, as it may help increase transparency. Ensure applicability of the model to the health plan. To ensure that the model is relevant to the health plan, models should allow end users to modify assumptions to better reflect patient demographics and clinical characteristics observed within a particular health plan. Additionally, sensitivity analyses can be included that allow end users to see the effect that certain variables have on the results. Creating user-friendly interfaces may also improve adoption and utility of the model.

Weaver: The benefits are, as I described, having a framework to think about economic or clinical end points relative to other therapies or other interventions. The challenges are biases within the model, nuances between the populations regarding prevalence of a condition or the cost of a therapy. If 2 payers are using the same model, but 1 payer pays 10% more for the product, being able to adjust the model to account for that is important. That lends itself to flexibility in the models and training for people to use the models so they can be accurate and useful and relevant. The biases can challenge the usability or the extrapolation of the results.

AJMC^®: How can a payer best work with a manufacturer to develop/validate an economic model?

Weaver: One is to pull results that are loaded into the model from their own data, so they can understand the prevalence of disease in their own population and the underlying assumptions within a model. They can also utilize their data for the cost of therapy or an outcome to more accurately portray how the model would apply to themselves. That can help validate what is in the model. If there is an opportunity to adjust the model, having their own
data points … is useful.

AJMC^®: At what point in a product life cycle would you like to receive a manufacturer’s economic model?

Weaver: Before launch, prospective pricing can help us understand how effective the therapy will have to be to be worth the price point or how that compares with other therapies in the space. It can be useful early on. Later in the launch cycle, after the products have been used and there is RWE to suggest how well the product works and what the discontinuation rates are, a Markov model developed with RWE can be very helpful for us to update and rethink how much value the product is bringing in the market and whether it’s delivering on its promise.

AJMC^®: What can be done to increase confidence in the findings of a manufacturer’s model?

Weaver: Educate people in the payer side and evidence committees on how these models can be useful to them. People poke holes in the model, because they think, “That’s not what I pay for the drug. I pay less for that medical outcome.” Help them understand that you can adjust those parts of the model and that the model itself is just like any other tool. If I use a hammer, the hammer isn’t better or worse; it’s the carpenter swinging the hammer that creates precision. If we train our decision-makers better on what is represented in the model, how the tools work, and how to make the tool effective, it will get better uptake.

REFERENCES

1. Brixner D, Biskupiak J, Oderda G, Burgoyne D, Malone DC, Arondekar B, Niyazov A. Payer perceptions of the use of real-world evidence in oncology-based decision making. J Manag Care Spec Pharm. 2021;27(8):1096-1105. doi:10.18553/jmcp.2021.27.8.1096
2. Arondekar B, Duh MS, Bhak RH, DerSarkissian M, Huynh L, Wang K, Wojciehowski J, Wu M, Wornson B, Niyazov A, Demetri GD. Real-world evidence in support of oncology product registration: a systematic review of new drug application and biologics license application approvals from 2015-2020. Clin Cancer Res. 2022;28(1):27-35. doi:10.1158/1078-0432.CCR-21-2639
3. Oderda G, Brixner D, Biskupiak J, Burgoyne D, Arondekar B, Deal L, Quek RGW, Niyazov A. Payer perceptions on the use of patient-reported outcomes in oncology decision making. J Manag Care Spec Pharm. 2022;28(2):188-195. doi:10.18553/jmcp.2021.21223
4. Biskupiak J, Oderda G, Brixner D, Burgoyne D, Arondekar B, Niyazov A. Payer perceptions on the use of economic models in oncology decision making. J Manag Care Spec Pharm. 2021;27(11):1560-1567. doi:10.18553/jmcp.2021.27.11.1560

Download PDF