A Value Proposition for Pragmatic Clinical Trials

Pragmatic clinical trials (PCTs), which test if and how interventions work in the real world, are popular and well funded. An assessment of the benefits and challenges of PCTs shows that they are a promising vehicle to enhance health care value.

PCTs are rigorous, feasible, and relevant to everyday practice. They typically rely on existing electronic data to rapidly assess programs, demonstrate if patient outcomes improve, and foster scientist–health care workforce partnerships.
Embedded in health care systems, PCTs can produce usable results quickly and inform important public health decisions.
Despite challenges, PCTs can deliver sizable gains to health care systems and society at large, but these gains are not yet fully realized.

_____

In 2001, the Institute of Medicine (IOM; now the National Academy of Medicine) released the influential report Crossing the Quality Chasm: A New Health System for the 21st Century.¹ It was a manifesto for redesigning US health care to deliver safe, timely, effective, efficient, equitable patient-centered care. Achieving these goals has required fundamentally rethinking how knowledge is produced, disseminated, and integrated into clinical care. It demands acknowledging that traditional randomized clinical trials (RCTs)—the gold standard for studying new interventions—provide evidence of efficacy under controlled conditions, but they often offer little or no guidance on how to bring proven care to patients.

Pragmatic clinical trials (PCTs) embedded in health care settings address this limitation. They differ from traditional RCTs not so much by their design (PCTs often are randomized as well) as by their purpose.² PCTs test if and how interventions work in the real world. PCTs have a growing foothold in research because they can produce quickly usable results, typically by relying on electronic health record data. Because PCTs are embedded in delivery systems, even null trial results are directly relevant to practice. They can inform real-world decisions by demonstrating that alternative existing treatments yield equivalent results or that a proposed intervention should not be pursued, and they can help shape other pragmatic studies and clinical improvements.

The recent 20th anniversary of the IOM report makes this the right time to reflect on how PCTs can contribute to research and, ultimately, care quality and patient-centered outcomes. By improving overall care, PCTs also can enhance health care value. According to Porter, value consists of optimizing health outcomes relative to cost and placing patients and other stakeholders at the center of a health care system’s activities.³ The question we should ask is: What is the worth of PCTs in enhancing health care value that is achieved through improved quality of care? In other words: How can PCTs support health care decision-making that leads to better real-world outcomes, and what is the likelihood that these trials will fulfill that role in the foreseeable future? The answer formulates a value proposition to guide investments in PCTs to ensure the high-quality care called for in the IOM report.

Origin

Interest in funding PCTs increased more than a decade ago through the establishment of the Health Care Systems Research Collaboratory, supported by the National Institutes of Health (NIH) Common Fund. Other funders, including the Patient-Centered Outcomes Research Institute (PCORI)⁴ and the Agency for Healthcare Research and Quality,⁵ started investing in PCTs soon thereafter. The purpose of the NIH Collaboratory was to involve health care system stakeholders (eg, patients, clinicians, system leaders, caregivers, payers) in setting research priorities, conducting PCTs, and disseminating best practices. Collaboratory-enabled research intends to use rigorous methods, be practical and relevant to everyday practice, rapidly assess and improve outcomes for patients, and foster scientist–health care workforce partnerships.⁶ These features of PCTs are shared by other transformative research such as the learning health care system, translational research, and dissemination and implementation science that similarly advance many of the aims of the IOM report. This movement, which for the past 20 years has sought to reimagine US health care based on the report’s principles, creates synergies and opportunities that can enhance PCTs by increasing stakeholder interest and participation.

Advantages and Challenges

Embedding studies into health care systems enables collaborative, results-based science that strives to be “better, faster, cheaper, and more responsive to current needs.”⁶ Such science uses existing electronic health records and gathers data generated through everyday contact with patients, which optimizes the capabilities of big data systems and may lessen the burden on clinical staff who contribute to science while performing their tasks. It is true that the goals of efficiency and relevance may not always be aligned and may be hard to pursue simultaneously in health care research and practice. Even so, the embedded nature of PCTs can help highlight where such alignment is possible and how to achieve it.

Relevance is one of PCTs’ main benefits. PCTs involve personnel who are in close touch with patient needs. PCTs address topics and critical care gaps where research can support rapid quality improvements.⁷ By answering urgent questions that matter to patients and physicians, PCTs can generate rigorous, applicable evidence to guide decision makers from leadership to frontline staff who need up-to-date knowledge for patient care. Investigators partnering with clinicians contribute expertise to care improvement and the ability to evaluate effectiveness in the context of care.⁷ This synergy fosters speedier acceptance of evidence-based interventions and translation into clinical care than with RCTs. PCTs can match health care system priorities and resources, increasing their appeal to leaders. Most important, asking questions that patients, clinicians, and leaders care about will make it more likely that the answer from PCTs has an impact.

Ultimately, it is the public that benefits most from PCTs. Society has a clear interest in research that promotes better health outcomes faster and is conducted with ethical and scientific integrity despite occurring outside the more controlled, artificial structure of RCTs.⁸ An added benefit of PCTs is that robust trial funding demands good stewardship of public monies supporting the trials. Often taking less time than traditional trials, PCTs promise to be more cost-effective when measured in cost per enrolled participant, which is typically high in traditional efficacy or explanatory clinical trials. PCTs can test clinical outcomes and strategies for implementation, dissemination, and spread of valuable interventions, therefore maximizing return on public investment.

The challenges of implementing PCTs should be acknowledged for a balanced view of their value. PCTs inevitably are influenced by the complexities of real-world practice, where treatment efficacy, quality-improvement efforts, and delivery and contextual changes affect trial results. When null results occur, distinguishing the ineffectiveness of the tested intervention from a poor fit with local needs or implementation failures can be difficult. Signals of benefit may be missed, along with opportunities to improve care quality.

The time required to design, get funding for, and execute a PCT may be unattractive to clinicians and health care system decision makers who seek urgent answers. Although PCTs are faster, less expensive, and more likely generalizable than traditional RCTs, the former are slower and more expensive than acting without evidence. Further, research-related tasks may strain the capacity of clinical teams already caring for patients.⁹ Relying on multiple stakeholder partnerships and embedding in health systems are strengths of PCTs, but unforeseen events such as leadership or staffing changes, electronic health record barriers, or conditions such as the COVID-19 pandemic can affect studies.¹⁰ Health care system leaders may be reluctant to participate in PCTs that ask important questions but may not lead to short-term quality or safety gains.⁹ Embedding PCTs in integrated delivery systems can facilitate successful trials but undermine the claim that findings are generalizable to other settings.

Conclusions

On balance, we feel that the growth in popularity of PCTs is well justified. This study design adds value to research because it reflects the conditions of everyday clinical care and is close to the needs and concerns of health care leaders, clinicians, and staff. This gives PCTs the strong potential to deliver sizable gains to health care systems and society at large.

The value of PCTs is less in the results observed to date—although they are significant—and more in the development of methods and the creation of a learning community of investigators. With 22 PCTs supported by the NIH Collaboratory across 1100 clinical sites,¹¹ as well as numerous additional trials supported by PCORI and other funders, we have learned many lessons on how best to conduct PCTs. The next challenge is focusing such lessons and refined methods on the right questions: those most important to health care system stakeholders and public health.

Perhaps the key question is whether ongoing funding and greater participation of investigators and health system stakeholders will produce more evident added value. Innovative strategies to strengthen investment in PCTs could reduce financial and administrative burdens for health care systems participating in trials through reimbursements or other practical assistance. Professional incentives could help make PCTs attractive to clinicians and feed an investigator pipeline. These and other creative approaches would encourage delivery systems to participate in PCTs that have significant public health value but no immediate payoff for the systems’ patients or staff.⁹

The PCT arena has seen enormous growth, although the benefits are still in development. Our experience indicates that this work spans the gamut, from learning about the most clinically relevant research questions to testing creative strategies and interventions to determining the potential for participatory research in developing PCTs. Although all signs indicate that PCTs are here to stay, we believe we still have much work to do to advance PCTs, evaluate what aspects of their design are essential or adaptable, and make the most of what they can offer to render the value proposition of PCTs a reality.

Author Affiliations: Kaiser Permanente Washington Health Research Institute (LP, LT, GES, EBL), Seattle, WA.

Source of Funding: This work was supported by the National Institutes of Health (NIH) Common Fund, through cooperative agreement U24AT009676-02 from the Office of Strategic Coordination within the Office of the NIH Director. The views presented here are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Author Disclosures: Dr Larson was employed by Kaiser Permanente Washington Health Research Institute at the time of writing this article and has received NIH grants but reports no conflicts with the current work. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (LP, LT, GES, EBL); acquisition of data (EBL); analysis and interpretation of data (EBL); drafting of the manuscript (LP, LT, GES); critical revision of the manuscript for important intellectual content (LP, LT, GES, EBL); provision of patients or study materials (EBL); obtaining funding (EBL); administrative, technical, or logistic support (EBL); and supervision (LP, EBL).

Address Correspondence to: Lorella Palazzo, PhD, Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA 98101. Email: Lorella.G.Palazzo@kp.org.

REFERENCES

1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. The National Academies Press; 2001.

2. Heagerty PJ, DeLong ER; for the NIH Health Care Systems Research Collaboratory Biostatistics and Study Design Core. Experimental designs and randomization schemes.In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. NIH Health Care Systems Research Collaboratory; 2021. doi:10.28929/002

3. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477-2481. doi:10.1056/NEJMp1011024

4. Pragmatic clinical studies. Patient-Centered Outcomes Research Institute. Updated March 1, 2022. Accessed August 1, 2022. https://www.pcori.org/research-results/pragmatic-clinical-studies

5. Agency for Healthcare Research and Quality. Accessed August 1, 2022. https://www.ahrq.gov/

6. Lauer MS. Time for a creative transformation of epidemiology in the United States. JAMA. 2012;308(17):1804-1805. doi:10.1001/jama.2012.14838

7. Health care systems interactions. NIH Collaboratory Living Textbook of Pragmatic Clinical Trials. June 16, 2020. Accessed June 16, 2020. https://rethinkingclinicaltrials.org/cores-and-working-groups/health-care-systems-interactions/

8. Kalkman S, van Thiel G, van der Graaf R, et al. The social value of pragmatic trials. Bioethics. 2017;31(2):136-143. doi:10.1111/bioe.12315

9. Platt R, Simon GE, Hernandez AF. Is learning worth the trouble? — improving health care system participation in embedded research. N Engl J Med. 2021;385(1):5-7. doi:10.1056/NEJMp2101700

10. Haff N, Choudhry NK. The promise and pitfalls of pragmatic clinical trials for improving health care quality. JAMA Netw Open. 2018;1(6):e183376. doi:10.1001/jamanetworkopen.2018.3376

11. NIH Collaboratory celebrates 10 successful years of Rethinking Clinical Trials with a new name and a refreshed living textbook. NIH Pragmatic Trials Collaboratory. January 10, 2022. Accessed August 1, 2022. https://rethinkingclinicaltrials.org/news/january-10-2022-nih-collaboratory-celebrates-10-successful-years-of-rethinking-clinical-trials-with-a-new-name-and-a-refreshed-living-textbook/