The article by Peikes et al in this issue of The American Journal of Managed Care (“Early Evaluations of the Medical Home: Building on a Promising Start”)1 critiques the findings of several recent studies of different models of “the medical home.” As lead investigators of one of these studies—the matched-pair, cluster-randomized evaluation of Guided Care—we were drawn to their discussion about the potential analytic effects of “clustering” (ie, the tendency for the outcomes of patients treated by the same physician or practice to be more similar than the outcomes of patients treated by different physicians or practices). For cluster-randomized studies, Peikes et al classify analyses that did not adjust for clustering as being “inconclusive” or of “uncertain statistical signifi cance” when the reported results were statistically significant (ie, P value <.05).
Their categorization of our findings is understandable. However, given that our evaluation enrolled only 7 matched pairs, it begs the question of how our analysis should account for clustering. More importantly, how should small, matched-pair, cluster-randomized studies of the future account for clustering, so that they can produce useful information about the effects of medical homes and other provider-based interventions?
Statisticians usually consider 2 strategies for analyzing matched-pair, cluster-randomized data: (1) site-level adjustment and (2) patient-level adjustment.2 For studies with small numbers of sites (in our case, matched pairs), the former approach is recommended. In this approach, the intervention-specific outcomes (or residuals, when covariate adjustment is required) within each matched pair are summarized into a scalar quantity (eg, mean). Statistical inference is then based solely on these pairs of numbers.
However, recent work by Imai et al3 argues that this approach can yield confi dence intervals and P values that are too conservative. In order to maximize the information content from small, matched-pair, cluster-randomized studies, it is essential that the inferential procedure be as precise as possible.
We intend to revisit our original analyses as well as the analytic strategy for the final results of the Guided Care trial that are to be reported later this year. We continue to
believe that small, matched-pair, cluster-randomized studies, which are more feasible within the context of our resource-constrained environment, can provide useful evidence to guide learning on patient-centered medical homes.
The authors would like to thank Constantine Frangakis, Thomas Louis, PhD, and Brian Caffo, PhD, from the Department of Biostatistics at the Johns Hopkins Bloomberg School of Public Health and Kosuke Imai, PhD, from the Department of Politics, Princeton University, for their assistance in the preparation of this editorial.
Daniel O. Scharfstein, ScD
Department of Biostatistics
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Cynthia M. Boyd, MD, MPH
Division of Geriatric Medicine and Gerontology
Johns Hopkins University School of Medicine
Center on Aging and Health
Baltimore, MD
Jennifer L. Wolff, PhD
Department of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Chad Boult, MD, MPH, MBA
Department of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health
Baltimore, MDAuthor Disclosures: The authors (DOS, CMB, JLW, CB) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Address correspondence to: Daniel O. Scharfstein, ScD, Professor, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, Rm E3547, Baltimore, MD 21205. E-mail: dscharf@jhsph.edu.1. Peikes D, Zutshi A, Genevro JL, Parchman ML, Meyers DS. Early evaluations of the medical home: building on a promising start. Am J Manag Care. 2012;18(2):105-116.
2. Hayes RM, Moulton LH. Cluster-Randomised Trials. Boca Raton, FL: Chapman & Hall/CRC Press; 2009:233-254.
3. Imai K, King G, Nall C. The essential role of pair matching in clusterrandomized experiments, with application to the Mexican Universal Health Insurance Evaluation. Stat Sci. 2009;24(1):29-53.
Racial/Ethnic Differences in Colorectal Cancer Screening in the US
August 4th 2025Data from the 2021 National Health Interview Survey showed racial/ethnic differences in colorectal cancer screening were due to demographic and socioeconomic factors, except for low colonoscopy use in Asian individuals.
Read More
Trends in Insulin Out-of-Pocket Costs and Use Disparities, 2008-2021
July 31st 2025Given trends in cost and use, insulin out-of-pocket cost reduction policies would be more efficient if they targeted members in high-deductible health plans with savings options and low-income patients.
Read More
Managed Care Reflections: A Q&A With Charles N. (Chip) Kahn III, MPH
July 30th 2025To mark the 30th anniversary of The American Journal of Managed Care (AJMC), each issue in 2025 includes a special feature: reflections from a thought leader on what has changed—and what has not—over the past 3 decades and what’s next for managed care. The August issue features a conversation with Charles N. (Chip) Kahn III, MPH, the president and CEO of the Federation of American Hospitals and a longtime member of the AJMC editorial board.
Read More