Early Evaluations of the Medical Home: Building on a Promising Start

Objectives:

To systematically review the current evidence on the patient-centered medical home (PCMH, or medical home), which aims to reinvigorate primary care and achieve the triple aim of better quality, improved experience, and lower costs.

Study Design:

Systematic review of quantitative evidence on the PCMH.

Methods:

Out of 498 studies published or disseminated from January 2000 to September 2010 on US-based interventions, 14 evaluations of 12 interventions met our inclusion criteria: (1) tested a practice-level intervention with 3 or more of 5 key PCMH components and (2) conducted a quantitative study of one of the triple aim outcomes or of healthcare professional experience. We synthesized fi ndings on interventions that were evaluated using rigorous methods. We also provide guidance to structure future evaluations to maximize learning.

Results:

The interventions most often cited to support the medical home can be viewed as precursors to the medical home. Evaluations of 6 of these interventions provided rigorous evidence on 1 or more outcomes. This evidence indicates some favorable effects on all 3 triple aim outcomes, a few unfavorable effects on costs, and many inconclusive results.

Conclusions:

Although the PCMH is a promising innovation, rigorous quantitative evaluations and comprehensive implementation analyses are needed to assess effectiveness and refi ne the model to meet stakeholders’ needs. Findings from future evaluations will help guide the substantial efforts practices and payers invest to adopt the PCMH with the goal of achieving the triple aim outcomes.

(Am J Manag Care. 2012;18(2):105-116)This article summarizes the current evidence on the patient-centered medical home (PCMH) and provides concrete suggestions for conducting more rigorous evaluations. We found:

• Most early evaluations tested precursors to the PCMH.

• Among the 14 evaluations reviewed, 6 provide rigorous evidence on at least 1 of the 3 triple aim outcomes (better quality, improved experience, and lower costs) or healthcare professional experience.

• Evidence indicates some favorable effects on all 3 triple aim outcomes, a few unfavorable effects on costs, and many inconclusive results.

• More rigorous quantitative evaluations and comprehensive implementation analyses are needed to assess effectiveness and refi ne the model to meet stakeholders’ needs.

Reinventing primary care is a task that is “far too important to fail”¹ and central to reforming healthcare delivery. The current healthcare system, with its incentives to furnish more care, has produced highly fragmented care that emphasizes specialty and acute care over coordination, patient centeredness, and population health management.^2-6 Although 93% of Americans want one place or doctor that provides primary care and coordinates care with specialists, only half report having such an experience.^7,8 The patientcentered medical home (PCMH) is a promising model that aims to reinvent primary care so that it is “accessible, continuous, comprehensive and coordinated and delivered in the context of family and community,”⁹ and, in so doing, improve the triple aim outcomes of quality, cost, and patient and family experience, as well as healthcare professional experience.

The medical home concept fi rst arose in the 1960s as a way of improving care for children with special needs, and policy interest outside of pediatrics grew over time.¹⁰ In 2007, primary care physician societies endorsed the joint principles of this primary care delivery model.⁹ Intrigued by the potential of the PCMH model, private insurers, major private and federal employers, provider organizations, Medicare, and state Medicaid agencies across the nation are rolling out pilots and demonstrations of different variants of the model. However, it will likely take many years before results of current evaluations become available. Transforming care will require recognizing and addressing many barriers to change using lessons from these evaluations.¹¹

PURPOSE

Against this backdrop, decision makers consider whether the evidence supporting the model is already strong enough to proceed with widespread adoption, or whether gathering additional evidence is warranted. To contribute to this discussion, the Agency for Healthcare Research and Quality (AHRQ) commissioned Mathematica Policy Research to systematically review quantitative evaluations of the medical home model to inform current efforts and provide guidance to structure future evaluations to maximize learning. Qualitative evaluations can provide powerful insights; however, we excluded them from this review because of our focus on outcomes and because evaluations rarely documented their implementation experiences. Given that interest in the model is recent, the expectation was that only precursors to the PCMH would have been evaluated so far. A longer paper provides a more detailed description of this review.¹²

The review limits synthesis of fi ndings to interventions evaluated using rigorous methods. While much can be learned from less rigorous, rapid-cycle evaluations of lowcost, individual components of the medical home, this review intends to fulfi ll stakeholders’ need for high-quality quantitative evidence on medical home interventions that test multiple components and are costly for payers and providers to implement.

Some readers may not consider an evidence review of the PCMH to be necessary, because they believe that the evaluations conducted to date, combined with the vast cross-sectional literature on the positive relationship between more primary care and better outcomes, provide suffi cient evidence to proceed with widespread adoption of the model. Others may feel that the model is being held to a higher standard than many clinical interventions that are currently being used without strong evidentiary support.

Historically, a number of promising healthcare interventionshave been shown not to actually work when evaluated using rigorous methods. For example, telephonic disease management seemed to address obvious problems in coordination and patient self-management, but a number of randomized trials showed many ineffective programs and pointed the way to refining the model to offer better integration with providers, more in-person contact, and careful focusing of efforts to

those most likely to benefit.^13-15 Similarly, rigorous evidence regarding the effectiveness of the PCMH model and how best to refine it is critical given the substantial investments this model requires.

This review makes 2 important methodological contributions. First, we limited the review to multicomponent interventions by requiring interventions to contain at least 3 of the 5 components in AHRQ’s defi nition of the PCMH model. Earlier reviews typically included results from interventions with as few as 1 component. One group of researchers found that only 1 of the 33 studies they reviewed was of an intervention modeled after the medical home, with the remaining studies testing selected components.¹⁶ Three others each reviewed the literature on individual components of the medical home, such as team-based care, rather than reviewing multicomponent interventions that more closely resemble the PCMH model.^17-19 Second, we limited the synthesis of the evidence to that generated by rigorous evaluations, which we assess using a systematic review. Three previous reviews did not consider the quality of the evidence.^20-22 Two reviews conducted a limited assessment by focusing on either comparison group studies¹⁶ or peer-reviewed studies,²² but neither assessed the strength of the analytical methods used by the studies or excluded weaker studies from their syntheses of the evidence.

METHODS

We began the review by fi rst identifying evaluations of interventions that met our inclusion criteria, then rating the rigor of these evaluations, and fi nally synthesizing the evidence on PCMH effectiveness using only rigorous evaluations.

Inclusion Criteria

We identifi ed 498 citations based on a search of published and gray literature from January 2000 to September 2010, inputs from experts in the fi eld, and a review of 100 relevant Web sites. Out of these citations, we selected 14 evaluations of 12 interventions that met the following criteria:

1. The evaluation tested a practice-level intervention in the United States with 3 or more of the 5 medical home components defi ned by AHRQ. AHRQ defines the PCMH as delivering care that is patient centered, comprehensive, coordinated, and accessible, and follows a systems-based approach to quality and safety. The defi nition (available at pcmh.ahrq.gov) also emphasizes the central role of health information technology, workforce development, and fundamental payment reform. It builds on the traditional definition of primary care established by the Institute of Medicine²³ and Barbara Starfield^24,25 and incorporates aspects of the expanded care model.^26,27 It is similar to the defi nition of the medical home emphasis on team-based care.

2. The evaluation used quantitative methods to examine effects on either (1) a triple aim outcome (quality of care, costs [or hospital or emergency department use, 2 major cost drivers], and patient experience) or (2) healthcare professional experience (refl ecting its importance in transforming primary care).

The first criterion excludes 2 studies of medical home interventions—the American Academy of Family Practice’s National Demonstration Project, which is often cited in the medical home literature, and the Illinois Medical Home Project—because rather than testing the effect of a medical home, they tested the effect of facilitation of a medical home (helping practices to become medical homes versus practices becoming medical homes on their own).

Rating the Rigor of the Evaluations

We developed a systematic approach to assess the rigor of the evaluations conducted to generate evidence on PCMH effectiveness. We drew broadly from the US Preventive Services Task Force review methods.²⁸ We also drew specifi c operational criteria from the What Works Clearinghouse review of educational interventions (which also typically use clustered designs like the many practice-level interventions reviewed here)²⁹ and from an evidence review of home-visiting programs for families with pregnant women and children.³⁰

Rather than give a global rating to each evaluation, we individually rated the internal validity of each analysis conducted as part of the evaluation as high, moderate, low, or excluded. We rated individual analyses because evaluations often used different designs, samples, and methods (and sometimes different subgroups of patients) to analyze different outcomes over varying follow-up periods. We did not factor generalizability (or external validity) into the rating because most interventions included in this review targeted a specifi c subset of primary care patients, were implemented in unique settings, and either purposefully selected practices or relied on them to volunteer; therefore, fi ndings from nearly all interventions have limited generalizability. We did, however, summarize the characteristics of patients and practice settings of rigorously evaluated interventions to alert decision makers to the possibility that fi ndings might differ in other contexts.

We rated each analysis using a sequence of criteria, starting with the most general (evaluation design) and ending with the most specifi c (such as whether the analysis controlled for outcome values before the start of the intervention (at baseline). Analyses were rated as excluded when the evaluation design or methods were not described in suffi cient detail to enable assessment. Analyses were rated low if they did not use a control or comparison group (such as those from pre-post and cross-sectional studies). Analyses from RCTs and nonexperimental comparison group evaluations were assessed for the strength of the methods to identify causal effects and produce unbiased estimates of the intervention’s effects, and were accordingly rated as high, moderate, or low.

Analyses from RCTs were given a high rating if they had:

• No systematic confounders

• No endogenous subgroups

• Low attrition

• Adjustment for any statistically signifi cant baseline differences in the outcome between the treatment and control groups. Analyses from comparison group evaluations and from RCTs with high attrition or endogenous subgroups were given a moderate rating if they had:

• No systematic confounders

• Baseline equivalence of the outcome between the treatment and comparison groups

• Adjustment for baseline outcomes.

Analyses from RCT and comparison group evaluations were given a low rating if they failed to meet criteria for high and moderate ratings.

Synthesizing Evidence With a High or Moderate Rating

We synthesized the evidence using only fi ndings from analyses rated as high or moderate. While each intervention represents an important effort to creatively improve healthcare delivery, we excluded from the synthesis fi ndings from analyses rated as low. It is possible that if these interventions were evaluated using better methods, the results might differ substantially. For example, results could change from suggesting an intervention did not work to suggesting it worked, or vice versa.

We categorized the fi ndings rated high and moderate as being (1) statistically signifi cant and favorable, (2) statistically significant and unfavorable, (3) inconclusive (that is, they fail to indicate whether or not the intervention worked) because they were not statistically signifi cant, or (4) inconclusive because their statistical signifi cance was uncertain due to lack of adjustment for clustering of patients within practices. While “inconclusive” may be a frustrating label, it accurately refl ects the lack of certainty about whether the intervention worked or not.

We considered findings that were not statistically significant to be inconclusive because we suspected that most evaluations had inadequate power to detect effects that might have existed because the practice-level interventions were implemented in too few practices (11 or fewer). A 2011 paper on power in PCMH evaluations estimates that, assuming a moderate amount of clustering, an intervention that is tested in 20 treatment (with 20 control) practices and targets all patients would need to reduce costs by a sizable 45% or more to have an 80% chance of detecting the effect.³¹ Even if costs were measured among the chronically ill, as was the case in many of these evaluations, the intervention might still need to reduce costs by 20% or more for the evaluation to have an 80% chance of detecting it. These are large effects for an intervention to achieve, and an evaluation would need even larger sample sizes to detect smaller, more plausible effects.

We also viewed fi ndings as inconclusive when evaluations of practice-level interventions did not correctly account for clustering of patients within practices, leaving their tests of statistical signifi cance inaccurate and the signifi cance of results uncertain. While we used published estimates of clustering to adjust statistical signifi cance for cost and service use when possible, there was too little published information for us to make similar adjustments for other outcomes.

STATE OF THE EVIDENCE

Evaluations to Date Assessed PCMH Precursors

Because the joint principles were released in 2007, and because it takes time to design, implement, and evaluate interventions, most interventions currently cited in support of the PCMH contain some, but not all, aspects of the PCMH and are best viewed as precursors to the PCMH. Most interventions essentially tested the addition of a care manager operating from within the primary care practice, rather than a fundamentally transformed practice (Table 1). While these early interventions included each of the 5 AHRQ PCMH components, they did so in a less integrated and comprehensive manner than current demonstrations do. This fi nding serves as a reminder that the evidence that is commonly cited on the PCMH is actually on PCMH precursors, and should be interpreted in that context. Furthermore, these precursor interventions differed considerably from one another.

Several Evaluations Comprehensively Assessed the Triple Aim Outcomes

Even among these early evaluations, 5 of the 14 were able to examine each of the triple aim outcomes (cost, quality, and patient experience). Understandably, only 5 evaluations examined patient experience, likely refl ecting the high cost of collecting survey data and the fact that these interventions predated the current interest in the PCMH, which emphasizes patient centeredness.

Many Evaluations Did Not Use Rigorous Methods

Table 2

eAppendix A

Overall, 6 of the 14 evaluations received a high or moderate rating for at least 1 outcome. Among the evaluations that examined a given outcome, typically only a subset used rigorous methods (). The lack of an appropriate comparison group was the most common reason for a low rating (see , available at www.ajmc.com). Appropriate comparison groups (such as those similar to the treatment group on baseline patient outcomes, as well as practice variables like the mix of patients, number of providers, and key infrastructure such as electronic health records) are important to establish what would have happened to the treatment group in the absence of the intervention, in other words, the counterfactual. An evaluation that compares patients in pioneering, high-performing practices that chose to become medical homes with patients in practices that had average performance at baseline may artifi cially make the intervention look more effective than it truly is. Two evaluations were excluded from the synthesis because they tested the intervention in a single practice; such a design represents an important opportunity to pilot a new intervention and break ground toward a larger evaluation, but cannot distinguish the effects of the intervention from the particular effects of the single treatment practice.

EVIDENCE ON PCMH EFFECTIVENESS

Table 3

eAppendix B

The rigorous evidence on the effectiveness of PCMH precursors contains some favorable effects on all triple aim outcomes, a few unfavorable effects on costs, and many inconclusive results. (presents the fi ndings; , available at www.ajmc.com, provides more detail). Below, we summarize the evidence on each outcome.

Improving the Quality of Care

Processes of Care. Of the 3 evaluations that provided rigorous evidence, only the evaluation of the Improving Mood- Promoting Access to Collaborative Treatment for Late-Life Depression (IMPACT) intervention found favorable effects. Evaluations of Geriatric Resources for Assessment and Care of Elders (GRACE) and Care Management Plus (CMP) did not adjust statistical signifi cance for clustering, so their fi ndings are inconclusive.

Health Outcomes. Of the 3 evaluations that provided rigorous evidence on health outcomes, 2 found that the interventions made some improvements. The evaluation of IMPACT reported the strongest evidence of such effects and the evaluation of GRACE found effects on some of the measures. The third evaluation, of Veterans Affairs Team-Managed Home-Based Primary Care (VA TM/HBPC), is inconclusive because the results were not statistically signifi cant.

Mortality. While mortality effects would not be expected in the general patient population over short follow-ups, they are theoretically possible in the high-risk patients served by most of these interventions. The results from the GRACE and CMP evaluations, which examined mortality among high-risk Medicare patients, were not statistically significant and are therefore inconclusive.

Reducing the Costs of Care

Costs (Including the Intervention). The evaluation of GRACE was the only one of 4 rigorous evaluations that found any evidence of savings, and these were limited to the evaluation’s high-risk subgroup of Medicare patients in the third (postintervention) year. The 23% savings were enough to offset cost increases for patients who were not high risk, leaving the intervention cost neutral that year. However, GRACE increased costs (by 28% and 14%) for its full sample of patients during both years of the intervention. Similarly, VA TM/HBPC increased costs by 12% during its 1-year operation. Evidence from the other 2 interventions, Guided Care and IMPACT, is inconclusive because results were not statistically signifi cant.

Hospital Use. One of the 5 rigorous evaluations on hospital use found that the intervention reduced the number of hospitalizations by 18% for all patients (GHS ProvenHealth Navigator, which served Medicare Advantage patients). In addition, GRACE and VA TM/HBPC had some favorable effects on the number of hospitalizations for high-risk subgroups of their enrollees; GRACE reduced hospitalizations by 40% and 44% in the second and third years, but results were not statistically significant in the first year. Similarly, VA TM/HBPC reduced readmissions by 22% in the fi rst 6 months, although the reduction was not sustained through the rest of the year as the results were no longer statistically signifi cant over 12 months. In contrast, the findings on Guided Care and CMP are inconclusive. Guided Care did not have a statistically signifi cant effect on the number of hospitalizations over the fi rst 8 or 20 months. In the case of CMP, results among all patients and the subgroup without diabetes were not statistically signifi cant, and results among the subgroup with diabetes had uncertain statistical signifi cance due to lack of adjustment for clustering, rendering all these findings inconclusive.

Emergency Department Use. The evaluation of GRACE is the only one of 3 rigorous evaluations to fi nd some favorable effects; the intervention reduced the number of emergency department visits by 24% among its target population of Medicare patients in the second year, driven by reductions of 35% among the high-risk Medicare patients. However, results for GRACE are inconclusive in the fi rst year because they were not statistically signifi cant. Similarly, evidence on Guided Care, where results were not statistically signifi cant, and CMP, where results were either not statistically signifi cant or had not been adjusted for clustering, is inconclusive.

Improving the Experience of Care

Patient and Caregiver Experience. Of the 3 evaluations that provided rigorous evidence on patient experience, 2 (VA TM/HBPC and IMPACT) found a preponderance of favorable effects. The third evaluation (Guided Care) did not adjust statistical significance for clustering, so its fi ndings are inconclusive.

The evaluation of VA TM/HBPC found favorable effects on some measures of caregiver experience. However, results for other measures were inconclusive, as were results for Guided Care, because they were either not statistically signifi cant or had uncertain statistical signifi cance due to lack of adjustment for clustering.

Healthcare Professional Experience. The lone evaluation to provide rigorous evidence on professional experience (Guided Care) was inconclusive because results either were not statistically signifi cant or had uncertain statistical significance due to lack of adjustment for clustering.

PLACING THE FINDINGS IN CONTEXT

Findings Are Less Favorable Than Most Prior Reviews

We found some promising results across all 3 triple aim outcomes; however, the majority of findings were inconclusive. The conclusions we draw are consistent with those of one review, which described the current evidence in favor of the medical home as “scant.”²² Our conclusions are more tentative than those of 3 others that claimed overwhelming evidence in support of the medical home.^16,20,21 We conclude that more work, including additional well-designed, well-implemented evaluations, is needed to guide decisions regarding this young and rapidly evolving model.

Initial Findings Reflect Unique Contexts and Populations

Table 4

All interventions were tested in practices that were part of larger delivery systems and targeted older and sicker-thanaverage patients (). As a caveat, we expect it may be harder to generate effects of the same size on costs and service use among healthier patients, who do not use many services.

Findings Suggest Older, Sicker Patients May Experience Greater Benefits

Of the 6 rigorous evaluations, 2 examined outcomes for different subgroups of patients among their target population of older or sicker patients. (CMP did so too, but the results are inconclusive due to lack of adjustment for clustering.) The evaluation of GRACE reported that even among its low-income, elderly patients, improvements were concentrated among the sickest patients. The evaluation of the VA TM/HBPC intervention found favorable effects among severely disabled patients but not among other high-risk patients; it is unclear whether this latter finding refl ects lack of power to detect effects (due to small samples) or a true lack of effects. These results, while limited, raise the question of whether focusing on older, sicker patients could be a useful approach for future evaluations that want to quickly show results. Theoretically, the highest risk patients present more actionable opportunities to providers to reduce service use and costs in the relatively short follow-up periods observed, because a medical home intervention is likely to reduce hospitalizations more for patients who are frequently hospitalized. Future evaluations might want to include separate analyses of effects for sicker patients, in whom effects may be concentrated. This subgroup analysis will reduce the likelihood of missing important benefi cial effects that are harder to detect, due to lack of power, among all patients.³¹ At the same time, this does not imply that the PCMH should be targeted only to patients with complex medical needs. The PCMH is a wholepractice intervention and is expected to improve care for all. It is critical not to confuse the goal and purpose of the intervention with suggestions for refi ning evaluations.

Decision Makers Should Consider Context When Interpreting Findings

The findings on effectiveness will differ if the full medical home model is implemented, and is done so with different practices, markets, and patients. For example, implementing the PCMH model in certain markets or delivery settings where there is overuse of care could produce results different from those in areas where there is underuse of care. Similarly, modifi cations of the interventions might alter outcomes. For example, it is possible that stronger fi nancial incentives to practices could improve outcomes. In addition, program designers may be able to identify areas in which effi ciency can be increased to achieve cost neutrality or generate savings (eg, carefully reviewing which team members can provide which parts of the interventions and deploying them accordingly).

Guidance to Improve the Future Evidence Base

This review highlights opportunities to improve the evidence base on the PCMH going forward. There is a large risk that research currently under way on PCMH interventions (not reviewed here) will fail to support decision makers’ needs for rigorous evidence. As we have noted, many early evaluations of precursors to the PCMH did not provide rigorous evidence. In addition, and perhaps more worrisome, a survey of 26 medical home pilots under way in 18 states concluded that only 40% of these pilots had well-developed evaluation plans. Among those with plans, about 40% planned to use a comparison group design, with the remaining ones planning to use pre-post designs, which typically provide weak evidence.⁷²

The challenges to conducting rigorous evaluations are not unique to the PCMH. In 2011, the Government Accountability Office described evaluations of 127 diverse healthcare interventions as having weak evaluation designs, limited generalizability, and not reporting on the outcomes of interest (in their case, quality and cost).⁷³ Below we describe a number of steps that can be taken to improve the evidence base.

Use of Stronger Evaluation Designs and Methods. Evaluators of PCMH interventions have a huge opportunity to fill the current knowledge gap and contribute to the ongoing learning on PCMH effectiveness. The challenge is to make sure the practices and patients in the intervention and comparison groups are comparable at baseline to distinguish effects of the medical home model from preexisting differences between the intervention and comparison groups. Evaluations should also use rigorous analytical methods, including adjusting analyses for clustering of patients within practices.31

Conduct Comprehensive Implementation Analyses. We found that most evaluations did not report how the intervention was implemented. While undertaking an implementation study requires additional expertise and resources, it adds tremendous value in identifying barriers and facilitators to improving outcomes, how fi ndings might generalize to other contexts, and ways to refi ne the model. Implementation evaluations can provide powerful insights on their own, as well as when combined with quantitative outcome studies (that is, a mixed-methods approach).⁷⁴ A mixed-methods approach is particularly informative when evaluating the PCMH model.⁷⁵

Test the Model in an Adequate Number of Practices and Measure Different Outcomes for Different Subgroups of Patients. Because the PCMH is a practice-level intervention, it needs to be tested in a large number of practices or else the evaluation might lack the statistical power to identify effects even when they might exist. As discussed above, measuring costs and service use among sicker patients provides the ability to detect smaller effects than among all patients. In contrast, measures for quality of care and patient experience typically take on a small number of values, resulting in less variation, and effects on these outcomes can more easily be detected among all patients.

Follow Outcomes for Longer Periods of Time. Evaluations examined outcomes for 1 to 3 years, with most following patients for 2 years. While most decision makers are eager to obtain results, given the dramatic changes many practices need to undergo to become medical homes, short-term results may be misleading. Consistent with this possibility, the GRACE evaluation substantially increased costs by 28% early in the evaluation, but became cost neutral in the third year. However, the VA TM/HBPC evaluation found that shortterm favorable effects on readmissions dissipated over time.

Improve Reporting and Documentation. Many evaluations were not documented well enough to assess the strength of their methods. To allow objective assessment of the evidence, evaluation results—even preliminary results or results from pilot studies—should be accompanied by a detailed description of the methods used.

Independently Evaluate the Models to Ensure Objectivity. Many evaluations were conducted by intervention developers. While this may be convenient, independent evaluations, or, as a next best alternative, peer review of evaluations conducted by intervention developers, would build a better evidence base.

Test the Model in Typical Practices and Among Typical Patients. All 6 interventions with rigorous evidence were tested in practices in larger delivery systems and on patients who were older or sicker than the average patient. Ideally, future research would test the model with a diverse set of practices, including independent practices that represent the primary care landscape in this country. Also, while testing effects for specifi c patients is appropriate for evaluating specifi c research questions, decision makers still require evidence of effectiveness for the general patient population.³¹

Examine a Core Set of Outcome Measures and Develop Standardized Measures of PCMH Components. Estimating effects on a standard list of outcome measures would enable a meta-analysis of fi ndings across different interventions. Such an analysis can dramatically improve the power to detect effects compared with individual evaluations, which are often underpowered. The body of evidence would also be improved if researchers used detailed, standardized measures of PCMH components and processes. Such measures would enable meta- analyses to discern which interventions are most effective in which settings and why. The Commonwealth Fund is convening a collaborative for medical home evaluators to support this type of uniform research infrastructure.⁷⁶

Measure Effects on All Triple Aim Outcomes and Healthcare Professional Experience. Improving one type of outcome may not warrant model adoption if it comes at the cost of deterioration in other outcomes. To examine the full range of outcomes, a number of barriers may need to be addressed, including payer concerns about confi dentiality of cost data, limited resources to collect and analyze multiple data sources, and lack of tools to measure certain outcomes. The recent release from AHRQ of the PCMH-Consumer Assessment of Healthcare Providers and Systems (PCMH-CAHPS) may address one such barrier and enable evaluators to more easily measure patient experience.

Explore Other Approaches to Conceptualize and Evaluate PCMH Interventions. A growing number of studies in the past decade have shown that healthcare interventions can be viewed as complex interventions within a complex adaptive system, similar to processes in ecology, computer science, and organizational science. The complex adaptive system framework can be used to design implementation studies of medical home interventions.^77-79 For example, quality of care delivered by a practice can be conceptualized as a system-level property that arises in part from the interactions among the members of the practice.⁸⁰ Therefore, relationships among practice team members become key levers for performance improvement and may help explain why an intervention did or did not work. The framework also emphasizes the importance of the external environment, such as the functioning of the medical neighborhood. For example, a recent paper’s modeling of healthcare delivery across a network of providers could be used to develop measures of the environment within which a PCMH operates.⁸¹

Insights from complexity science could also be used to improve the design of quantitative evaluations of the PCMH. For example, quantitative measures of the internal and external environment might be useful to select comparison practices that closely resemble the treatment practices. More generally, from a complexity framework, attempts to isolate the relative contributions of individual components of the medical home are ill advised and are likely to result in misleading fi ndings because these components are dependent on one another to achieve the desired outcomes of the medical home model. By moving away from a mechanistic and reductionist perspective, complexity frameworks and other new approaches may help us conduct better evaluations of the PCMH.

LOOKING FORWARD

The medical home model is a promising approach to reinvigorate primary care. To ensure that the substantial costs for practices and payers of implementing the model are worthwhile, many decision makers need rigorous assessments of the model’s likely benefi ts, as well as guidance on how to operationalize and refine the model. Such evidence can guide the way primary care is transformed.Acknowledgments

We wish to thank a number of people for their assistance with this review. The authors of many of the studies patiently answered questions about the interventions and their study methods. Kimberly Smith, PhD, Melissa Azur, PhD, and Kristin Geonnotti, PhD, at Mathematica Policy Research provided in-depth analyses of some of the papers included in this review. Michael Barr, MD, MBA, FACP, at the American College of Physicians and Randall Brown, PhD, Christopher Trenholm, PhD, Silvie Colman, PhD, Brian Goesling, PhD, and Tim Novak, MS, at Mathematica Policy Research provided helpful comments and guidance during the development of this review. The opinions expressed in this document are those of the authors and do not refl ect the offi cial position of Agency for Healthcare Research and Quality or the US Department of Health and Human Services.

Author Affiliations: From Mathematica Policy Research (DP, AZ), Prince ton, NJ; Agency for Healthcare Research and Quality (JLG, MLP, DSM), Rockville, MD.

Funding Source: This project was funded under contracts HHSA 290200900019I/HHSA29032002T and HHSA290200900019I/HHSA29032005T from the Agency for Healthcare Research and Quality, US Department of Health and Human Services.

Author Disclosures: The authors (DP, AZ, JLG, MLP, DSM) report no relationship or fi nancial interest with any entity that would pose a confl ict of interest with the subject matter of this article.

Authorship Information: Concept and design (DP, AZ, JLG, DSM); acquisition of data (DP, AZ); analysis and interpretation of data (DP, AZ, JLG, MLP, DSM); drafting of the manuscript (DP, AZ, JLG, MLP, DSM); critical revision of the manuscript for important intellectual content (DP, AZ, JLG, MLP, DSM); statistical analysis (AZ, DP); obtaining funding (DP); administrative, technical, or logistic support (JLG, DSM); and supervision (DP, AZ, DSM).

Address correspondence to: Deborah Peikes, Mathematica Policy Research, POB 2393, Princeton, NJ 08543. E-mail: dpeikes@mathematica-mpr.com.1. Meyers DS, Clancy CM. Primary care: too important to fail. Ann Intern Med. 2009;150(4):272-273.

2. Berenson RA, Rich EC. US approaches to physician payment: the deconstruction of primary care. J Gen Intern Med. 2010;25(6):613-618.

3. Bodenheimer T, Pham HH. Primary care: current problems and proposed solutions. Health Aff (Millwood). 2010;29(5):799-805.

4. Dentzer S. Reinventing primary care: a task that is far “too important to fail” [published correction appears in Health Aff (Millwood). 2010; 29(6):1275]. Health Aff (Millwood). 2010;29(5):757.

5. Rittenhouse DR, Shortell SM, Fisher ES. Primary care and accountable care—two essential elements of delivery-system reform. N Engl J Med. 2009;361(24):2301-2303.

6. Howell JD. Refl ections on the past and future of primary care. Health Aff (Millwood). 2010;29(5):760-765.

7. Schoen C, Osborn R, Doty MM, Bishop M, Peugh J, Murukutla N. Toward higher-performance health systems: adults’ health care experiences in seven countries, 2007. Health Aff (Millwood). 2007;26(6): w717-w734.

8. Stremikis K, Schoen C, Fryer A-K. A Call for Change: The 2011 Commonwealth Fund Survey of Public Views of the U.S. Health System. New York: The Commonwealth Fund; April 2011.

9. American Academy of Family Physicians, American Academy of Pediatrics, American College of Physicians, American Osteopathic Association. Joint Principles of the Patient-Centered Medical Home. http://www.aafp.org/online/etc/medialib/aafp_org/documents/policy/fed/jointprinciplespcmh0207.Par.0001.File.dat/022107medicalhome.pdf. Published February 2007. Accessed September 14, 2010.

10. Kilo CM, Wasson JH. Practice redesign and the patient-centered

medical home: history, promises, and challenges. Health Aff (Millwood).

2010;29(5):773-778.

11. Landon BE, Gill JM, Antonelli RC, Rich EC. Prospects for rebuilding primary care using the patient-centered medical home. Health Aff (Millwood). 2010;29(5):827-834.

12. Peikes D, Zutshi A, Smith K, Azur M, Genevro J, Meyers D. The Medical Home: What Do We Know, What Do We Need to Know? a Review of the Earliest Evidence on the Effectiveness of the Patient-Centered Medical Home Model. Final paper submitted to the Agency for Healthcare Research and Quality. Princeton, NJ: Mathematica Policy Research; 2012.

13. McCall N, Cromwell C. Results of the Medicare Health Support disease-management pilot program. N Engl J Med. 2011;365(18):1704-1712.

14. Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA. 2009;301(6):603-618.

15. Peikes D, Peterson G, Brown R, Schore J, Razafi ndrakoto C. Results from a radical makeover of a care coordination program show how program design affects success in reducing hospitalizations and costs: evidence from a randomized controlled trial before and after key changes in program design. Paper presented at: AcademyHealth Annual Research Meeting; June 27, 2010; Boston, MA.

16. Homer CJ, Klatka K, Romm D, et al. A review of the evidence for the medical home for children with special health care needs. Pediatrics. 2008;122(4):e922-e937.

17. Rosenthal TC. The medical home: growing evidence to support a new approach to primary care. J Am Board Fam Med. 2008;21(5): 427-440.

18. Robert Graham Center. The Patient Centered Medical Home: History, Seven Core Features, Evidence and Transformational Change. http://www.aafp.org/online/etc/medialib/aafp_org/documents/about/pcmh.Par.0001.File.dat/PCMH.pdf. Published November 2007. Accessed July 1, 2011.

19. DePalma JA. Evidence to support medical home concept for children with special health care needs. Home Health Care Management & Practice. 2007;19(6):473-475.

20. Grumbach K, Grundy P. Outcomes of Implementing Patient Centered Medical Home Interventions: A Review of the Evidence From Prospective Evaluation Studies in the United States. Patient Centered Primary Care Collaborative. http://www.pcpcc.net/fi les/evidence_outcomes_in_pcmh.pdf. Published November 2010. Accessed February 3, 2011.

21. Fields D, Leshen E, Patel K. Analysis & commentary: driving quality gains and cost savings through adoption of medical homes. Health Aff (Millwood). 2010;29(5):819-826.

22. Friedberg MW, Lai DJ, Hussey PS, Schneider EC. A guide to the medical home as a practice-level intervention. Am J Manag Care. 2009; 15(10)(suppl):S291-S299.

23. Donaldson MS, Yordy KD, Lohr KN, Vanselow NA, eds; Committee on the Future of Primary Care, Division of Health Care Services, Institute of Medicine. Primary Care: America’s Health in a New Era.Washington, DC: National Academies Press; 1996.

24. Starfi eld B. Is the Patient-Centered Medical Home the Same as "Primary Care”? Measurement Issues. http://ncvhs.hhs.gov/080520p01.pdf. Published May 20, 2008. Accessed December 6, 2011.

25. Starfi eld B. Primary Care: Concept, Evaluation, and Policy. New York: Oxford University Press; 1992.

26. Barr VJ, Robinson S, Marin-Link B, et al. The expanded Chronic Care Model: an integration of concepts and strategies from population health promotion and the Chronic Care Model. Hosp Q. 2003;7(1):73-82.

27. Glasgow RE, Orleans CT, Wagner EH. Does the chronic care model serve also as a template for improving prevention? Milbank Q. 2001; 79(4):579-612.

28. Harris RP, Helfand M, Woolf SH, et al; Methods Work Group, Third US Preventive Services Task Force. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20(3)(suppl):21-35.

29. What Works Clearinghouse. Procedures and Standards Handbook. Version 2.1. http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_ procedures_v2_1_standards_handbook.pdf. Accessed November 22, 2011.

30. Mathematica Policy Research. How effective is home visiting? http://www.mathematica-mpr.com/EarlyChildhood/homvee.asp. Accessed November 21, 2011.

31. Peikes D, Dale S, Lundquist E, Genevro J, Meyers D. Choosing the sample and sample size for medical home evaluations: how to ensure that studies can answer the key research questions. Paper presented at: Third National Medical Home Summit; March 14, 2011; Philadelphia, PA.

32. Hostetter M. Quality Matters Case Study: Aetna’s embedded case managers seek to strengthen primary care. The Commonwealth Fund. August/September 2010. http://www.commonwealthfund.org/Content/Newsletters/Quality-Matters/2010/August-September-2010/Case-Study.aspx. Accessed January 28, 2011.

33. Dorr DA, Wilcox AB, Brunker CP, Burdon RE, Donnelly SM. The effect of technology-supported, multidisease care management on the mortality and hospitalization of seniors. J Am Geriatr Soc. 2008;56(12):2195-2202.

34. Domino ME, Humble C, Lawrence WW Jr, Wegner S. Enhancing the medical homes model for children with asthma. Med Care. 2009;47(11): 1113-1120.

35. Lodh M; Mercer Government Human Services Consulting. ACCESS cost savings—state fi scal year 2004 analysis. Letter to Jeffrey Simms, State of North Carolina, Offi ce of Managed Care, Division of Medical Assistance. 2005.

36. Ricketts TC, Greene S, Silberman P, Howard H, Poley S. Evaluation of Community Care of North Carolina Asthma and Diabetes Initiative: January 2000-December 2002. A report submitted under a contract agreement with the Foundation for Advanced Health Programs, Inc. UNC contract # 5-35174. http://www.shepscenter.unc.edu/research_programs/health_policy/Access.pdf. Published April 15, 2004. Accessed January 31, 2011.

37. Steiner BD, Denham AC, Ashkin E, Newton WP, Wroth T, Dobson LA Jr. Community Care of North Carolina: improving care through community health networks. Ann Fam Med. 2008;6(4):361-367.

38. Wilhide S, Henderson T. Community Care Of North Carolina: A Provider-Led Strategy for Delivering Cost-Effective Primary Care to Medicaid Benefi ciaries. American Academy of Family Physicians. http://www.aafp.org/online/etc/medialib/aafp_org/documents/policy/state/medicaid/ncfull.Par.0001.File.tmp/ncfullreport.pdf. Published June 2006. Accessed January 31, 2011.

39. Gilfi llan RJ, Tomcavage J, Rosenthal MB, et al. Value and the medical home: effects of transformed primary care. Am J Manag Care. 2010;16(8):607-614.

40. Graff T. ProvenHealth Navigator ICSI update May 2009. Geisinger Health System. http://www.icsi.org/icsi_colloquium_-_2009_/proven_health_navigator_.html. Accessed January 31, 2011.

41. Paulus RA, Davis K, Steele GD. Continuous innovation in health care: implications of the Geisinger experience. Health Aff (Millwood). 2008;27(5):1235-1245.

42. Steele GD, Haynes JA, Davis DE, et al. How Geisinger’s advanced medical home model argues the case for rapid-cycle innovation. Health Aff (Millwood). 2010;29(11):2047-2053.

43. Bielaszka-DuVernay C. The “GRACE” model: in-home assessments lead to better care for dual eligibles. Health Aff (Millwood). 2011;30(3): 431-434.

44. Counsell SR, Callahan CM, Tu W, Stump TE, Arling GW. Cost analysis of the Geriatric Resources for Assessment and Care of Elders care management intervention. J Am Geriatr Soc. 2009;57(8):1420-1426.

45. Counsell SR, Callahan CM, Clark DO, et al. Geriatric care management for low-income seniors: a randomized control trial. JAMA. 2007; 298(22):2623-2633.

46. Counsell SR, Callahan CM, Buttar AB, Clark DO, Frank KI. Geriatric Resources for Assessment and Care of Elders (GRACE): a new model of primary care for low-income seniors. J Am Geriatr Soc. 2006;54(7): 1136-1141.

47. Group Health News. Medical home pays off with improved care and reduced costs. http://www.ghc.org/news/news.jhtml?reposid=%2fcommon%2fnews%2fnews%2f20100504-medicalHome.html. Published May 11, 2010. Accessed January 31, 2011.

48. Reid RJ, Coleman K, Johnson EA, et al. The group health medical home at year two: cost savings, higher patient satisfaction, and less burnout for provider. Health Aff (Millwood). 2010;29(5):835-843.

49. Reid RJ, Fishman PA, Yu O, et al. Patient-centered medical home demonstration: a prospective, quasi-experimental, before and after evaluation. Am J Manag Care. 2009;15(9):e71-e87.

50. Boult C, Reider L, Leff B, et al. The effect of guided care teams on the use of health services: results from a cluster-randomized controlled trial. Arch Intern Med. 2011;171(5):460-466.

51. Boyd CM, Reider L, Frey K, et al. The effects of guided care on the perceived quality of health care for multi-morbid older persons: 18-month outcomes from a cluster-randomized controlled trial. J Gen Intern Med. 2010;25(3):235-242.

52. The Johns Hopkins University. Guided Care Web site. http://www.guidedcare.org/index.asp. Published 2010. Accessed November 15, 2010.

53. Leff B, Reider L, Frick KD, et al. Guided care and the cost of complex health care: a preliminary report. Am J Manag Care. 2009;15(8):555-559.

54. Marsteller JA, Hsu YJ, Reider L, et al. Physician satisfaction with chronic care processes: a cluster-randomized trial of guided care. Ann Fam Med. 2010;8(4):308-315.

55. Wolff JL, Rand-Giovanetti E, Palmer S, et al. Caregiving and chronic care: the guided care program for family and friends. J Gerontol A Biol Sci Med Sci. 2009;64(7):785-791.

56. Wolff JL, Rand-Giovanetti ER, Boyd CM, et al. Effects of guided care on family caregivers. Gerontologist. 2010;50(4):459-470.

57. Hunkeler EM, Katon W, Tang L, et al. Long term outcomes from the IMPACT randomised trial for depressed elderly patients in primary care. BMJ. 2006;332(7536):259-263.

58. University of Washington, Department of Psychiatry & Behavioral Sciences. Evidence-based depression care. IMPACT Web site. http:// impact-uw.org/. Accessed December 8, 2010.

59. Levine S, Unützer J, Yip J, et al. Physicians’ satisfaction with a collaborative disease management program for late-life depression in primary care. Gen Hosp Psychiatr. 2005;27(6):383-391.

60. Unützer J, Katon WJ, Williams JW Jr, et al. Improving primary care for depression in late life: the design of a multi-center randomized trial. Med Care. 2001;39(8):785-799.

61. Unützer J, Katon WJ, Callahan CM, et al; IMPACT Investigators. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002;288(22): 2836-2845.

62. Unützer J, Katon WJ, Fan MY, et al. Long-term cost effects of collaborative care for late-life depression. Am J Manag Care. 2008;14(2):95-100.

63. McCarthy D, Nuzum R, Mika S, Wrenn J, Wakefi eld M. The North Dakota Experience: Achieving High-Performance Health Care Through Rural Innovation and Cooperation. The Commonwealth Fund Commission on a High Performance Health System. http://www.common wealthfund.org/Content/Publications/Fund-Reports/2008/May/The-North-Dakota-Experience--Achieving-High-Performance-Health-Care-Through-Rural-Innovation-and-Coo.aspx. Published May 2008. Accessed January 31, 2011.

64. Palfrey JS, Sofi s LA, Davidson EJ, Liu J, Freeman L, Ganz ML; Pediatric Alliance for Coordinated Care. The Pediatric Alliance for Coordinated Care: evaluation of a medical home model. Pediatrics. 2004;113(5)(suppl):1507-1516.

65. Silvia TJ, Sofi s LA, Palfrey JS. Practicing Comprehensive Care: A Physician’s Manual for Implementing a Medical Home for Children With Special Health Care Needs. Boston, MA: Institute for Community Inclusion, Children’s Hospital; January 2000. http://www.communityinclusion. org/article.php?article_id=193&type=topic&id=2. Accessed January 31, 2011.

66. AcademyHealth State Health Research and Policy Interest Group. The Pennsylvania Chronic Care Initiative. Presentation at the Annual AcademyHealth Research Meeting. http://www.academyhealth.org/ files/interestgroups/shrp/SHRP_Breakfast_2009_Magistro.pdf. Published February 3, 2009. Accessed January 31, 2011.

67. Chronic Care Management, Reimbursement and Cost Reduction Commission. Prescription for Pennsylvania. Right State, Right Plan, Right Now. Strategic plan. http://health-equity.pitt.edu/931/1/ChronicCareCommissionReport.pdf. Published 2008. Accessed January 31, 2011.

68. Houy M; for the Maryland Health Quality and Cost Council. The Pennsylvania Chronic Care Initiative. Appendix. http://dhmh.maryland.gov/mhqcc/materials/council/121208/PA_appendix_121208.pdf. Accessed January 31, 2011.

69. Torregrossa AS. Pennsylvania’s Efforts to Transform Primary Care.Governor’s Offi ce of Health Care Reform, Commonwealth of Pennsylvania. http://www.cthealthpolicy.org/webinars/20100223_atorregrossa_webinar.pdf. Published January 31, 2007. Accessed January 31, 2011.

70. Department of Veterans Affairs. Home-Based Primary Care Program.VHA Handbook 1141.01. Transmittal sheet. http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1534. Published January 31, 2007. Accessed August 30, 2007.

71. Hughes SL, Weaver FM, Giobbie-Hurder A, et al; Department of Veterans Affairs Cooperative Study Group on Home-Based Primary Care. Effectiveness of team-managed home-based primary care: a randomized multicenter trial. JAMA. 2000;284(22):2877-2885.

72. Bitton A, Martin C, Landon BE. A nationwide survey of patient centered medical home demonstration projects. J Gen Intern Med. 2010; 25(6):584-592.

73. US Government Accountability Offi ce. Value in Health. Key Information for Policymakers to Assess Efforts to Improve Quality While Reducing Costs. Report to Congressional Requesters. http://www.gao.gov/new.items/d11445.pdf. Published July 2011. Accessed July 26, 2011.

74. Creswell JW, Klassen AC, Plano Clark VL, Smith KC; for the Office of Behavioral and Social Sciences Research. Best practices for mixed methods research in the health sciences. National Institutes of Health.http://obssr.od.nih.gov/mixed_methods_research. Published August 2011. Accessed December 1, 2011.

75. Crabtree BF, Chase SM, Wise CG, et al. Evaluation of patient centered medical home practice transformation initiatives. Med Care. 2011; 49(1):10-16.

76. The Commonwealth Fund. The Commonwealth Fund Connection. The Patient-Centered Medical Home Evaluators’ Collaborative. http://www.commonwealthfund.org/Content/Newsletters/The-Commonwealth-Fund-Connection/2011/Mar/March-18-2011/Announcements/Patient-Centered-Medical.aspx. Accessed April 15, 2011.

77. Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A Guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. 2011;6:99.

78. Nutting PA, Crabtree BF, Stewart EE, et al. Effect of facilitation on practice outcomes in the National Demonstration Project model of the patient-centered medical home [published correction appears in Ann Fam Med. 2010;8(4):369]. Ann Fam Med. 2010;8(suppl 1):S33-S44, S92.

79. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research fi ndings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

80. Lanham HJ, McDaniel RR Jr, Crabtree BF, et al. How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Saf. 2009;35(9): 457-466.

81. Parchman ML, Scoglio CM, Schumm P. Understanding the implementation of evidence-based care: a structural network approach. Implement Sci. 2011;6:14-23.

Download PDF: Early Evaluations of the Medical Home: Building on a Promising Start