An Evaluation of a Care Coaching and Provider Referral Intervention for Behavioral Health Needs

ABSTRACT

Objectives: To evaluate changes in health care spending and utilization associated with a telehealth-based care coach–supported and behavioral health (BH) provider referral intervention in the United States.

Study Design: Observational retrospective cohort study with propensity score matching of treated and control groups.

Methods: Difference-in-differences (DID) analysis was used to calculate per-member per-month (PMPM) savings and changes in utilization in a treated group relative to matched controls over 36 months. The study included 1800 adults with substance use disorder (SUD), anxiety, or depression who were eligible for the intervention. Treated members (n = 900) graduated from the program. Matched control members (n = 900) were eligible but never enrolled. Primary outcomes included all-cause and disease-attributable health care cost and utilization PMPM, categorized by place of service.

Results: There were statistically significant reductions in total all-cause medical costs of $485 PMPM (P < .001) and a 66% pre-post reduction in inpatient encounters, with $488 PMPM DID savings for inpatient admissions (P < .001) among the treated cohort compared with the control cohort over 36 months. Conversely, there were statistically significant cost increases ($110 PMPM; P < .001) for all-cause office visits in the treated cohort compared with the control cohort. Similar results were seen in SUD-attributable and BH-attributable costs.

Conclusions: Although the results could be affected by unmeasured confounding, they suggest that care coaching interventions that offer BH provider referrals may produce long-term savings, reductions in avoidable utilization, and increases in targeted services to treat BH conditions. Rigorous evaluations are needed to confirm these findings.

Am J Manag Care. 2022;28(12):644-652. https://doi.org/10.37765/ajmc.2022.89274

Takeaway Points

A care coaching intervention that offers behavioral health (BH) provider referrals produced significant long-term savings, reductions in avoidable inpatient utilization, and increases in office visits and targeted services to treat BH conditions.

• All-cause medical savings were driven by reductions in inpatient cost and utilization. Conversely, there were increases in all-cause office visits and BH-attributable costs.
• Savings were sustained throughout the 24-month postindex time period, indicating the durability of the program.

These results provide support for a shift toward clinical integration of BH interventions to achieve long-term savings, which is crucial as the US health care system moves toward value-based care with an increased focus on BH interventions.

Among an estimated 1 in 5 Americans (46.6 million in 2017) with behavioral health (BH) conditions, only about 43% will receive BH care each year.¹ During the COVID-19 pandemic, prevalence of BH conditions such as anxiety, depression, and substance use doubled,² and care for these conditions became more fragmented.³ Stigma, lack of access, cost, lack of clinical incentives to diagnose and treat BH, and a shortage of qualified providers all contribute to this care gap.⁴ BH conditions, including depression, anxiety, and substance use disorders (SUDs), are major drivers of cost in the US health care system and are associated with a doubling or tripling of total cost of care.⁵

The use of telehealth interventions for the treatment of BH conditions has expanded substantially over the last decade, with even more rapid change to this landscape occurring because of the COVID-19 pandemic.⁶ Several recent reports summarize the adaptation of practice and suggest that remote delivery is likely to remain common after the pandemic subsides.^7-10 Evidence supporting these telehealth interventions lags behind the growth of the sector, and there have been very few formal evaluations of the cost impacts of these interventions.^11-13 This report summarizes an evaluation of a BH intervention that combines telephonic-based care coaching with connecting members to BH providers to address behavioral issues that are associated with high health care costs.

METHODS

Description of Program

Ontrak is a national organization that offers an integrated provider-led intervention for individuals with unaddressed BH conditions and chronic diseases. Ontrak contracts with health plans, including commercial and Medicare Advantage lines of business, to serve clinically complex, high-cost members who have anxiety, depression, and/or SUD and comorbid acute and chronic physical conditions such as hypertension, chronic obstructive pulmonary disease, and chronic pain. Eligible members enroll in a 52-week BH telephonic care coaching program designed to improve health outcomes. Trained care coaches work with members to establish care treatment pathways, activate durable health-seeking behavior by setting personal health goals, and drive toward durable behavior change to prevent large but avoidable costs associated with health care utilization. The approach is based on evidence-based coaching standards such as motivational interviewing,¹⁴ self-determination theory,¹⁵ and person-centered care.¹⁶ Health coaches connect individuals to BH providers (virtual or in person) for therapy and/or medication management. Program completion was defined as (1) 365 days of enrollment; (2) primary care provider visit completed; (3) at least 1 behavior changed related to self-identified goal, including decreased substance use, improved diet, or social risk resolution, among others; and (4) member-specific adherence to provider follow-ups and physician recommendations.

In this article, we test the hypothesis that care coaching and provider referral models improve health-related outcomes while lowering costs.

Study Design

This longitudinal, retrospective observational study compared cost and utilization between an enrolled group of members who completed the Ontrak program (treatment cohort) and a propensity score–matched control cohort of members who were eligible for the Ontrak program but never enrolled. The preindex period was the 12 months preceding the enrollment month. The postindex period began with the enrollment day (simulated for control members) and continued thereafter for 24 months post enrollment (Figure 1).

Data Source

We used a proprietary administrative claims data set containing health plan eligibility history and longitudinal member data from regional and national health plans contracted with Ontrak, representing commercial and Medicare Advantage plans in the Northeastern, Southern, Midwestern, and Western regions of the United States.

Study Population

Inclusion criteria were (1) continuous health plan eligibility for 36 months; (2) diagnosis of SUD, depression, or anxiety; (3) being at least 18 and younger than 85 years on the date of enrollment; and (4) meeting a cost threshold of $7500 in health care–related costs during the 12-month preindex period. Exclusion criteria included specific clinical conditions or procedures and facility and professional claims exceeding $100,000 within the last 12 months. Inclusion and exclusion criteria are described in eAppendix Table 1 (eAppendix available at ajmc.com); clinical International Classification of Diseases, Ninth Revision (ICD-9) and Tenth Revision (ICD-10) exclusion codes are available upon request. Except for continuous health plan coverage, these inclusion and exclusion criteria also reflect Ontrak program eligibility.

Individuals in the treatment cohort were members who enrolled in the Ontrak program between January 1, 2014, and September 30, 2018, and graduated from the program (n = 900), allowing for continuous eligibility requirements (Figure 1). For treated members, the index date was the date that they enrolled in the program. For each control group member, a simulated enrollment date was set to the date they were first eligible to enroll plus 119 days, the mean time to enrollment for individuals in the program. Individuals in the control cohort were propensity score matched to members meeting Ontrak program and study eligibility criteria who never enrolled in the program.

Primary End Points

Primary end points were health care resource utilization and plan-paid costs for enrolled and nonenrolled members during preindex and postindex periods. All-cause health care costs included the sum of all medical costs associated with the utilization of health care paid by the health plan during the preindex and postindex periods and were adjusted to 2020 US$ based on US Census Bureau inflation ratios. Disease-attributable costs were defined as the medical costs associated with claims identified as SUD, anxiety, or depression based on ICD-9 or ICD-10 diagnosis codes; BH-attributable costs were defined as costs associated with BH services identified by a list of Current Procedural Terminology and Healthcare Common Procedure Coding System codes (code lists available upon request). Costs included claims categorized by place of service codes as inpatient hospitalizations, emergency department (ED) visits, physician office visits, and outpatient visits (utilization that occurs in a hospital outpatient location, which could include blood work, diagnostic test workups, or other nonemergency procedures). Cost savings are reported as gross savings.

All-cause utilization was defined as the sum of encounters for each member associated with inpatient hospitalization, ED visits, physician office visits, or outpatient visits. Disease-attributable utilization applied the same diagnosis code criteria as the cost variables. Costs and utilization were normalized to per-member per-month (PMPM) values for the preindex and postindex periods (12 months and 24 months, respectively). We report outcomes among individuals with at least 1 utilization in the preindex or postindex time periods, as well as for all members (eAppendix Table 2 [A-D]).

Covariates

Patient demographic and clinical characteristics are shown in Table 1.^17,18 Clinical characteristics were further quantified using the Elixhauser comorbidity index, which categorizes and weights comorbidities to provide a score related to the likelihood of mortality in 6 months.¹⁸ The mean Elixhauser score was calculated for members during their preindex time period to assess level of comorbidity within the population. Distributions of preindex and postindex all-cause medical costs are shown in the eAppendix Figure.

Statistical Analysis

One-to-one propensity score matching (PSM) was used to identify a matched control cohort of individuals who were eligible but did not enroll in the program. Although PSM is widely used and acknowledged,^19,20 it is an imperfect method for controlling selection bias. Only randomization can control for unobserved differences that may be present when similar individuals make different decisions about program enrollment. The outcome of the propensity model was propensity for enrollment in the Ontrak program. Features included in the forced match (line of business, age category, health plan, SUD/anxiety/depression diagnosis) are shown in eAppendix Table 3. Features included in the PSM model are listed in eAppendix Table 3 and included date first identified, sociodemographic/clinical characteristics, health plan, line of business, and slope of 1-month pre-enrollment costs (we identified costs 1 month prior to enrollment, subtracted PMPM value for the 12-month preindex time period, and matched treated and control members who had the same directionality of slope to match on trend of 1-month preindex costs relative to 12-month preindex period).

Pairwise propensity score distances were identified between each treated member and every control member, and couplings were sorted in ascending order by propensity score distance. The first time a control member was assigned to a treated member, based on having the smallest propensity score distance, all later couplings involving that control member were removed. Similarly, once any treated member attained a matched control, all later couplings involving that treated member were removed.

The quality of the match was assessed using a standardized differences threshold of 0.2 across the treatment and control groups on all variables used in the PSM model. Statistical differences between groups were determined using χ² tests for categorical variables and Mann-Whitney tests for non–normally distributed continuous variables.²¹ We conducted sensitivity analyses on the balance of the PSM-matched cohorts by assessing additional comparison groups including a never-enrolled (but eligible to enroll) cohort, unmatched controls, and a random sample of unmatched controls (eAppendix Table 4).

Difference-in-differences (DID) calculations were applied to the matched treated and control groups to estimate the average treatment effect for the outcomes of interest, including all-cause and disease-attributable cost and utilization outcome variables. Following DID methods, the pre-post change in PMPM cost and utilization was derived for both the treated cohort and the matched controls (ie, for each cohort, the postindex values for cost and utilization were subtracted from the preindex values to create the within-member differences used in the analysis). Thus, DID = (postindex cost – preindex cost)_Treatment – (postindex cost – preindex cost)_Control.

Two-sample t tests were applied to assess the statistical significance of the DID values because the variables used in the DID statistics were continuous and our sample size was sufficiently large. Difference statistics between preindex and postindex periods were reported as the mean value with a 95% CI and SD for the treatment and control cohorts separately, as well as for the DID across the cohorts. CIs were calculated using the simple bootstrap method.²² To investigate the sensitivity of the findings to outlier values, we took the logarithmic value of the larger of cost and $1 and ran a standard DID regression. We also winsorized using thresholds from 0% to 40% on member-level DID.

RESULTS

The final sample that met all predefined study eligibility criteria included 1800 individuals (900 treated, 900 matched controls; this even number of treated and control cohorts was arrived at by chance following application of our study eligibility criteria and exact matching criteria) (Figure 2), with a mean age of 58 years, 50% in Medicare Advantage and 50% in commercial plans, and 64% to 65% female (Table 1). Mean Elixhauser comorbidity scores were 2.08 and 2.01 in the treated and control cohorts, respectively (P = .72). The 3 most common medical comorbidities were hypertension, diabetes, and chronic pain. Among psychiatric comorbidities, anxiety, SUD, and depression were most common (Table 1). Many individuals had multiple, overlapping psychiatric and clinical comorbidities.

Descriptive statistics of all measured clinical and sociodemographic variables for the treated and control cohorts and members included or excluded from matching are provided in Table 1 and eAppendix Table 4. Differences across clinical or sociodemographic characteristics between the 2 cohorts were nonsignificant (Table 1) and below our threshold of less than 0.2 standard units across these variables, indicating a well-balanced matched control cohort (eAppendix Table 5). After matching, preindex period inpatient-related costs differed between treatment and control cohorts by 19%; all-cause costs differed by 11% (Table 2).

Over 24 months, all-cause costs were statistically significantly lower in the treated vs control cohort, by $485 PMPM (95% CI, $170-$780; P < .001). This was largely driven by a reduction in inpatient utilization; we found a 66% pre-post reduction in inpatient encounters among the treated cohort, with $488 PMPM DID savings for inpatient admissions (95% CI, $183-$683; P < .001) compared with the control cohort (Table 2). Results from our sensitivity analyses (eAppendix Table 6) indicated a DID savings of $426 (all-cause costs) and the magnitude of savings was smallest when 15% of upper and lower bounds were winsorized (eAppendix Table 6; $384 PMPM). These values both fall within the 95% CIs described for the main effects ($170-$780), indicating a change in magnitude of savings, but no change in directionality.

We identified statistically significant cost increases for all-cause office visits in the treated cohort ($110 PMPM; P < .001) (Table 2). All-cause outpatient costs and utilization decreased modestly for both the treated and control cohorts, with $74 PMPM DID savings (P = .06) for the treated cohort compared with the control cohort. ED cost and utilization also decreased slightly for both cohorts, with a small but significant reduction in ED visits for the treated cohort compared with the control cohort (0.02 visits PMPM; P < .05).

Similar trends were found in disease-attributable costs (Table 3). There were statistically significant savings for SUD-attributable total medical costs ($224 PMPM; P < .05) among the treated cohort compared with the control cohort. These savings are likely driven by statistically significant inpatient cost savings ($630 PMPM; P < .01) and a small but significant reduction in utilization (0.07 hospitalizations PMPM; P < .01). Office visit costs and utilization attributable to anxiety, depression, and BH increased significantly (P < .001) for the treated cohort between the preindex and postindex periods compared with the control cohort (by $69 and 0.109 visits; $61 and 0.133 visits; and $75 and 0.124 visits PMPM, respectively).

DISCUSSION

Completing a telephonic care coaching and BH provider referral program was associated with a significant reduction in all-cause medical costs for high-cost individuals with BH conditions and complex acute and chronic clinical comorbidities. The cost differences were largely driven by a reduction in all-cause inpatient utilization. Conversely, we identified statistically significant cost increases for all-cause office visits in the treated cohort compared with the control cohort, suggesting that enrolled individuals had fewer inpatient encounters but more office visits.

Mean PMPM costs declined in both the control and the treatment cohorts. This was not unexpected: Eligibility criteria favored individuals with high health care spending. Greater cost decrease in the treatment cohort may reflect a shift away from more costly inpatient treatment and ED services to more efficient, appropriate outpatient care. Individuals who enrolled in the Ontrak program had statistically significant decreases in inpatient encounters and increases in costs related to office visits, driven predominantly by increases in SUD-attributable and BH-attributable care.

Well-established longitudinal methodologies identifying linear clinical care pathways to establish causality and measure treatment effect of BH interventions are still in development. However, consistent with our findings, investigators assessing Medicaid administrative claims described cost differences between a cohort who enrolled in a BH self-care digital tool and those who did not and found a DID cost reduction of $382 per user and return on investment between 142% and 695%.¹¹ The point estimates described in these results are substantially higher than in other published results^11,23,24; however, direct comparison is difficult because of differences in study population and postenrollment follow-up time. A strength of our study is the 24-month postindex follow-up period, which is considerably longer than those found in other published reports.¹¹

Patients with BH conditions have reported a low preference for involvement in their medical decision-making, even when they preferred to have higher involvement in their BH care,^25-27 and BH conditions have been identified as a risk factor for lower adherence to medical treatment.^28,29 Lower patient engagement may decrease the effectiveness of coordinated care delivery³⁰ and presents a challenge for addressing higher medical costs and avoidable utilization within this population. However, in our claims analysis, members of the treated cohort had a mean of 10 provider encounters associated with Ontrak referrals and a mean of 23 contacts with care coaches. This indicates high engagement with providers, care coaches, and BH-related services among the treated cohort, despite low BH care utilization in the general population.¹ High engagement may reflect high levels of patient activation and the effect of the program on increased health-seeking behavior. The study provides evidence for the durability of program impact; the 12-month engagement in this program is longer than that of most BH programs. Additional evaluation will be necessary to confirm that the benefits of the program remain observable beyond 24 months post enrollment.

Limitations

Our conclusions should be interpreted considering several important limitations. First, this observational study was limited to graduates of the Ontrak program. When compared with the ever-enrolled population, our treated cohort was closely matched on clinical and sociodemographic characteristics (eAppendix Table 7). Second, sample size was limited by 36-month health plan eligibility criteria. This criterion was necessary to measure primary end points within our study design. Third, although we employed both 1:1 PSM and DID together to address limitations of cross-sectional matched pair studies, there may be endogeneity inherent in using enrollment (which is not randomized) in our propensity score model. We attempted to control for this in a matching methodology, and yet we recognize this as a limitation. We encourage prospective randomized controlled trials to address this limitation. Due to the limitations of what is possible to control within retrospective, nonrandomized study designs, we implemented several sensitivity analyses. We found the magnitude of savings was smaller but within the 95% CI ranges described, indicating a change in magnitude of savings but no change in directionality. This also indicates that results could be sensitive to cost outliers, and future evaluations should consider this finding.

The use of administrative claims data limits our analysis in a few ways. Data on race, ethnicity, marital status, or income are not available. Plan type information is not available (eg, health maintenance organization, preferred provider organization), and we were not able to evaluate differences in allowed amount, co-payments, and out-of-pocket costs across these plan types, as well as network coverage. Pharmacy data were not available for all participants and could not be used in the analysis. Costs were limited to plan-paid costs.

Finally, although Ontrak has also contracted with Medicaid insurers, this was a newer population to the business and did not meet continuous eligibility criteria for our study design. Therefore, our analysis was limited to commercial and Medicare Advantage lines of business.

CONCLUSIONS

Point estimates presented in this article should be interpreted while considering limitations in our study design and data capabilities. Our findings suggest that care coaching interventions offering BH provider referrals may both increase services to treat SUD and BH conditions and reduce total all-cause health care costs. Findings of our observational retrospective cohort study suggest that telephonic care coaching programs to treat BH conditions may provide long-term savings, reductions in avoidable utilization, and increases in targeted services to treat BH conditions. We encourage additional rigorous evaluations to confirm our findings, ideally incorporating prospective, randomized study designs. Overall, our results may provide support for the shift toward integration of BH interventions to achieve long-term savings in US health care costs. This is critical as we develop a perspective of value-based care in the United States focusing on behavioral interventions embedded within clinical settings.

Acknowledgments

The authors would like to thank and acknowledge Avram Aelony, Dean Elzinga, Jo-Hsin Chen, Greg Mellstrom, and Kimberly Morgan for their contributions to this analysis.

Author Affiliations: Ariadne Labs (HEDP), Boston, MA; Ontrak, Inc (HEDP, BD, MG-H, PB), Santa Monica, CA; Clinical Excellence Research Center, Stanford School of Medicine (RMK, JG), Palo Alto, CA.

Source of Funding: None.

Author Disclosures: Dr Placzek, Ms Darby, Ms Garcia-Huynh, and Dr Bearse are employed by Ontrak, and Drs Placzek and Bearse own stock in Ontrak. Dr Kaplan and Dr Glassman are paid consultants for Ontrak and received payment from Ontrak for their involvement in the preparation of this manuscript.

Authorship Information: Concept and design (HEDP, RMK, PB); acquisition of data (HEDP, BD, PB); analysis and interpretation of data (HEDP, BD, RMK, JG, MG-H, PB); drafting of the manuscript (HEDP, BD, RMK, JG); critical revision of the manuscript for important intellectual content (HEDP, BD, RMK, JG, PB); statistical analysis (HEDP, BD, RMK, MG-H, PB); administrative, technical, or logistic support (HEDP, BD, MG-H); supervision (HEDP, BD, RMK); and serving as principal investigator (HEDP).

Address Correspondence to: Hilary E. D. Placzek, PhD, MPH. Email: hplaczek@gmail.com.

REFERENCES

1. Mental health information: statistics. National Institute of Mental Health. Accessed January 15, 2021. https://www.nimh.nih.gov/health/statistics/mental-illness

2. Czeisler MÉ, Lane RI, Petrosky E, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic — United States, June 24-30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(32):1049-1057. doi:10.15585/mmwr.mm6932a1

3. Panchal N, Kamal R, Cox C, Garfield R. The implications of COVID-19 for mental health and substance use. Kaiser Family Foundation. Accessed February 10, 2021. https://www.kff.org/coronavirus-covid-19/issue-brief/the-implications-of-covid-19-for-mental-health-and-substance-use/

4. Clarke JL, Skoufalos A, Medalia A, Fendrick AM. Improving health outcomes for patients with depression: a population health imperative: report on an expert panel meeting. Popul Health Manag. 2016;19(suppl 2):S1-S12. doi:10.1089/pop.2016.0114

5. Melek SP, Norris DT, Paulus J, Matthews K, Weaver A, Davenport S. Potential economic impact of integrated medical-behavioral healthcare: updated projections for 2017. Milliman. January 2018. Accessed January 15, 2021. https://www.milliman.com/-/media/milliman/importedfiles/uploadedfiles/insight/2018/potential-economic-impact-integrated-healthcare.ashx

6. Telehealth for the treatment of serious mental illness and substance use disorders. Substance Abuse and Mental Health Services Administration. 2021. Accessed January 15, 2021. https://store.samhsa.gov/sites/default/files/SAMHSA_Digital_Download/PEP21-06-02-001.pdf

7. Thompson-de Benoit A, Kramer U. Work with emotions in remote psychotherapy in the time of Covid-19: a clinical experience. Couns Psychol Q. 2021;34(3-4):368-376. doi:10.1080/09515070.2020.1770696

8. Humer E, Stippl P, Pieh C, Pryss R, Probst T. Experiences of psychotherapists with remote psychotherapy during the COVID-19 pandemic: cross-sectional web-based survey study. J Med Internet Res. 2020;22(11):e20246. doi:10.2196/20246

9. Probst T, Haid B, Schimböck W, et al. Therapeutic interventions in in-person and remote psychotherapy: survey with psychotherapists and patients experiencing in-person and remote psychotherapy during COVID-19. Clin Psychol Psychother. 2021;28(4):988-1000. doi:10.1002/cpp2553

10. Rosen CS, Morland LA, Glassman LH, et al. Virtual mental health care in the Veterans Health Administration’s immediate response to coronavirus disease-19. Am Psychol. 2021;76(1):26. doi:10.1037/amp0000751

11. Abhulimen S, Hirsch A. Quantifying the economic impact of a digital self-care behavioral health platform on Missouri Medicaid expenditures. J Med Econ. 2018;21(11):1084-1090. doi:10.1080/13696998

12. Pande RL, Morris M, Peters A, Spettell CM, Feifer R, Gillis W. Leveraging remote behavioral health interventions to improve medical outcomes and reduce costs. Am J Manag Care. 2015;21(2):e141-e151.

13. Thase ME, McCrone P, Barrett MS, et al. Improving cost-effectiveness and access to cognitive behavior therapy for depression: providing remote-ready, computer-assisted psychotherapy in times of crisis and beyond. Psychother Psychosom. 2020;89(5):307-313. doi:10.1159/000508143

14. Miller W, Rollnick S. Motivational Interviewing: Helping People Change. 3rd ed. The Guilford Press; 2013.

15. Ryan R, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55(1):68-78. doi:10.1037/0003-066x.55.1.68

16. Caldwell KL, Gray J, Wolever RQ. The process of patient empowerment in integrative health coaching: how does it happen? Glob Adv Health Med. 2013;2(3):48-57. doi:10.7453/gahmj.2013.026

17. Defining rural population. Health Resources and Services Administration. Accessed December 2020. https://www.hrsa.gov/rural-health/about-us/definition/index.html

18. van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6):626-633. doi:10.1097/MLR.0b013e31819432e5

19. Placzek HED, Masters ET, Gu T, et al. Prior authorization in the treatment of patients with pDPN and FM. Pain Pract. 2015;15(1):E9-E19. doi:10.1111/papr.12258

20. Daw JR, Hatfield LA. Matching and regression to the mean in difference-in-differences analysis. Health Serv Res. 2018;53(6):4138-4156. doi:10.1111/1475-6773.12993

21. Nahm FS. Nonparametric statistical tests for the continuous data: the basic concept and the practical use. Korean J Anesthesiol. 2016;69(1):8-14. doi:10.4097/kjae.2016.69.1.8

22. Austin PC, Small DS. The use of bootstrapping when using propensity-score matching without replacement: a simulation study. Stat Med. 2014;33(24):4306-4319. doi:10.1002/sim.6276

23. Liu M, Yuan X, Ouyang J, et al. Evaluation of four disease management programs: evidence from Blue Cross Blue Shield of Louisiana. J Med Econ. 2020;23(6):557-565. doi:10.1080/13696998.2020.1722677

24. de Bruin SR, Heijink R, Lemmens LC, Struijs JN, Baan CA. Impact of disease management programs on healthcare expenditures for patients with diabetes, depression, heart failure or chronic obstructive pulmonary disease: a systematic review of the literature. Health Policy. 2011;101(2):105-121. doi:10.1016/j.healthpol.2011.03.006

25. Adams JR, Drake RE, Wolford GL. Shared decision-making preferences of people with severe mental illness. Psychiatr Serv. 2007;58(9):1219-1221. doi:10.1176/ps.2007.58.9.1219

26. Patel SR, Bakken S. Preferences for participation in decision making among ethnically diverse patients with anxiety and depression. Community Ment Health J. 2010;46(5):466-473. doi:10.1007/s10597-010-9323-3

27. Moise N, Ye S, Alcantara C, Davidson KW, Kronish I. Depressive symptoms and decision-making preferences in patients with comorbid illnesses. J Psychosom Res. 2017;92:63-66. doi:10.1016/j.jpsychores.2015.12.001

28. DiMatteo MR, Lepper HS, Crogan TW. Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence. Arch Intern Med. 2000;160(14):2101-2107. doi:10.1001/archinte.160.14.2101

29. Cramer JA, Rosenheck R. Compliance with medication regimens for mental and physical disorders. Psychiatr Serv. 1998;49(2):196-201. doi:10.1176/ps.49.2.196

30. Benzer JK, Singer SJ, Mohr DC, et al. Survey of patient-centered coordination of care for diabetes with cardiovascular and mental health comorbidities in the Department of Veterans Affairs. J Gen Intern Med. 2019;34(suppl 1):S43-S49. doi:10.1007/s11606-019-04979-8