Systematic Analysis of Outcomes Evaluations Applied to Drug Management Programs

Objective: To evaluate prior analyses that quantify how drug management programs impact health plans and patients with respect to those analyses’ effectiveness in measuring program quality.

Methods: We examined 77 US studies from 1996 to 2007 that evaluated the respective drug management programs of managed care, indemnity insurance, Veterans Health Administration, and Medicaid health plans. Our review included those studies that assessed the ways in which specific drug management tools and the drug management program overall impacted economic, clinical, and/or humanistic outcomes.

Results: The 77 studies included in our review used 11 types of outcome end points. A total of 52 studies (68%) incorporated an economic end point; of these, 35 (68%) reported economic data only and did not address clinical or humanistic outcomes. Overall, 33 (43%) evaluated clinical or humanistic end points; of these, 17 (52%) also reported on an economic end point.

Conclusions: Although the number of studies evaluating drug management programs has trended upward, only a handful have integrated economic, clinical, and/or humanistic outcomes when assessing the effects these programs have on health plans and patients. The efforts of these researchers to assess the overall quality of drug management programs have fallen short. To ensure that drug management tools have a desired effect on outcomes and medical costs, measures used to evaluate drug management programs must be improved.

(Am J Manag Care. 2008;14(11 Spec No.):SP36-SP45)

Few studies designed to evaluate the impact of drug management programs have adequately assessed their quality, and more comprehensive assessments are needed to ensure the following:

• Drug management program tools improve or remain neutral to patient outcomes.
• Health plans use drug management tools that deliver the most value (best health outcomes) per dollar spent.

Health plans generally introduce drug management programs and tools to manage and control how members use prescription drugs. These programs influence the way patients are treated medically and must be viewed as healthcare interventions; they also have health outcome and cost implications. Just as with other healthcare interventions, it is essential to ensure that drug management programs do not lead to negative outcomes for patients that may subsequently increase overall medical costs. To this end, it is important to accurately assess the quality of existing programs.

Before quality can be measured, however, it must be defined. Genichi Taguchi was one of the earliest promoters for improving quality in manufacturing. He defined poor quality as “the loss a product imposes on society after it is shipped.”¹ If we apply Taguchi’s definition to healthcare, poor quality might be described as the loss an intervention imposes on society after it is implemented. Although the manufacturing industry now has procedures for quality assessment firmly in place, measuring the quality of healthcare services—including prescription drug programs—and pinpointing a precise definition of “quality” appear much more elusive. The Institute of Medicine has defined quality in healthcare as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”²

The US healthcare industry has made progress in developing measures to evaluate medical care delivery and patient outcomes for individuals with private and public health insurance. National Committee for Quality Assurance accreditation and quality reporting (based on the Healthcare Effectiveness Data and Information Set) and the Joint Commission on Accreditation of Healthcare Organizations performance measurement initiatives are quality measurement programs³ that specify disease treatment and management measures, and incorporate appropriate indices for prescribing medication. Neither program outlines specific criteria for assessing drug management programs. The following organizations have made independent efforts to develop and implement quality measures for pharmaceutical care, broadly and in the payer/managed care setting:

• The Academy of Managed Care Pharmacy Formulary Submission Format designed an evidence-based, effectiveness-driven approach to formulary management.4• The National Quality Forum is developing voluntary standards for reporting the quality of therapeutic drug management.5

• The URAC Pharmacy Benefit Management Accreditation Format has established a benchmark for the quality of care in pharmacy benefit management.⁷

Although these programs make a good beginning, none is comprehensive and each restricts itself to a particular stakeholder perspective. There is no consensus among the programs as to what should be measured, how it should be measured, and what outcomes should constitute thresholds for different quality measures. This lack of consensus has led to significant variations in data and results. Without standardized quality measures and public reports on drug management programs’ performance, it is difficult for stakeholders to assess or compare quality claims presented in the literature or by health plans.

In addition to analyzing the methodologies used in the literature to quantify the impact of drug management programs, we wanted to offer insight as to whether these initiatives help ensure high quality overall (eg, the best possible outcomes for health plans and patients). This may help managed care organizations and other stakeholders in the United States improve the processes for measuring quality, better determine the effectiveness of drug management programs in the United States, and provide a framework for improving quality.

METHODS

Applying the search terms listed in the eAppendix Table, we identified 1099 potentially relevant articles. After careful screening to exclude studies that did not meet inclusion criteria, we identified 77 articles suitable for this review. The Table details each of the 77 articles included in our analysis.^8-84 Research on the effects of drug management programs evolved over the study period. Only 5 (6.5%) of the 77 studies we reviewed that evaluated drug management programs were published before 2001; after 2001, the number of these types of studies increased dramatically, as did the breadth of end points considered. Although only 1 study published in 1996-1999 met our inclusion criteria, there were 4 qualifying studies published in 2000-2001, 15 in 2002-2003, 31 in 2004-2005, and 26 in 2006-2007.

Data Source

The studies we examined measured the effects of 1 or more of 10 drug management tool types on various end points. Cumulatively, the studies encompassed the following 10 drug management tool types: formulary restrictions, preferred drug list, use of copayments or cost-sharing measures, prior authorization procedures, quantity level limits, step-therapy edits, therapeutic substitutions, generic drug programs, pharmacy cognitive services, and over-the-counter drug coverage (Figure 1). A majority of the studies (77%) evaluated only 1 drug management tool type, with cost sharing being the most common (n = 39; 50.6%), followed by formulary restrictions (n = 27; 35.1%).

Study End Points

W. Edwards Deming, one of the founders of quality theories, surmised that “Better quality at lower price has a chance to capture a market. Cutting costs without improvement of quality is futile.”85 Operating on this tenet, researchers must strive to prove that healthcare interventions positively impact the quality of healthcare in addition to measuring how they affect cost and consumption. Otherwise, improved health outcomes might come at unacceptable costs or, conversely, economic savings might be achieved at the expense of a patient’s health.

Limitations to Previous Analyses

One of the main purposes of our analysis was to identify the extent to which evaluations of drug management programs measure quality, or applying the Institute of Medicine definition, identify the degree to which health services for individuals and populations increase the likelihood of desired health-related outcomes and are consistent with current professional knowledge.² Although the studies we reviewed quantified the impact of drug management programs, they usually included a narrow scope of outcomes and were not specifically designed to measure quality.

Although plans and researchers are demonstrating growing interest in the effects of managed care drug management initiatives on clinical outcomes, even the newer studies fall short of providing a comprehensive quality assessment as a result of their failure to consider both economic and patient -centered end points. Despite the increasing number of studies to incorporate a diverse panel of outcomes, none appeared to have a predefined purpose of quality measurement by which to proceed with their assessments.

Analyze Economic and Patient Outcomes. To help ensure that quality is appropriately assessed, several aspects need to be considered when designing evaluations for drug management programs. The study’s objective should include a patient outcomes measure. Ideally, outcomes measures should be end-point measures, such as physiologic response to an intervention, hospitalizations because of treatment failures or complications, or patient-reported outcomes. In cases where patient follow-up time makes some of these measures unfeasible, intermediate outcomes (eg, medication compliance, short-term resource use) may be the most viable options until long-term data are available to evaluate the program’s impact. Any analysis of patient outcomes should be accompanied by a comprehensive economic analysis that goes beyond the program’s impact on the pharmacy budget and encompasses its effect on the total costs to plans and patients.

Use Data From Multiple Sources. When planning how to execute a study on drug management programs, researchers should carefully consider data collection methods. They must be prepared to use appropriate data from multiple sources, with the goal of considering a combination of economic, clinical, and/or humanistic outcomes. For example, to evaluate economic and physiologic outcomes, researchers could augment claims data with EMR; to validate patient-reported outcomes and support an economic assessment, patient survey data could be combined with claims data.

Adopt a Long-Term Perspective. A comprehensive quality analysis of a drug management program considers the program from a long-term perspective. This may compete with a health plan’s need to deliver short-term benefits. Even if the average retention for health plan members is only 3 years, however, the drug management program’s long-term impact is important for several reasons. It is conceivable—if not likely—that patients subject to drug management program interventions who leave the plan will later return. In these cases, the plan would benefit from implementing a drug management program that encourages positive outcomes in the long term; conversely, the plan would suffer from earlier short-sighted decisions. Additionally, innovative employers or payers will recognize benefits from interventions that deliver long-term cost savings and improve patient outcomes because these results align with their goal of having a healthy and productive workforce.

Limitations of This Study

Historically, studies that have evaluated the effects of drug management programs have neglected to assess quality as a measurement of health outcome per cost. We believe our analysis is the first to consider the methodologic approaches used in assessing how drug management programs or tools impact outcomes in the United States and the degree to which previous studies have evaluated program quality. Other reviews have synthesized the findings of drug management program research, such as the assessment of previous economic findings by Hadley,⁸⁶ or reviewed the findings of studies that evaluated prior authorization, like the one by MacKinnon and Kumar.⁸⁷ Goldman et al recently reviewed literature that assessed the association between cost sharing, medication utilization, and outcomes.⁸⁸ None of these reviews considered the methodologic approaches used in other studies to evaluate the impact of drug management programs and whether they were comprehensive enough to draw valid conclusions about overall program quality.

Deming said that “Cutting costs without improvement of quality is futile.”⁸⁵ Healthcare resources are limited; therefore, assessments of drug management programs need to evolve beyond an analysis of short-term benefits. They need to ensure that drug management tools live up to their intent, which is to control medication use and lower overall medical costs without hindering patient outcomes. Although some studies we examined did look beyond a single outcome perspective, only a small number considered both economic and healthrelated outcomes. This limits the usefulness of current literature as a tool to guide plan designers working to enhance the effectiveness of drug management programs.

The continued publication of drug management program assessments is highly encouraged. Such evaluations serve to disseminate the best practices and facilitate consensus on quality assessment approaches for drug management program tools.

AcknowledgmentsThe authors would like to thank Diana I. Brixner, PhD, RPh, for her leadership; and Brian T. Agnew, MB A, Marie Howson, RN , MB A, and Barbara L. Roper, BA, for their support of this project.

Author Affiliations: From the Department of Pharmacotherapy (CM-M, A-PH, GMO), University of Utah, Salt Lake City; Millcreek Outcomes Group (CM-M, A-PH, GMO), Salt Lake City; and Pfizer, Inc (DTS, BE), New York.

Funding Source: This study was funded by Pfizer, Inc.

Author Disclosures: Ms McAdam-Marx, Dr Holtorf, and Mr Oderda are consultants with Millcreek Outcomes Group, which received payment from Pfizer, Inc for the development and preparation of this manuscript. Drs Schaaf and Eng are employees of Pfizer, Inc and report owning stock in that company.

Authorship Information: Concept and design (CM-M, DTS, A-PH, BE, GMO); acquisition of data (CM-M, A-PH, BE); analysis and interpretation of data (CM-M, DTS, A-PH, BE, GMO); drafting of the manuscript (CM-M, DTS, A-PH); critical revision of the manuscript for important intellectual content (CM-M, DTS, A-PH, GMO); statistical analysis (BE); obtaining funding (DTS, BE, GMO); administrative, technical, or logistic support (A-PH); and supervision (GMO).

Address correspondence to: Carrie McAdam-Marx, RPh, MS, Department of Pharmacotherapy, University of Utah, 421 Wakara Wy, Rm 208, Salt Lake City, UT 84108. E-mail: carrie.mcadam-marx@pharm.utah.edu.

1. Taguchi G. Introduction to Quality Engineering. New York, NY:UNIPAB; 1986.

3. Agency for Health Care Research and Quality. AHRQ Quality Indicators. http://www.qualityindicators.ahrq.gov/. 2008.

5. National Quality Forum. National voluntary consensus standards for the reporting of therapeutic drug management quality. http://www.fmcpnet.org/cfr/waSys/%20f.cfc?method=getListFile&id=A2D0F502. Accessed October 25, 2006.

7. Pharmacy Benefit Management Accreditation. URAC. http://www.pbm.urac.org/files/URAC-PBM-Brochure.pdf. Accessed October 14,

8. Abourjaily P, Gouveia WA, Selker HP, Zucker DR. Evaluating the nondrug costs of formulary coverage restrictions. Manag Care. 2005;14(8):50-57, 62.

10. Adams AS, Soumerai SB, Ross-Degnan D. Use of antihypertensive drugs by Medicare enrollees: does type of drug coverage matter? Health Aff (Millwood). 2001;20(1):276-286.

12. Artz MB, Hadsall RS, Schondelmeyer SW. Impact of generosity level of outpatient prescription drug coverage on prescription drug events and expenditure among older persons. Am J Public Health. 2002;92(8):1257-1263.

14. Briesacher B, Kamal-Bahl S, Hochberg M, Orwig D, Kahler KH. Three-tiered-copayment drug coverage and use of nonsteroidal anti-inflammatory drugs. Arch Intern Med. 2004;164(15):1679-1684.

16. Bukstein DA, Cherayil GA, Gepner AD, Luskin AT, Kooistra JB, Olson RM. The economic burden associated with prior authorizations in an allergist office. Allergy Asthma Proc. 2006;27(2):119-122.

18. Christian-Herman J, Emons M, George D. Effects of generic-only drug coverage in a Medicare HMO. Health Aff (Millwood). 2004;(suppl web exclusives):w4-w455-w468.

20. Cox E, Henderson R, Motheral B. Health plan member experience with point-of-service prescription step therapy. J Manag Care Pharm. 2004;10(4):291-298.

22. Dunn JD, Cannon HE. Effects of a polypharmacy edit and reduced quantity limits on the utilization of triptans and overall costs in an integrated health system. Manag Care Interface. 2006;19(5):46-51.

24. Fairman KA, Motheral BR, Henderson RR. Retrospective, long-term follow-up study of the effect of a three-tier prescription drug copayment system on pharmaceutical and other medical utilization and costs. Clin Ther. 2003;25(12):3147-3161.

26. Fink KS, Byrns PJ. Changing prescribing patterns and increasing prescription expenditures in Medicaid. Ann Fam Med. 2004;2(5):488-493.

28. Fischer MA, Avorn J. Economic consequences of underuse of generic drugs: evidence from Medicaid and implications for prescription drug benefit plans. Health Serv Res. 2003;38(4):1051-1064.

30. Gershovich OE, Billups SJ, Delate T, Hoffman CK, Carroll N. Assessment of clinical, service, and cost outcomes of a conversion program of sumatriptan to rizatriptan ODT in primary care patients with migraine headaches. J Manag Care Pharm. 2006;12(3):246-253.

32. Gibson TB, Mark TL, McGuigan KA, Axelsen K, Wang S. The effects of prescription drug copayments on statin adherence. Am J Manag Care. 2006;12(9):509-517.

34. Gilman BH, Kautter J. Consumer response to dual incentives under multi-tiered prescription drug formularies. Am J Manag Care. 2007;13(6, pt 2):353-359.

36. Harris BN, West DS, Johnson JT, Hong SH, Stowe CD. Effects on the cost and utilization of proton pump inhibitors from adding over-the-counter omeprazole to drug benefit coverage in a state employee health plan. J Manag Care Pharm. 2004;10(5):449-455.

38. Hartung DM, Ketchum KL, Haxby DG. An evaluation of Oregon’s evidence-based Practitioner-Managed Prescription Drug Plan. Health Aff (Millwood). 2006;25(5):1423-1432.

40. Hsu J, Price M, Huang J, et al. Unintended consequences of caps on Medicare drug benefits. N Engl J Med. 2006;354(22):2349-2359.

42. Huskamp HA, Deverka PA, Epstein AM, et al. Impact of 3-tier formularies on drug treatment of attention-deficit/hyperactivity disorder in children. Arch Gen Psychiatry. 2005;62(4):435-441.

44. Johnsrud M, Lawson KA, Shepherd MD. Comparison of mailorder with community pharmacy in plan sponsor cost and member cost in two large pharmacy benefit plans. J Manag Care Pharm. 2007;13(2):122-134.

46. Kamal-Bahl S, Briesacher B. How do incentive-based formularies influence drug selection and spending for hypertension? Health Aff (Millwood). 2004;23(1):227-236.

48. Landon BE, Rosenthal MB, Normand SL, et al. Incentive formularies and changes in prescription drug spending. Am J Manag Care. 2007;13(6, pt 2):360-369.

50. Lurk JT, DeJong DJ, Woods TM, Knell ME, Carroll CA. Effects of changes in patient cost sharing and drug sample policies on prescription drug costs and utilization in a safety-net-provider setting. Am J Health Syst Pharm. 2004;61(3):267-272.

52. Mahoney JJ. Lowering patient drug acquisition costs to improve health outcomes and costs. Manag Care Interface. 2006;19(4):13.

54. Meissner BL, Moore WM, Shinogle JA, Reeder CE, Little JM. Effects of an increase in prescription copayment on utilization of low-sedating antihistamines and nasal steroids. J Manag Care Pharm. 2004;10(3):226-233.

56. Mojtabai R, Olfson M. Medication costs, adherence, and health outcomes among Medicare beneficiaries. Health Aff (Millwood). 2003;22(4):220-229.

58. Motheral B, Fairman K. Effect of a three-tier prescription copay on pharmaceutical and other medical utilization. Med Care. 2001;39(12):1293-1304.

60. Motheral BR, Heinle SM. Predictors of satisfaction of health plan members with prescription drug benefits. Am J Health Syst Pharm. 2004;61(10):1007-1014.

62. Nair KV, Ganther JM, Valuck RJ, McCollum MM, Lewis SJ. Impact of multi-tiered pharmacy benefits on attitudes of plan members with chronic disease states. J Manag Care Pharm. 2002;8(6):477-491.

64. Oh T, Mehta SS, Bonanno C. Managed care organization’s experience with a therapeutic conversion program in hypertensive patients. Am J Health Syst Pharm. 1999;56(Jun):P-51.

66. Panzer PE, Regan TS, Chiao E, Sarnes MW. Implications of an SSRI generic step therapy pharmacy benefit design: an economic model in anxiety disorders. Am J Manag Care. 2005;11(12 suppl):S370-S379.

68. Richards K, Johnsrud M, Shepherd M. A review of prior authorization programs: antihemophilic, antineoplastic, and antiretroviral agents. The Center for Pharmacoeconomic Studies. http://www.utexas.edu/pharmacy/research/institutes/pharmacoeconomics/pareport.pdf. Accessed October 13, 2008.

70. Sansgiry SS, Sikri S, Kawatkar A. Consumer knowledge and perceptions of formularies. Manag Care Interface. 2004;17(1):21-26, 30.

72. Sprague MS, Gray DR. Impact of a formulary switch from amlodipine to felodipine at a VA medical center. Am J Health Syst Pharm. 1998;55(Dec):P-424E.

74. Sweet BV, Stevenson JG. Pharmacy costs associated with nonformulary drug requests. Am J Health Syst Pharm. 2001;58(18):1746-1752.

76. Thiebaud P, Patel BV, Nichol MB, Berenbeim DM. The effect of switching on compliance and persistence: the case of statin treatment. Am J Manag Care. 2005;11(11):670-674.

78. Trygstad TK, Hansen RA, Wegner SE. Evaluation of product switching after a state Medicaid program began covering loratadine OTC 1 year after market availability. J Manag Care Pharm. 2006;12(2):108-120.

80. Tseng CW, Brook RH, Keeler E, Steers WN, Mangione CM. Cost-lowering strategies used by Medicare beneficiaries who exceed drug benefit caps and have a gap in drug coverage. JAMA. 2004;292(8):952-960.

82. Wilson J, Axelsen K, Tang S. Medicaid prescription drug access restrictions: exploring the effect on patient persistence with hypertension medications. Am J Manag Care. 2005;11(spec no):SP27-SP34.

84. Zeber JE, Grazier KL, Valenstein M, Blow FC, Lantz PM. Effect of a medication copayment increase in veterans with schizophrenia. Am J Manag Care. 2007;13(6, pt 2):335-346.

86. Hadley J. Cost Containment Strategies for Prescription Drugs: Assessing the Evidence in the Literature. March 2005. Kaiser Family Foundation report 7295. http://www.kff.org/rxdrugs/7295.cfm.Accessed September 30, 2008.

88. Goldman DP, Joyce GF, Zheng Y. Prescription drug cost sharing: associations with medication and medical utilization and spending and health. JAMA. 2007;298(1):61-69.

Download PDF: Systematic Analysis of Outcomes Evaluations Applied to Drug Management Programs