Health Plan Use of Immunization Information Systems for Quality Measurement

Objective:

To evaluate a health plan’s business case for using a state immunization information system (IIS) as the primary data source for members’ immunization histories.

Study Design:

Case study of Priority Health, a Michigan managed care organization, to investigate use of IIS data for Healthcare Effectiveness Data and Information Set (HEDIS) compliance, quality measurement, and a provider incentive program.

Methods:

Primary data were collected through key informant interviews and group discussions with Priority Health and IIS managers. Priority Health’s information systems were populated with claims data and supplemental data, before chart reviews, to simulate immunization and health plan quality measures for 2004 to 2007 in the absence of IIS data. Simulated rates were compared with historical rates that included IIS data. The study included a cost-benefit analysis.

Results:

For 2007, IIS data increased observed immunization rates from 6.49 to 54.13 percentage points for childhood immunizations and 57.63 to 77.97 percentage points for adolescent immunizations. The HEDIS administrative rate for childhood immunizations doubled from 43.38% in 2003 to 88.08% in 2007. The most significant source of savings was in administration of the health plan’s Physician Incentive Program, which saw 18,881 fewer chart reviews from 2004 to 2007 when IIS data were used compared with when they were not used. Total costs of using IIS data were estimated to be $14,318 and net benefits were $107,854—corresponding to a benefit-to-cost ratio of 8.06.

Conclusions:

Health plans using a state IIS as a single point of data entry may realize cost savings and have improved assurance of immunization coverage.

(Am J Manag Care. 2010;16(3):217-224)

This case study examines the business case for a health plan to use its state immunization information system (IIS) as its primary data source.

• The IIS provides Priority Health, a managed care organization, with more timely and comprehensive data on members’ immunization status.
• IIS data reduced chart reviews for the Healthcare Effectiveness Data and Information Set and physician quality and incentive programs by nearly 20,000 reviews in 2004 to 2007.
• Savings over a 4-year period corresponded to an 8.06 benefit-to-cost ratio.

Immunization information systems (IIS) are confidential, computerized information systems that collect and consolidate vaccination data from multiple healthcare providers, generate reminder and recall notifications, and assess vaccination coverage within a defined geographic area. Most IIS have additional capabilities such as vaccine management, adverse event reporting, birth-to-death vaccination histories, and interoperability with other health information systems.¹ The benefits of IIS are believed to be so great that the Centers for Disease Control and Prevention established a national objective for at least 95% of US children age 6 years or younger to be included in fully operational population-based immunization registries.²

Health plans have been described as “natural partners” for IIS because they use data similar to those in IIS to measure product performance.³ The National Committee for Quality Assurance (NCQA) established the Healthcare Effectiveness Data and Information Set (HEDIS, formerly known as the Health Plan Employer Data and Information Set) as a set of performance benchmarks against which health plans can be compared, including up-to-date immunization status for children at 2 years of age. HEDIS reporting permits data collected from patient chart reviews, claims, and (when approved by NCQA) IIS.

Studies have been conducted to determine the costs associated with implementing IIS^4-8 and to illustrate that provider participation is critical for IIS success.^9,10 Yet there is little in the literature examining relationships between health plans and IIS.¹¹ This case study reviews the business case for Priority Health, a managed care organization, to establish Michigan’s IIS, the Michigan Care Improvement Registry (MCIR), rather than claims as its primary mechanism for acquiring members’ immunization data. Kilpatrick et al¹² define a business case as a scenario in which an organization realizes a positive return on investment for a particular intervention. We examined Priority Health’s decisional processes for establishing MCIR as the primary data source for immunization histories, the use of IIS data for quality and physician performance programs, and the incremental costs and benefits accruing to Priority Health as a consequence. We concluded that, although attributes of the case may be specific to Priority Health and MCIR, the data interchange partnership is replicable for other IIS and health plans.

BACKGROUND

Priority Health is a Michigan-based health plan with 480,000 members. Quality, as reflected by HEDIS measures, is integral to Priority Health’s corporate strategy. Priority Health’s performance on quality measures is used heavily in marketing products and programs to employers and healthcare providers. Indeed, Chernew et al¹³ conducted a study of 17 large employers and found that, although price was the predominant factor in health plan selection, these employers were more likely to offer plans with strong absolute and relative HEDIS performance measures. In 2003, NCQA affirmed MCIR’s acceptability for HEDIS, and Priority Health began populating its information systems with IIS data in 2004.

The Michigan Department of Community Health (MDCH) established MCIR in 1996, and childhood immunization providers are required by law to record immunizations within 72 hours of administration. As of 2007, 95% of children ages 19 to 35 months had 2 or more immunization records in MCIR. The provider participation rate is 94% and MCIR has approximately 12,000 users per day.¹⁴

Priority Health uses IIS data in 3 efforts: HEDIS reporting, a physicians’ incentive program, and a physician quality rating program. HEDIS reporting was described in the introduction. The second effort, the Physician Incentive Program (PIP), was implemented in 1997 when Priority Health moved from a risk-based business model to capitation. Immunization for children was among Priority Health’s first incentive measures and remained among its 17 measures for 2007. Adolescent immunization was incentivized from 2004 through 2006, after which it was retired as a HEDIS measure by NCQA.

To receive incentive payments, providers must meet the 90th percentile HEDIS benchmark for Priority Health members participating in commercial health maintenance organization (HMO) and point-of-service plans who have been continuously enrolled for 12 months. For 2006, providers received $175 per eligible member if they demonstrated that at least 87% of Priority Health members in their practice who turned 2 years old during the reporting year completed the Advisory Committee on Immunization Practices recommended series by their second birthday. Priority Health awarded $1.1 million in incentive payments for 2006’s immunization performance, one-sixth of that year’s $6.5-million incentive payout.

The third effort is a physician quality reporting program called the “apples on the Web.” Physician performance toward meeting HEDIS benchmarks is monitored and reported on Priority Health’s public Web site using a scale of 1 to 4 apples. Priority Health has not quantified the extent to which apples influence patients’ choice of physicians; however, it is believed that physicians monitor the apples closely.

Therefore, Priority Health rewards providers for meeting the up-to-date immunization coverage benchmark twice: first with a financial incentive and second with a high-quality rating on its Web site. Because the data informing these programs play a critical role, Priority Health has an interest in data capture and management becoming more effective and efficient.

METHODS

The case study relied on data collected through key informant interviews, simulations of Priority Health’s data systems with and without IIS data, and a cost-benefit analysis.

Primary Data Collection

Primary data were collected over the course of 2 visits to Priority Health’s headquarters in February and April 2008. Fifteen individuals from Priority Health, MDCH, and the Michigan Public Health Institute, which operates MCIR under contract from MDCH, participated in key informant interviews and group discussions facilitated by a semistructured interview guide. The purpose of the interviews was to elucidate the history of Priority Health’s programs relying on IIS data and to clarify the processes involved in the collection, exchange, and use of immunization data. Priority Health representatives included managers for Provider Services, Medical Outcomes and Quality Improvement, Network Expansion, Information Systems, and Medical Affairs. The MDCH and the Michigan Public Health Institute were represented by MCIR’s project coordination team. Interviews were guided by the 2 study leaders from RTI International (an economist and a public health scientist) and were attended by a representative from the Centers for Disease Control and Prevention’s Immunization Information Systems Support Branch. Notes were taken and exchanged among study authors to ensure accurate characterization of participants’ comments. All interviews were recorded to resolve any disagreement between authors’ assessments of any particular discussion topic. Data collection protocols were approved by RTI International’s institutional review board.

Simulations With and Without IIS DataFollowing methodologies pioneered by Griliches¹⁵ and Mansfield et al,¹⁶ the impact of IIS data on Priority Health’s immunization coverage was measured by simulating Priority Health’s HEDIS and PIP measures with claims data only for 2004 to 2007 and comparing the results with historical data that included IIS data. Priority Health already incentivized immunization, and the existence of its incentive and ratings programs was not contingent on MCIR’s availability. Our approach was to measure the incremental benefits of using IIS data for these programs and not the benefits of the programs themselves.

First, we assessed the impact of IIS data on Priority Health’s HEDIS rates prior to patient chart reviews. HEDIS reporting requires health plans to review a random sample of 411 patient charts annually for up-to-date immunization status for each population group in each commercial product. The number of charts to be reviewed may be reduced if, for each member in the sample, the health plan has a complete internal up-to-date immunization record. The up-to-date immunization rate from internal records is referred to as the final-sample-size administrative rate. The higher this rate is, the fewer charts that must be reviewed at providers’ offices. The coverage rate that includes chart reviews is referred to as the hybrid reported rate—the final HEDIS measure reported to NCQA. To simulate the impact of IIS data on Priority Health’s final-sample-size administrative rate, Priority Health’s MedMeasures software (ViPS, Inc, Baltimore, MD) was loaded with historical claims files only to regenerate final-sample-size administrative rates for 2004 to 2007. That enabled us to simulate how IIS data reduced chart reviews for HEDIS.

Next, we simulated the impact of IIS data on Priority Health’s annual PIP settlement, which calculates and distributes incentive payments, by estimating how IIS data reduced chart reviews for PIP. The number of doses from claims, MCIR, and supplemental submissions for 2003 to 2007 were reviewed. During the settlement, when Priority Health has no IIS data or claims record for a dose, providers are permitted to use a supplemental submission to document that the dose was administered and should be counted toward the incentive payment calculation. Currently, that submission is required to be a printed record from MCIR.

Before 2004, providers were alerted to missing doses via an automated message from Priority Health’s patient registries. Providers then faxed patient charts, which Priority Health reviewed to update patient registries. (All supplemental submissions were assumed to be for members in commercial products: only members in HMO and point-of-service plans may be counted toward an incentivized measure, and providers have no incentive to furnish data beyond claims for members participating in other products. Data for members not in HMO or point-of-service plans were excluded from the analysis.) Priority Health indicated hat the historical provider response rate to the automated messages was approximately 90%. The final-sample-size administrative rate and the 90% response rate from 2003 were held constant and applied to the annual number of members reaching the birthday milestone in each of the following years. This calculation yielded the number of supplemental submissions Priority Health would have expected to receive and process had it not downloaded IIS data. The difference between simulated and observed submissions was used to estimate the additional chart reviews that would have otherwise been conducted.

Cost-Benefit AnalysisOne-time and ongoing costs were predominantly limited to the Information Systems group; Priority Health expended little effort beyond notifying providers of the policy change because it estimated that 80% to 90% of its providers were already entering immunizations into MCIR. Labor effort was monetized using the mean hourly wage ($38.08) for “computer software engineers, systems software” in Michigan for 2007, as compiled by the Bureau of Labor Statistics.¹⁷ The wage rate was multiplied by 2 to account for administrative, overhead, and fringe expenses. (Estimates for administrative, overhead, and fringe expenses were applied because actual rates are proprietary to Priority Health and could not be disclosed.) No additional hardware or software was purchased. All dollar values are in real terms (2007).

Registered nurse subcontractors perform chart reviews for Priority Health at an estimated cost of $6 per review. Horne et al¹⁸ estimated the complete cost of 1 chart review to be $14.50, including costs for pulling and refiling charts. The lower estimate used in this analysis was only for the review and excluded chart pulling and refiling costs, which Priority Health does not incur but which were included by Horne and colleagues.

RESULTS

Results from Key Informant Interviews From Priority Health’s perspective, MCIR performs an electronic data interchange role (Figure). The Michigan Care Improvement Registry contains Priority Health’s unique patient identifiers, and vice versa. Providers can insert or update members’ MCIR identifiers stored in Priority Health’s internal registry through a secure Web portal. Data from MCIR and claims are merged weekly and matched using a proprietary algorithm. In a recent merge, Priority Health manually reviewed only 100 out of 7400 records. The Michigan Public Health Institute developed multiple tools for providers to load data into MCIR, including an interface for accepting automatic batch uploads from practice management software. The Michigan Public Health Institute also assists providers with MCIR account setup, training, outreach and education, and ongoing technical assistance, per its contract with MDCH. The MCIR benefits from Priority Health’s requirement that providers enter members’ immunizations because this requirement drives data to the IIS, furthering MDCH’s immunization surveillance goals.

Priority Health’s immunization measures are sensitive to members’ ages, and accurate and timely data are required to track performance. Priority Health believes that IIS data offer 4 advantages. First, MCIR is a single point of electronic data entry, and that enables Priority Health to leverage Michigan providers’ legal obligation to record childhood immunizations in MCIR.

Second, MCIR enables providers to demonstrate progress in meeting immunization measures at no additional cost or reporting burden, irrespective of whether a claim was filed. Priority Health indicated that claims forms may not be completed correctly or providers may neglect to mark forms for immunizations. Under capitation, there is little financial penalty for billing errors made by providers or their staff, and claims may underreport immunizations.

Third, MCIR gives Priority Health an opportunity to acquire immunization data on members who receive immunizations outside the provider network (eg, health departments) and members who transfer to Priority Health from other health plans. Although Priority Health’s incentives do not encourage nonnetwork providers to provide updates to MCIR, Priority Health does encourage in-network providers to do so by obtaining records from the parents or from other providers directly if the in-network providers did not provide updates to MCIR. Notably, we were unable to quantify the extent to which IIS data permitted providers and Priority Health to avoid extraimmunization.

Fourth, when launching PIP, Priority Health designed the program to be transparent by using HEDIS-based benchmarks, permitting providers to update data via supplemental submissions, and providing tools for providers to monitor their progress toward achieving benchmarks. Using the same physician-reported data for HEDIS and PIP furthered these transparency goals.

Results from Simulations and Cost-Benefit Analysis

Table 1 illustrates Priority Health’s immunization coverage rates with and without IIS data for 2007, before chart reviews. For childhood immunizations, IIS data increased demonstrated coverage rates from 20.72 to 54.13 percentage points, with the exception of the measles, mumps, and rubella vaccine and the varicella vaccine, for which coverage rates from claims were relatively high. For the childhood combination, IIS data increased the demonstrated up-to-date immunization rate from 35.09% to 88.80%. For the adolescent combination, the impact of IIS data on up-to-date immunization coverage was even more pronounced, increasing it from 2.82% to 74.62%.

Table 2 illustrates HEDIS rates for childhood and adolescent immunizations for 2003 through 2007. With the inclusion of IIS data to calculate the administrative rate for 2004, the rate of childhood immunizations nearly doubled from 43.38% in 2003 to 80.89%. The number of charts reviewed for HEDIS declined from 582 in 2003 to 51 in 2007. During that period, 1481 fewer charts were manually reviewed for HEDIS, translating into savings of $8886.

Leveraging MCIR was useful for the 4 years during which up-to-date adolescent immunization was a HEDIS and PIP measure. The historical number of supplemental submissions between 2004 and 2007 was observed to be 1108 (Table 3). Assuming the supplemental submission rate would hold in the absence of IIS data, additional chart reviews for PIP would likely have totaled 19,989. Thus, 18,881 additional submissions would have been reviewed. At a $6 processing cost per submission, the savings amounted to $113,286 between 2004 and 2007.

Priority Health’s network engineers and software programmers spent 3.5 weeks of labor effort to program Priority Health’s systems to receive data from MCIR, a one-time cost of $10,662. Ongoing support above business-as-usual levels to accept IIS data was estimated at 1 hour per month, or $914 per year. Total costs over the period were estimated to be $14,318, benefits were $122,172, and net benefits were $107,854 (Table 4). Assuming a discount rate of 10%, the benefit-to-cost ratio was 8.06. (The benefit-to-cost ratio is the ratio of the net present value of benefits to the net present value of costs, given a common discount rate.) For every dollar Priority Health spent, a savings of $8.06 was realized.

DISCUSSION

Priority Health’s quality assurance programs are predicated on the ability to obtain timely and accurate data, as well as providers’ perceptions that the data informing the measures are true. IIS data permit Priority Health to overcome the known deficiencies of claims data for performance measurement.^19-21 Although Priority Health accrues cost savings that amount to a large return on investment, Priority Health believes the principal source of value is enhanced relations with providers. The IIS data also increased Priority Health’s assurance that its members were up to date on immunizations and permitted providers to quickly determine the immunization status of their patients.

Certain factors may limit the generalizability of Priority Health’s experience. This case study included only 1 health plan, which may represent as much as half the membership of its providers’ practices, particularly in and around its core service area of Grand Rapids. We also could not ascertain the extent to which Priority Health members are better represented in MCIR than members of other health plans and how Priority Health may have impacted MCIR’s effectiveness measures. Except for Medicaid patients, MCIR does not record individuals’ insurance plan information.

The quality of MCIR’s IIS data has been affirmed by NCQA. Priority Health did not perform its own cost-benefit analyses a priori. Rather, Priority Health’s decision to accept IIS data was contingent on NCQA assurance of the quality and the acceptability of using IIS data for HEDIS audits. Without an affirmation of MCIR’s data quality, Priority Health would have been reluctant to expose its information systems to the uncertainties of an additional data source. Other IIS, however, may not have the participation levels or support that would enable them to be used by health plans for HEDIS. In particular, although all but 4 states have some form of IIS, there is considerable state-to-state variation in IIS capacities to perform the data-collection and reporting functions that are found within MCIR. Currently, 17 states report that 95% to 100% of children have 2 or more shot records in their states’ IIS.²² Anecdotal reports indicate that among this number, only a few states have IIS programs that may approach MCIR in terms of the level of participation. Other health plans may benefit from using IIS as a single point of immunization data entry for providers, following NCQA review of the quality and acceptability of an IIS.

CONCLUSION

The relationship between Priority Health and MCIR is further evidence of the growth of public—private partnership: leveraging complementary mandates for mutual benefit. Health plans can drive data to IIS, which helps the public health community monitor immunization levels with greater assurance. In turn, the health plan and providers benefit from reliable timeseries data and a single point of data entry into IIS.

Studying relationships between additional health plans and their local IIS may offer further insights. Such insights may clarify barriers and facilitators to implementation, enable more reliable cost measures, and indicate how the IIS may contribute to improving healthcare quality. These insights may be useful for health plans attempting to improve their quality measures, public health agencies interested in partnering with private health plans, and other stakeholders within the pubic and private healthcare sectors. Particularly in light of healthcare reform and increasing interest in electronic health records, IIS appear to offer a useful and versatile tool.

Author Affiliations: From RTI International (ACO, CML), San Francisco, CA; Priority Health (TJO), Grand Rapids, MI; Hoyle Consulting Inc (TMH), Delton, MI; and the Centers for Disease Control and Prevention (BR), Atlanta, GA.

Funding Source: This study was funded by the Centers for Disease Control and Prevention.

Author Disclosure: Mr Osbeck reports having a patent pending for the patient registry tool mentioned in this study. The other authors (ACO, CML, TMH, BR) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (ACO, CML, TJO, BR); acquisition of data (CML, TJO, TMH, BR); analysis and interpretation of data (ACO, CML, TJO); drafting of the manuscript (ACO, CML); critical revision of the manuscript for important intellectual content (ACO, TJO, TMH, BR); statistical analysis (ACO); provision of study materials or patients (TJO, TMH); obtaining funding (CML, BR); administrative, technical, or logistic support (CML, TJO, TMH, BR); and supervision (CML, BR).

Address correspondence to: Alan C. O’Connor, MBA, RTI International, 114 Sansome St, Ste 500, San Francisco, CA 94104. E-mail: oconnor@rti.org.

1. National Vaccine Advisory Committee (NVAC). Development of Community- and State-Based Immunization Registries: Report of the National Vaccine Advisory Committee. Approved January 12, 1999. US Department of Health and Human Services, Centers for Disease Control and Prevention. http://www.hhs.gov/nvpo/report071100.pdf. Accessed December 11, 2008.

2. Centers for Disease Control and Prevention (CDC). Immunization information systems progress—United States 2006. MMWR Morb Mortal Wkly Rep. 2008;57(11):289-291. http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5711a4.htm. Accessed January 25, 2009.

3. American Immunization Registry Association and the Every Child by Two Campaign. Partnering With Health Plans: A Practical Guide. A Resource for Immunization Registry Staff. http://www.ecbt.org/registries/media/pdf/RegistryHealthPlanGuide.pdf. Accessed December 11, 2008.

4. Rask KJ, LeBaron CW, Starnes DM. The costs of registry-based immunization interventions. Am J Prev Med. 2001;21(4):267-271.

5. Fontanesi JM, Flesher DS, De Guire M, Lieberthal A, Holcomb K. The cost of doing business: cost structure of electronic immunization registries. Health Ser Res. 2002;37(5):1291-1307.

6. McKenna VB, Sager A, Gunn JE, Tormey P, Barry MA. Immunization registries: costs and savings. Public Health Rep. 2002;117(4):386-392.

7. Glazner JE, Beaty BL, Pearson KA, Lowery NE, Berman S. Using an immunization registry: effect on practice costs and time. Ambul Pediatr. 2004;4(1):34-40.

8. Bartlett DL, Washington ML, Bryant A, Thurston N, Perfili CA. Cost savings associated with using immunization information systems for Vaccines for Children administrative tasks. J Public Health Manag Pract. 2007;13(6):559-566.

9. Bordley WC, Dempsey-Tanner T, Freed GL, Lister ME. Challenges to private provider participation in immunization registries. Am J Prev Med. 1997;13(2):66-70.

10. Clark SJ, Cowan AE, Bartlett DL. Private provider participation in statewide immunization registries. BMC Public Health. 2006;6:33.

11. Kelly JS, Zimmerman LA, Reed K, Enger KS. Immunization Information Systems national research and evaluation agenda. J Public Health Manag Pract. 2007;13(1):35-38.

12. Kilpatrick KE, Lohr KN, Leatherman S, et al. The insufficiency of evidence to establish the business case for quality. Int J Qual Health Care. 2005;17(4):347-355.

13. Chernew M, Gowrisankaran G, McLaughlin C, Gibson T. Quality and employers’ choice of health plans. J Health Econ. 2004;23(3):471-492.

14. Michigan Department of Community Health, Division of Immunization. Michigan Childhood Immunization Registry (MCIR) business plan 2006-2007. Lansing, MI. 15. Griliches Z. Research costs and social returns: hybrid corn and related innovations. J Political Econ. 1958;66(4):370-371.

16. Mansfield E, Rapoport J, Romeo A, Wagner S, Beardsley G. Social and private rates of return from industrial innovations. Q J Econ. 1977;91:221-240.

17. Bureau of Labor Statistics (BLS). Occupational employment statistics: May 2007 state occupational and wage estimates—Michigan. http://www.bls.gov/oes/2007/may/oes_mi.htm. Accessed February 11, 2010.

18. Horne P, Saarlas K, Hinman AR. Costs of immunization registries: experiences from the All Kids Count II Projects. Am J Prev Med. 2000:19(2):94-98.

19. Dresser MV, Feingold L, Rosenkranz SL, Coltin KL. Clinical quality measurement. Comparing chart review and automated methodologies. Med Care. 1997:35(6):539-552.

20. Naessens JM, Ruud KL, Tulledge-Scheitel SM, Stroebel RJ, Cabanela RL. Comparison of provider claims data versus medical records review for assessing provision of adult preventive services. J Ambul Care Manage. 2008;31(2):178-186.

21. Pawlson LG, Scholle SH, Powers A. Comparison of administrativeonly versus administrative plus chart review data for reporting HEDIS hybrid measures. Am J Manag Care. 2007;13(10):553-558.

22. Centers for Disease Control and Prevention (CDC). 2008 Child Participation Map: Percentage of U.S. Children >4 months and <6 years with 2+ Immunizations in Immunization Information Systems (IIS), 2008. http://www.cdc.gov/vaccines/programs/iis/rates/2008-child-map.htm. Accessed November 16, 2009.

Download PDF: Health Plan Use of Immunization Information Systems for Quality Measurement