Importance of Reasons for Stocking Adult Vaccines

ABSTRACT

Objectives: To identify the most important reasons underlying decisions to stock or not stock adult vaccines.

Study Design: US physicians, nurses, pharmacists, and administrators of internal medicine, family medicine, obstetrics/gynecology, and multispecialty practices who were involved in vaccine stocking decisions (N = 125) completed a best—worst scaling survey online between February and April 2018.

Methods: Sixteen potential factors influencing stocking decisions were developed based on key informant interviews and focus groups. Respondents selected factors that were most and least important in vaccine stocking decisions. Relative importance scores for the best—worst scaling factors were calculated. Survey respondents described which vaccines their practice stocks and reasons for not stocking specific vaccines. Subgroup analyses were performed based on the respondent’s involvement in vaccine decision making, role in the organization, specialty, and affiliation status, as well as practice characteristics such as practice size, insurance mix, and patient age mix.

Results: Relative importance scores for stocking vaccines were highest for “cost of purchasing vaccine stock,” “expense of maintaining vaccine inventory,” and “lack of adequate reimbursement for vaccine acquisition and administration.” Most respondents (97%) stocked influenza vaccines, but stocking rates of other vaccines varied from 39% (meningococcal B) to 83% (tetanus-diphtheria-pertussis). Best—worst scaling results were consistent across respondent subgroups, although the range of vaccine types stocked differed by practice type.

Conclusions: Economic factors associated with the purchase and maintenance of vaccine inventory and inadequate reimbursement for vaccination services were the most important to decision makers when considering whether to stock or not stock vaccines for adults.

Am J Manag Care. 2019;25(11):e334-e341Takeaway Points

• Prior research has shown that many providers do not stock adult vaccines and this can be a barrier to patients receiving recommended vaccinations.
• We surveyed US physicians, nurses, pharmacists, and administrators involved in vaccine stocking decisions to understand barriers to stocking vaccines for adults.
• We used novel best—worst scaling methods (from marketing) to quantify how decision makers rank the relative importance of different barriers.
• Relative importance scores for stocking vaccines overall were highest for “cost of purchasing vaccine stock,” “expense of maintaining vaccine inventory,” and “lack of adequate reimbursement for vaccine acquisition and administration.”

Although vaccination rates for childhood vaccines are high in the United States, rates are lower for adult vaccines.¹ The United States has achieved the Healthy People 2020 (HP2020) target of 30% for the shingles vaccine but not the adult influenza or pneumococcal immunization targets.^1,2 The HP2020 target is for 70% of noninstitutionalized adults to be vaccinated for influenza,² but just 43.3% of adults were immunized at the end of the 2016-2017 season.³ The HP2020 targets for pneumococcal vaccination are 60% for high-risk adults aged 18 to 64 years and 90% for noninstitutionalized adults 65 years or older,² yet coverage rates are stagnant at 24% and 67%, respectively.⁴ Estimates from Medicare claims data show even lower pneumococcal vaccine coverage for adults 65 years or older, at 59% for any pneumococcal vaccine and just 24% for both the pneumococcal polysaccharide vaccine (PPSV23) and the pneumococcal conjugate vaccine (PCV13).⁵

Provider recommendations are a key predictor of adult vaccination; however, providers are less likely to recommend vaccines they do not stock.^6,7 A survey conducted in 2009 of family practice and internal medicine physicians found that many do not stock all adult vaccines and that reimbursement was a barrier to stocking adult vaccines.⁸ A survey conducted in 2012 of primary care providers found that financial barriers were important to decisions about recommending and stocking vaccines for adults.⁹ More recent surveys of providers have reported dissatisfaction with reimbursement levels in relation to their costs and profitability varying widely between payers.^10,11 A survey conducted in 2015 of obstetricians and gynecologists found that many adult vaccines were not stocked and that financial barriers were most important to stocking decisions.¹² Most of these studies have focused on general internal medicine and family medicine providers, and several have asked about how these barriers affect both recommending and stocking vaccines. The majority of these studies also relied on a Likert scale to understand how providers feel about the barriers. In the time since those studies were published, there have been changes in insurance reimbursement for vaccines with the Affordable Care Act, vaccination has expanded in pharmacies, and pharmacists are able to administer vaccines in all 50 states.¹³

In this current study, we seek to identify the most important reasons underlying decisions to stock adult vaccines. We focus on provider stocking decisions and expand the pool of providers beyond those studied in the past to include pharmacists. We also expand beyond general internal medicine and family medicine physicians to survey physicians specializing in obstetrics and gynecology (OB/GYN) and multispecialty practices. This study uses an innovative best—worst scaling methodology from the field of marketing to quantify the ranking of importance of these barriers.¹⁴

METHODS

Best—worst scaling is a survey approach used to identify the relative values associated with objects.¹⁴ It does so by repeatedly asking respondents to identify the most important and least important objects from different lists of objects. Best—worst scaling has been shown to be more reliable than Likert rating scales¹⁵ and can be used to avoid scale biases.^16,17 In this analysis, the objects that we are interested in ranking are the factors that may influence vaccine stocking decisions.

Survey Development, Design, and Administration

We used qualitative data from 3 focus groups (n = 12 total participants) and key informant interviews (n = 9 key informants) along with relevant literature^6-12 to identify 16 factors potentially influencing vaccine stocking decisions (Table 1). The focus groups and key informant interviews involved vaccine stocking decision makers in medical practices and pharmacies and were conducted from January to October 2017 using a standardized interview guide to identify potential factors influencing these decisions.

Based on the results from the focus groups and key informant interviews, we developed a survey to assess the importance of factors in stocking vaccines. The survey included an introduction to the factors and their definitions, as shown in Table 1. Each respondent was asked to assess 4 sets of 6 factors selected from the overall 16 that were developed using a balanced incomplete block design methodology to ensure even representation of factors across survey versions (Figure 1).¹⁸ For each question, respondents were asked to select the factors that were most and least important in the decision to stock vaccines in general. Respondents were randomized to 1 of 4 versions of the survey. Finally, survey respondents were presented with a list of 11 commonly used adult vaccines and asked which vaccines their practice stocks and the reasons for not stocking specific ones. We also collected information on each respondent’s role in their organization, practice type, organization affiliation, characteristics of the practice, and at what organizational level the decisions to stock vaccines are made.

The survey underwent cognitive pretesting to ensure that respondents understood the questions as intended. In April 2018, the survey was administered online to a standing panel (Qualtrics; Seattle, Washington) of US physicians, nurses, pharmacists, and administrators involved in vaccine stocking decisions. Although this was a convenience sample, respondents represented diverse regions across the United States.

Analysis

We calculated relative importance scores for each of the 16 factors that influence vaccine decision making. For each factor, we obtained the number of times it was chosen as most important and the number of times it was chosen as least important. We subtracted the number of times a factor was chosen as least important from the number of times it was chosen as most important. We then divided this difference by the number of times each factor was available to be chosen as least or most important. The availability-adjusted relative importance scores were then ranked from the highest to the lowest to assess the relative importance of each of the 16 factors on vaccine decision making (Figure 2).¹⁹

Subgroup Analyses

Subgroup analyses were performed based on the respondent’s involvement in vaccine decision making, role in the organization, specialty, and affiliation status, as well as practice characteristics (ie, practice size, insurance mix, and patient age mix). We also performed stratified analysis by type of vaccine stocked to identify the factors that are important for stocking decisions.

Regression Analysis

In order to examine the overall relationships between factors and stocking of specific vaccines, we conducted a set of logistic regressions evaluating the probability of stocking individual vaccines based on respondent characteristics and the respondent relative importance scores of the factors determined to be most important from the best—worst scaling analysis. Because we conducted multiple regressions, we used the Bonferroni correction to assess statistical significance.

RESULTS

A total of 125 providers completed the survey. Most respondents (56%) were physicians, 19% were pharmacists, 14% were nurses, and a combined 10% were practice managers or health system administrators (Table 2 [part A and part B]). Many were in private, independent organizations (44%) and/or hospitals/academic medical centers (38%). There were a wide variety of specialties represented, with most in OB/GYN (36%), pharmacy (19%), family medicine (18%), and internal medicine (14%).

Physician office respondents were from a broad array of practice sizes, with 13% in a solo practice and 29% in organizations with 13 or more providers (Table 2). Of pharmacists, 29% were in organizations that filled 100,000 or fewer prescriptions per year and 25% were in organizations that filled more than 250,000 per year. Additional respondent characteristics can be found in eAppendix Table 1 (eAppendix available at ajmc.com).

Vaccine Stocking

Most respondents’ organizations stocked at least 1 type of the 11 vaccines considered in this study. The most common vaccine stocked (97% of organizations) was for influenza (Table 2). However, other vaccines, such as meningococcal B, were stocked by fewer than 50% of organizations.

As seen in Table 3, some respondent characteristics were associated with higher stocking rates. Pharmacists were more likely to stock pneumococcal and zoster vaccines than physicians. Internal medicine/family medicine physicians were significantly more likely to stock almost all vaccines (except influenza and tetanus-diphtheria-pertussis [Tdap]) than OB/GYN physicians. Clinics affiliated with hospitals and/or academic medical centers were also significantly more likely to stock almost all vaccines (except influenza, Tdap, and hepatitis A and B) than private, independent clinics. Larger practices with 6 or more physicians were significantly more likely to stock PCV13, zoster, hepatitis A, hepatitis A and B, and meningococcal ACWY and B vaccines than smaller practices.

Of those respondents who replied that they did not stock a specific vaccine, we asked for reasons why that was the case. By far the most common reason across all vaccines was that stocking that vaccine was “not a priority for our practice/organization” (between 54% and 80% of responses). The second most common response for not stocking influenza, PCV13, PPSV23, Tdap, tetanus-diphtheria (Td), and hepatitis B vaccines was that it was “challenging to keep up with the changes to recommendations for this vaccine” (between 8% and 25% of responses). For other vaccines (zoster, hepatitis A, hepatitis A and B, meningococcal ACWY, and meningococcal B), the second most common response was that the “vaccine is too costly/purchase and inventory management costs” (between 9% and 22% of responses). Details can be found in eAppendix Figure 1.

Importance of Barriers to Stocking Vaccines

The best—worst scaling analysis revealed that the most important reasons were the “cost of purchasing vaccine stock,” “expense of maintaining vaccine inventory,” and “lack of adequate reimbursement for vaccine acquisition and administration” (Figure 2). Other insurance-related issues were of medium or lower importance: “Patients not having consistent insurance coverage for vaccines” was more likely to be listed as most important than least important, whereas “problems with vaccine claims getting rejected,” “prior authorization needed for some or all vaccines,” and “time required interacting with insurance concerning vaccine claims” were about equally likely to be listed as most important or least important. “Patients’ insurance coverage out-of-network for vaccines” was more likely to be chosen as least important than most important. Of non–insurance-related issues, “patient attitudes toward vaccination,” “patient out-of-pocket costs for some vaccines,” and “loss due to expired vaccines” were about as likely to be listed as most important or least important. Other issues were more likely to be ranked as least important.

Subgroup Analyses

We examined the best—worst rankings by characteristics of the respondents, as well as whether they stocked specific vaccines. Although the rankings of medium- or lower-importance factors changed, the top 3 factors remained the “cost of purchasing vaccine stock,” “lack of adequate reimbursement for vaccine acquisition and administration,” and the “expense of maintaining vaccine inventory” regardless of subgroup analyzed (eAppendix Figure 2). The best—worst scaling of barriers was also not different based on whether the respondents stocked or did not stock specific vaccines (eAppendix Figure 3).

Patient demand and patient attitudes about vaccines were generally of minor importance or the least important barriers to stocking, suggesting that patient interest is sufficiently high across the various specialties to not adversely affect stocking decisions. The potential exceptions to this are pharmacists, practice managers, and administrators, for whom “little demand for vaccines from patients” and “patient attitudes toward vaccination” were of moderate importance. Also of potential interest, practices with more than 50% of their patients 65 years or older ranked patient demand as being of the least concern.

Regression Analysis

We evaluated 10 logistic regressions for each vaccine, with the exception of the influenza vaccine because variation in stocking was so low. The most important relative importance factors included in the regression were cost of purchasing, expense of maintaining inventory, and inadequate reimbursement. In these regressions, some factors were associated with stocking of particular vaccines (eAppendix Table 2). The OB/GYN group had lower stocking of both pneumococcal vaccines, Td, hepatitis A, hepatitis B, and both meningococcal vaccines. Hospitals/academic medical centers were more likely to stock Td and meningococcal B vaccines. Providers with 51% to 100% of their patients aged 19 to 64 years were less likely to stock the zoster vaccine.

DISCUSSION

Using novel best—worst scaling survey methods and surveying a wide range of providers, we found that issues directly related to the revenue and expenses associated with vaccination were the most important barriers to stocking adult vaccines. The overall reimbursement level compared with the costs of purchasing and maintaining inventory were consistently rated as the most important factors, regardless of subgroup analyzed. Challenges dealing with insurance companies, lack of patient demand, and difficulty understanding recommendations were not as important barriers to deciding to stock vaccines for adults.

These results are similar to those of other studies.^8-12 Those studies found that general internal medicine, family medicine, and OB/GYN practices identified financial barriers as the most important barriers to recommending and stocking vaccines. We found this to be the case when focusing on just stocking vaccines but also when surveying a broader set of providers, including pharmacists. The best—worst scaling methodology seems to yield similar results as the Likert scale questions used in other studies. We also find that issues such as patients refusing vaccines or lack of vaccine demand are of low importance as barriers. This is consistent with the findings of other publications that demonstrated that adults are interested in vaccines in general but often report that their provider did not mention vaccines during their visit.²⁰

Although influenza vaccines were stocked by almost all providers, other adult vaccines were stocked to varying extents. Pharmacists had high rates of stocking most vaccines, but they were particularly more likely to stock pneumococcal and zoster vaccines, both of which are recommended for older adults. In our sample, pharmacists were the group with the highest proportion of patients older than 65 years (eAppendix Table 3). The live zoster vaccine is recommended at 60 years or older; however, it is covered under Medicare Part D (a pharmacy benefit), which may be challenging for medical providers to bill but can be billed with less difficulty by pharmacists. In 2018, the Advisory Committee on Immunization Practices (ACIP) recommended a new inactivated zoster vaccine for individuals 50 years or older.²¹ This new recommendation may result in more adults being vaccinated at a younger age and before they are on Medicare, which may affect stocking of zoster vaccines by practices that are less able to bill Medicare Part D. Meanwhile, although OB/GYN physicians were less likely to stock most vaccines, they frequently reported stocking influenza and Tdap vaccines, which are the only 2 vaccines routinely recommended during pregnancy.^22,23

We found some slight differences between our results and the prior literature. Studies conducted in 2012⁹ and 2015¹² found that about 60% of providers listed “difficulty determining if a patient’s insurance will reimburse for a vaccine” and “patients not having insurance coverage for vaccines” as being “major barriers” or “moderate barriers.” We did not ask the exact same questions used in prior studies, but we found that “patients not having consistent insurance coverage” was somewhat more likely to be listed as most important than least important and that “time required interacting with insurance concerning vaccine claims” was about equally likely to be listed as most important as least important than in prior studies. It is possible that as time has passed since the passage of Affordable Care Act, some of those issues have become less important as health insurers and nongrandfathered health plans must cover immunizations recommended by the ACIP. However, issues of claims rejection and inadequate payment appeared to be highest among respondents who reported having more than 25% of their patients on Medicaid. Medicaid payments for vaccines and vaccination vary widely by state and may be more of a barrier in some areas.²⁴

It is challenging to compare respondent reasons for not stocking specific vaccines with their answers to the best—worst scaling questions about barriers in general. When answering about not stocking specific vaccines, between 54% and 80% of respondents noted that the vaccine was not a priority. Between 4% and 25% of respondents noted that it was challenging to keep up with changes to recommendations for that specific vaccine. In addition, the fraction of respondents saying that the reason for not stocking was that the “vaccine is too costly/purchase and inventory management costs” varied from 0% for influenza to 22% for zoster. Similarly, 0% of respondents said that “insufficient payment” was a barrier for influenza, but 20% said it was a barrier for zoster. However, when responding to the best–worst scaling questions about barriers in general, financial barriers were most important and difficulty interpreting guidelines was not ranked highly. This apparent contradiction is possibly because respondents may have subsumed financial considerations into the “not a priority” category when answering about specific vaccines.

Implications for Practice

This study can shed some light on actions to encourage stocking of vaccines for adults. From our results, arrangements that mitigate the up-front and maintenance costs of vaccine inventories could result in increased stocking of adult vaccines by providers. Higher insurance reimbursement levels for both the vaccine and its administration, or lower costs to administer vaccines, may reduce financial barriers to stocking adult vaccines. Reimbursement levels vary by payer and costs to vaccinate vary by provider, so appropriate solutions for a given practice likely depend on payer mix and type of vaccination provider.

Problems with claims being rejected were listed as a moderate concern. More accurate claims submission and processing—potentially assisted by third-party vaccine billing solutions for provider offices that require such assistance—may help. In addition, better knowledge about reasons for and rates of vaccine nonpayment may help providers to make better decisions about stocking vaccines and improve billing accuracy.

Other strategies may not be as effective at encouraging stocking of vaccines. Healthcare Effectiveness Data and Information Set (HEDIS) measures are commonly used in the insurance industry and can be used to incentivize providers.²⁵ Although there are several HEDIS measures that pertain to vaccination, in this provider population surveyed, lacking rewards for meeting immunization targets was not perceived to be an important barrier to stocking vaccines. However, at the time of the survey, only influenza and pneumococcal vaccination HEDIS measures were available for adults. HEDIS 2019 will include additional adult vaccination measures (a prenatal composite vaccination measure with Tdap and influenza and an adult composite measure for influenza, pneumococcal, Td/Tdap, and zoster vaccines).²⁶ It may be that some of the factors that we found to be less important contribute indirectly to adult vaccination. For example, rewards or penalties for meeting immunization targets may encourage providers to recommend vaccines.

Nevertheless, there may be some barriers that are insurmountable for certain medical providers. In those cases, there are benefits of having places like a local pharmacy available to refer patients for immunizations. Continuing efforts should be made to ensure that those referral processes are seamless.

Limitations

Our survey used a convenience sample of physicians, nurses, pharmacists, and administrators from an existing internet panel. Therefore, responses may not be representative of all vaccination providers in the United States. With that said, rates of stocking were similar to those reported by Hurley et al,⁹ who used quota sampling of family medicine and internal medicine physicians.

This study focused only on stocking vaccines, but there may be other important provider-related barriers to patient immunization even if their providers stock required vaccines. These could include incorporation of patient vaccine needs assessment and recommendations into patient flow, as well as adequate staff time to implement vaccination services.²⁷ Additional barriers not closely related to providers may include mistaken assumptions by patients (eg, healthy people do not need immunizations), fear of adverse events, and infrequent provider visits.²⁰

Our best—worst scaling questions asked about vaccines in general. To minimize survey burden, we did not ask these questions about each vaccine individually. This makes it more difficult to make conclusions about barriers to stocking specific vaccines.

CONCLUSIONS

Not all adult vaccines are stocked by providers who care for adults. This new study finds that financial barriers are reported as the most important issues influencing decisions about which vaccines to stock among a wide range of providers, including pharmacists and OB/GYN physicians. Efforts to reduce the actual and perceived risks of insurance claim rejection and address issues of vaccine payment adequacy might help encourage more providers to stock the full complement of routinely recommended vaccines for their adult patients.Author Affiliations: University of Michigan (DWH, AR, DCS, LAP), Ann Arbor, MI; Immunization Services Division, National Center for Immunization and Respiratory Disease, CDC (CBB, DK, JP), Atlanta, GA; Berry Technology Solutions Inc (CBB), Peachtree City, GA.

Source of Funding: This research was supported as part of a cooperative agreement with the CDC, 5-U01-IP-000965-03. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (DWH, DCS, CBB, DK, JP, LAP); acquisition of data (DWH, AR, DCS); analysis and interpretation of data (DWH, DCS, CBB, DK, JP, LAP); drafting of the manuscript (DWH, AR, DCS, CBB, JP); critical revision of the manuscript for important intellectual content (DWH, DCS, CBB, DK, JP, LAP); statistical analysis (DWH); provision of patients or study materials (DCS); obtaining funding (DWH); administrative, technical, or logistic support (AR, CBB, DK, JP); and supervision (JP).

Address Correspondence to: David W. Hutton, PhD, Department of Health Management and Policy, University of Michigan School of Public Health, 1415 Washington Heights, Room M3525, Ann Arbor, MI 48109-2029. Email: dwhutton@umich.edu.REFERENCES

1. Williams WW, Lu PJ, O’Halloran A, et al; CDC. Surveillance of vaccination coverage among adult populations—United States, 2014. MMWR Surveill Summ. 2016;65(1):1-36. doi: 10.15585/mmwr.ss6501a1.

2. Immunization and infectious diseases. Healthy People website. healthypeople.gov/2020/topics-objectives/topic/immunization-and-infectious-diseases/objectives?topicId=23. Accessed August 8, 2018.

3. Flu vaccination coverage, United States, 2016-17 influenza season. CDC website. cdc.gov/flu/fluvaxview/coverage-1617estimates.htm. Updated September 28, 2017. Accessed October 12, 2018.

4. Immunization and infectious diseases: IID-13.1 increase the percentage of noninstitutionalized adults aged 65 years and older who are vaccinated against pneumococcal disease. Healthy People website. healthypeople.gov/2020/data-search/Search-the-Data#objid=4670;. Accessed December 18, 2018.

5. Black CL, Williams WW, Warnock R, Pilishvili T, Kim D, Kelman JA. Pneumococcal vaccination among U.S. Medicare beneficiaries aged ≥65 years, 2009-2017. CDC website. cdc.gov/vaccines/imz-managers/coverage/adultvaxview/pubs-resources/pcv13-medicare-beneficiaries.html. Updated September 4, 2018. Accessed October 18, 2018.

6. Ding H, Black CL, Ball S, et al; CDC. Influenza vaccination coverage among pregnant women—United States, 2013-14 influenza season. MMWR Morb Mortal Wkly Rep. 2014;63(37):816-821.

7. Schneeberg A, Bettinger JA, McNeil S, et al. Knowledge, attitudes, beliefs and behaviours of older adults about pneumococcal immunization, a Public Health Agency of Canada/Canadian Institutes of Health Research Influenza Research Network (PCIRN) investigation. BMC Public Health. 2014;14:442. doi: 10.1186/1471-2458-14-442.

8. Freed GL, Clark SJ, Cowan AE, Coleman MS. Primary care physician perspectives on providing adult vaccines. Vaccine. 2011;29(9):1850-1854. doi: 10.1016/j.vaccine.2010.12.097.

9. Hurley LP, Bridges CB, Harpaz R, et al. U.S. physicians’ perspective of adult vaccine delivery. Ann Intern Med. 2014;160(3):161-170. doi: 10.7326/M13-2332.

10. Lindley MC, Hurley LP, Beaty BL, et al. Vaccine financing and billing in practices serving adult patients: a follow-up survey. Vaccine. 2018;36(8):1093-1100. doi: 10.1016/j.vaccine.2018.01.015.

11. Hurley LP, Lindley MC, Allison MA, et al. Primary care physicians’ perspective on financial issues and adult immunization in the era of the Affordable Care Act. Vaccine. 2017;35(4):647-654. doi: 10.1016/j.vaccine.2016.12.007.

12. O’Leary ST, Riley LE, Lindley MC, et al. Immunization practices of U.S. obstetrician/gynecologists for pregnant patients. Am J Prev Med. 2018;54(2):205-213. doi: 10.1016/j.amepre.2017.10.016.

13. Burson RC, Buttenheim AM, Armstrong A, Feemster KA. Community pharmacies as sites of adult vaccination: a systematic review. Hum Vaccin Immunother. 2016;12(12):3146-3159. doi: 10.1080/21645515.2016.1215393.

14. Cheung KL, Wijnen BF, Hollin IL, et al. Using best—worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34(12):1195-1209. doi: 10.1007/s40273-016-0429-5.

15. Kiritchenko S, Mohammad SM. Best-worst scaling more reliable than rating scales: a case study on sentiment intensity annotation. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017;2:465-570. doi: 10.18653/v1/P17-2074.

16. Baumgartner H, Steenkamp JBEM. Response styles in marketing research: a cross-national investigation. J Mark Res. 2001;38(2):143-156. doi: 10.1509/jmkr.38.2.143.18840.

17. Baumgartner H, Steenkamp JBEM. Assessing measurement invariance in cross-national consumer research. J Consum Res. 1998;25(1):78-90. doi: 10.1086/209528.

18. Kuhfeld WF. Marketing research methods in SAS: experimental design, choice, conjoint, and graphical techniques. SAS website. support.sas.com/techsup/technote/mr2010.pdf. Published October 1, 2010. Accessed March 20, 2019.

19. Louviere JJ, Flynn TN, Marley AAJ. Best—Worst Scaling: Theory, Methods and Applications. Cambridge, United Kingdom: Cambridge University Press; 2015.

20. Johnson DR, Nichol KL, Lipczynski K. Barriers to adult immunization. Am J Med. 2008;121(7 suppl 2):S28-S35. doi: 10.1016/j.amjmed.2008.05.005.

21. Dooling KL, Guo A, Patel M, et al. Recommendations of the Advisory Committee on Immunization Practices for use of herpes zoster vaccines. MMWR Morb Mortal Wkly Rep. 2018;67(3):103-108. doi: 10.15585/mmwr.mm6703a5.

22. Grohskopf LA, Sokolow LZ, Broder KR, Walter EB, Fry AM, Jernigan DB. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices—United States, 2018-19 influenza season. MMWR Recomm Rep. 2018;67(3):1-20. doi: 10.15585/mmwr.rr6703a1.

23. Liang JL, Tiwari T, Moro P, et al. Prevention of pertussis, tetanus, and diphtheria with vaccines in the United States: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep. 2018;67(2):1-22. doi: 10.15585/mmwr.rr6702a1.

24. Stewart AM, Lindley MC, Cox MA. Medicaid provider reimbursement policy for adult immunizations. Vaccine. 2015;33(43):5801-5808. doi: 10.1016/j.vaccine.2015.09.014.

25. de Bruin SR, Baan CA, Struijs JN. Pay-for-performance in disease management: a systematic review of the literature. BMC Health Serv Res. 2011;11(1):272. doi: 10.1186/1472-6963-11-272.

26. NCQA updates quality measures for HEDIS 2019 [news release]. Washington, DC: National Committee for Quality Assurance; July 2, 2018. ncqa.org/news/ncqa-updates-quality-measures-for-hedis-2019/. Accessed December 6, 2018.

27. National Vaccine Advisory Committee. Recommendations from the National Vaccine Advisory Committee: standards for adult immunization practice. Public Health Rep. 2014;129(2):115-123. doi: 10.1177/003335491412900203. 

Download PDF: Importance of Reasons for Stocking Adult Vaccines