Pharmaceuticals From Development to Practice: Learnings From an Online Curriculum

Objective

: To report outcomes of a pilot online curriculum educating clinicians about off-label prescribing, finding unbiased sources of pharmaceutical information, and responding to patient inquiries about direct-to-consumer advertised medications.

Study Design

: Case study.

Methods

: We developed 3 case-oriented Internet modules to educate healthcare professionals about off-label prescribing, accessing and appraising unbiased drug information, and talking with patients about direct-to-consumer advertising. Modules included reflective questions and opportunities to asynchronously ask questions of faculty. Pilot testing occurred among 52 Kaiser Permanente Colorado prescribing clinicians from May 2007 through September 2007. We assessed self-reported changes in knowledge, intent-to-change practice, and barriers to practice change. We also examined curriculum participant pre—post changes in prescribing of selected highly advertised medications.

Results

: Most participants reported changes in knowledge after curriculum completion, many made intent-to-change practice statements at curriculum completion, and several reported changes in practice 3 to 6 months after curriculum completion. Six categories of barriers to practice change were noted. Prescribing of targeted medications rose by 0.8 prescription per month in the intervention group and by 6.3 prescriptions per month in the control group, but this difference was not significant (P = .17).

Conclusions

: Targeted curricula on aspects of the pharmaceutical development, approval, and marketing process can lead to subjective changes in clinician knowledge and can facilitate intent-to- change practice. Larger studies are needed to assess the effect of such interventions on prescribing patterns.

(Am J Manag Care. 2009;15(12):e109-e114)

• Interactive reflective online educational curricula can help educate and prompt clinicians to make practice changes in prescribing heavily marketed medications.
• Clinicians may not avail themselves of online curricula because of competing demands, multiple educational opportunities, and lack of recognition of the effects of pharmaceutical advertising.
• Curricula on off-label prescribing, accessing and appraising pharmaceutical information, and talking with patients about direct-to-consumer advertising should not be used in isolation but may be useful in multifaceted efforts toward more cost-effective prescribing.

Pharmaceutical spending represents an important and large part of US healthcare expenditures.¹ As costs of healthcare continue to outpace inflation,² the amount of money spent on pharmaceutical advertising³ and its influence on patients and clinicians^3-5 have come under increasing attention, especially in cases when newer heavily advertised medications offer marginal, if any, therapeutic advantage over more established agents.

In 2004, Pfizer, Inc (New York, NY) entered into a $430-million settlement with a consortium of state attorneys general relating to off-label marketing of Neurontin (gabapentin).^6,7 The consortium allocated some settlement funds to a grant for developing programs designed to improve the cost-effectiveness of prescribing by (1) educating health professionals about pharmaceutical development and approval processes, (2) increasing clinician awareness of and ability to evaluate pharmaceutical industry marketing techniques, and (3) providing strategies for accessing and evaluating drug information.

Kaiser Permanente Colorado (KPCO) cares for 440,000 members in the Denver-Boulder, metropolitan area via a group model health maintenance organization (HMO) and another 40,000 members in the Colorado Springs, area via a preferred provider organization (PPO) model. Kaiser Permanente Colorado was 1 of 24 initial grantees to receive educational curricular development grant funding under the Neurontin settlement. Using the results of a needs assessment that we conducted among HMO and PPO prescribing clinicians and KPCO pharmacy and other leaders,¹ we focused our curriculum on the following 3 topics: (1) off-label prescribing, (2) accessing and appraising unbiased drug information, and (3) talking with patients about direct-to-consumer advertising. This case study describes our curriculum and the results from the pilot phase of its deployment in KPCO.

Methods

We developed 3 Internet-based modules (1 for each topic) that were accredited for a total of four (4) AMA PRA Category 1 Continuing Medical Education (CME) Credit(s)™. Modules consisted of online self-paced presentations (PowerPoint; Microsoft, Redmond, WA). The presentations included didactic information (eg, facts and figures on prescribing rates, results from physician and patient surveys, and screenshots of evidencebased pharmaceutical information Web sites) that was supported by case examples using selected, heavily marketed antibiotics, antidepressants, and diabetes medications (target areas of focus for KPCO identified from the needs assessment as areas in which potential participants desired to improve prescribing). Modules also contained multiple-choice questions and interspersed reflective questions asking participants about past experiences with the module topics and about their intent-to-change practice after reviewing the material (eg, “Recall the last time a patient asked you about a specific medication advertised directly to the public. Which medication was it? What did you end up prescribing at that visit?”). We also included a series of tips on topics (such as a checklist for appraising articles on pharmaceutical effectiveness) that were also made available as downloadable pdf files for future use. Participants could pose questions (asynchronously) to module faculty (DWP and MAR), who responded by e-mail within 72 hours. At the completion of each module, participants were asked to complete an evaluation form and were given the option to access the other modules. The curricula are publicly available from the Federation of State Medical Boards (http://www.fsmb.org/re/open/modules.html). Copies of each of the modules are available on request from the author.

The evaluation at the end of each module asked participants to rate whether the module met each of its learning objectives (using a 4-point Likert-type scale) and to subjectively rate their improvement in knowledge (improved, somewhat improved, about the same, or less than before the module). To assess participant-perceived value of the modules and to assess the potential influence of the modules, we also asked participants to indicate what changes in practice they intended to make after module completion (open-ended question). Three to six months after completing the modules, participants were invited to complete a follow-up survey asking them to self-report the changes they had made and the barriers they experienced in making intended practice changes (open-ended questions).

Approximately 1800 potential participants were contacted and recruited in two 3-month waves initially by e-mail, supplemented with hard-copy mail for e-mail nonrespondents. Eligible participants included Colorado Permanente Medical Group physicians and KPCO allied health professionals of all specialties practicing in the Denver-Boulder area (HMO cohort) and physicians and allied health professionals in the Colorado Springs area affiliated with the KPCO PPO network.

Using administrative data from internal pharmacy databases, we also compared participant (intervention group) pre—post prescribing of brand-name medications from drug classes targeted in the curriculum (included based on the results of the needs assessment¹) 6 months after the curriculum compared with 12 months before the curriculum by Wilcoxon signed rank test. Interpretation of pre—post analysis results is subject to bias associated with temporal trends that can be unassociated with the study intervention. Therefore, to assist in elucidating temporal prescribing trends, we conducted the same comparison and analysis for noncurriculum participants (control group) by examining overall organizational prescribing trends for these medications, excluding data from curriculum takers. Prescribing information was assessed only in the Denver-Boulder cohort, for whom these data were more readily accessible. Although the number of study subjects in the intervention group was small (n = 52), we analyzed the number of target medications by comparing intervention group and control group prescribing before and after the curriculum using a generalized estimating equation analysis with a Poisson distribution, an efficient method for modeling count data (number of prescriptions). The interaction of group and time (before and after the curriculum) was entered as the independent variable, with the number of prescriptions for targeted brand-name medications as the dependent variable. The group-×-time interaction term was used as the independent variable to minimize the possible temporal bias that may exist when looking at prescription counts over time. Using a comparison group in the analysis (rather than a pre–post design with only the intervention group) helps to control for this potential temporal bias.

Results

During the pilot study (May 2007 through September 2007), 52 of approximately 1800 eligible clinicians enrolled in the curriculum. Forty-seven clinicians (33 physicians and 14 allied health professionals) completed at least 1 module; 8 of 47 were from the PPO. Fourteen participants had been in practice 20 years or more and 12 participants for 5 years or less; 14 participants had practiced with KPCO for more than 10 years and 12 participants for less than 2 years. Sixteen physicians (8 each in family medicine and internal medicine) and 2 allied health professionals were primary care providers, 5 physicians and 1 nurse practitioner were obstetrics and gynecology specialists, and the remaining participants worked in psychiatry, general surgery, orthopedic surgery or trauma, ophthalmology, dermatology, endocrinology, palliative care, or hospital medicine. One dental surgeon participated; 6 allied health professionals did not indicate a specialty.

Thirty participants completed all 3 modules, 4 participants completed 2 modules, and 13 participants completed 1 module. Forty-five participants completed the module on offlabel prescribing, 33 participants completed the module on accessing and appraising unbiased drug information, and 33 participants completed the module on talking with patients about direct-to-consumer advertising. Fourteen participants used the “ask the faculty” e-mail option.

Forty-five participants indicated that the modules met their educational objectives. We also found some changes in clinician perception of off-label drug promotion. Initially, 10 clinicians reported positive viewpoints and openness to specific off-label use after exposure to off-label drug promotion, 9 clinicians were skeptical, and 4 clinicians were reluctant or refused to prescribe off label. After completing the module on off-label prescribing, 20 clinicians believed that, had they seen the module before off-label promotion of a specific medication, they likely would have had a different reaction to the promotion. Sixteen clinicians (10 of whom were skeptical of or refused the promotion initially) indicated that seeing the module first would not have changed their perception of off-label promotion.

Table 1

summarizes the assessment of change in knowledge among participants after completion of each module, the number of participants who made an intent-to-change practice statement after each module, and the total number of intent-to-change practice statements made at the end of each module. It also summarizes the number of clinicians making (by self-report) at least 1 practice change and the number of practice changes made on 3-month to 6-month follow-up. Representative self-reported changes in practice included the following statements: “I have more aggressively pursued proposed off-label medication indications and any underlying clinical studies to support those indications.” “I have also made the effort to hold my ground more frequently in the face of patient pressure. This has been difficult but ultimately, I think, rewarding.”

Table 2

Nineteen of 47 participants who completed at least 1 module responded to the follow-up survey. Six participants reported no barriers to making intended practice changes. A total of 22 barriers in 6 general categories (determined based on review and consensus categorization of the qualitative barrier comments among the authors) were identified ().

Forty-three of 47 participants who completed at least 1 module consented to an evaluation of their prescribing of target classes of medications. In the 12 months before engaging in the curriculum, group model HMO participants prescribed a mean of 19.0 target medications per month. In the 6 months following the curriculum, these participants prescribed a mean of 19.8 target medications per month (P = .60). Among curriculum nonparticipants, a mean of 17.5 target medications per month were prescribed in the 12-month precurriculum period, and a mean of 23.8 target medications per month were prescribed in the 6-month postcurriculum period (P <.001). Therefore, although curriculum participants did not decrease target medication prescribing, their target medication prescribing did not increase, while nonparticipant prescribing of written by the curriculum participants versus the nonparticipants before and after the curriculum was offered showed no differences in prescribing changes between the 2 groups (prescribing of targeted medications rose by 0.8 prescription per month in the intervention group and by 6.3 prescriptions per month in the control group; P = .17).

Discussion

The results of this pilot study indicate that prescribing clinicians who participated subjectively improved their knowledge base after completing the curriculum. Some of these clinicians indicated their intent-to-change practice and identified barriers to practice change, and several clinicians reported changes in practice 3 to 6 months after completion of the curriculum. Prescribing of targeted medications did not change among participants after completion of the curriculum; however, prescribing of the same medications significantly increased among nonparticipants during the same time frame.

Despite broadly publicizing this curriculum via e-mail and postal mail, few KPCO HMO and PPO clinicians participated in this curriculum, limiting the generalizability of our findings. This is likely because of multiple factors such as competing demands for clinician time at work and interest in and availability of several other CME opportunities. Furthermore, topics in this curricula are broadly applicable across multiple conditions; previous surveys have indicated that Kaiser Permanente physicians tend to prefer disease-specific CME topics focusing on guidelines and evidence, while expressing less interest in programs focused on cross cutting topics such as clinician—patient communication and culturally competent care.⁸ The lack of participation also may reflect a sense among clinicians that they are immune to the effects of advertising and can discern marketing efforts from dissemination of balanced scientific content.^9,10 Recent work by Eva and Regehr¹¹ and by Davis et al¹² shows that clinicians, especially those who have the most room for improvement, may not accurately selfassess and may not be motivated to engage in activities such as the educational curriculum we offered. Group model HMO clinicians particularly may not perceive the need to participate in this type of activity given other organizational efforts toward promoting cost-effective prescribing. Mandatory participation in curricula such as the one we developed may not lead to changes in practice, as clinicians may approach the activity with a “just get through it” mind-set rather than with openness to examining and changing practice.

The 19 participants who responded to the follow-up survey noted several barriers to practice change. Despite the low response rate, the barriers identified are not unexpected given this topic; in addition, the barriers are consistent with those noted in other CME programs.¹³ Therefore, the barriers that participants noted in our study are likely generalizable to other settings and to similar curricula.

Our analysis showed that nonparticipant prescribing of targeted medications increased, while participant prescribing remained fairly constant. Our direct comparison of participants with nonparticipants did not show a statistically significant difference between groups; however, our study may have been underpowered to detect a difference because of few participants. In addition, participation in this curriculum was voluntary, with potential selection bias toward participants who were more predisposed to modify their prescribing. If this is the case, one possible explanation is that our curricula helped participants buck the trend of increased prescribing of heavily advertised medications. Multiple other explanations (including no effect) are possible. Therefore, we can only speculate about the effects that curricula such as ours might have on prescribing of heavily marketed medications. Pending additional studies, the effect on prescribing trends among a broader selection of prescribers remains unknown.

Despite these limitations, we noted several positive findings that other investigators can apply when developing interventions aimed to improve cost-effective prescribing. The multifaceted needs assessment method we used¹ was useful targeted medications significantly increased in the same period. Results from the analysis of the number of prescriptions for designing a linician-relevant curriculum on the pharmaceutical development, approval, and marketing process. Participants readily answered the open-ended questions in the modules; 30% of participants asked e-mail questions of faculty. This provided a low-cost, easily implemented method of promoting self-reflection and providing faculty interaction during online CME; interactivity and reflection have been shown to be more effective in leading to practice change than strictly didactic CME formats.^14-16 Many participants made intent-to-change practice statements after completing the CME modules; intent-to-change practice statements have been shown to correlate with actual change in practice.^17,18

What might be done to increase participation in curricula such as this not only in future studies but also in real-world practice? We suggest that prework, especially with prescribers who have increased rates of prescribing heavily marketed high-cost medications, should be performed (perhaps on an individual level) to encourage participants to engage in activities of this sort in a mode in which they are ready to learn. For example, brief individual academic detailing discussions¹⁹ conducted by key opinion leaders²⁰ could be used to highlight the differences between an individual’s prescribing habits and those of respected peers who prescribe in a more cost-effective manner. As part of these discussions, individuals could then be encouraged in a supportive manner to engage in curricula to help improve their prescribing. Embedding concepts of offlabel prescribing, accessing and appraising unbiased drug information, and talking with patients about direct-to-consumer advertising within disease-specific CME programs could help engender physician interest for subsequent broader-based curricula such as ours. Alternatively, repeatedly including and reinforcing these concepts in serial disease-specific CME programs could lead to generalized learning. We have had some success using these approaches with other cross-cutting topics; whether these approaches would influence prescribing over the long term and among a larger audience should be examined.

The findings of our study suggest that curricula targeted on aspects of the pharmaceutical development, approval, and marketing process and positioned as part of a multifaceted effort might help mitigate the influence of pharmaceutical industry advertising on clinician prescribing behavior. Future studies should include larger samples, target clinicians based on prescribing patterns, engage clinicians in discussions using the clinician’s data to encourage participation, and tailor curricula to specific areas of prescribing to create a sense of need and relevance among participants. Consistent with studies^14-16,21 on the effectiveness of CME, further educational endeavors should use multiple formats to present the information and provide reinforcement after the curriculum using audit and feedback, reminders, and opportunities for further questions, perhaps by individual academic detailing. Future endeavors might include the provision of incentives for participating in the curriculum and for completing postcurriculum intent-to-change practice statements. We also suggest that future efforts attempt to correlate intent-to-change practice statements with objective changes in practice. Finally, future studies should include prescribing information to facilitate direct comparison of prescribing patterns among curriculum participants versus nonparticipants.

Author Affiliations: From Kaiser Permanente Institute for Health Research (DWP, MAR, DAC, NMC), Denver, CO; and Kaiser Permanente Colorado (LAW), Denver.

Funding Source: This study was supported by a grant from the Attorney General Consumer and Prescriber Education Grant Program, which is funded by the multistate settlement of consumer fraud claims regarding the marketing of the prescription drug Neurontin. The grantors had no involvement in the design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.

Author Disclosures: The authors (DWP, MAR, DAC, LAW, NMC) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (DWP, MAR); acquisition of data (MAR, DAC, LAW, NMC); analysis and interpretation of data (DWP, MAR, DAC, LAW, NMC); drafting of the manuscript (DWP, MAR, LAW, NMC); critical revision of the manuscript for important intellectual content (DWP, MAR, DAC, NMC); statistical analysis (DAC, LAW, NMC); obtaining funding (DWP, MAR); and supervision (DWP).

Address correspondence to: David W. Price, MD, Kaiser Permanente Institute for Health Research, 10065 E Harvard Ave, Ste 300, Denver, CO 80231. E-mail: david.price@kp.org.

1. Price D, Raebel M, Conner D, Wright L. Assessing prescriber and organizational leader educational needs about heavily marketed drugs.

Permanente J. 2008;12(2):28-35.

2. National Center for Health Statistics. Health, United States, 2007, With Chartbook on Trends in the Health of Americans. Hyattsville, MD: US Dept of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics; 2007. http://www.cdc.gov/nchs/data/hus/hus07.pdf. Accessed August 31, 2009.

3. National Institute for Health Care Management Research and Educational Foundation. Prescription drugs and mass media advertising, 2000. Washington, DC: NIHCM Foundation; 2001. http://www.nihcm.org/~nihcmor/pdf/DTCbrief2001.pdf. Accessed October 17, 2008.

4. Lyles A. Direct marketing of pharmaceuticals to consumers. Annu Rev Public Health. 2002;23:73-91.

5. Wazana A. Physicians and the pharmaceutical industry: is a gift ever just a gift? JAMA. 2000;283(3):373-380.

6. Krautkramer CJ. Neurontin and off-label marketing. Virtual Mentor. 2006;8(6):397-402.

7. Steinman MA, Bero LA, Chren MM, Landefeld CS. Narrative review: the promotion of gabapentin: an analysis of internal industry documents. Ann Intern Med. 2006;145(4):284-293.

8. Price DW, Overton CC, Duncan JP, et al. Results of the first national Kaiser Permanente continuing medical education needs assessment survey. Permanente J. 2002;6(1):76-84.

9. Bihlmeyer S, Schreiber CP, Farrand S. The impact of commercial support on medical education: the physician’s perspective. CE Meas. 2008;2(3):43-49.

10. American Association of Medical Colleges. The Scientific Basis of Influence and Reciprocity: A Symposium. Washington, DC: American Association of Medical Colleges; January 2008. http://www.aamc.org/publications. Accessed August 31, 2009.

11. Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(10)(suppl):S46-S54.

12. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094-1102.

13. Price DW, Felix KG. Journal clubs and case conferences: from academic tradition to communities of practice. J Contin Educ Health Prof. 2008;28(3):123-130.

14. Thomson O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes [update in Cochrane Database Syst Rev. 2009;(2):CD003030]. Cochrane Database Syst Rev. 2001;(2):CD003030.

15. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867-874.

16. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288(9):1057-1060.

17. Dolcourt JL. Commitment to change: a strategy for promoting educational effectiveness. J Contin Educ Health Prof. 2000;20(3):156-163.

18. Wakefield J, Herbert CP, Maclure M, et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof. 2003;23(2):81-93.

19. O’Brien MA, Rogers S, Jamtvedt G, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;17(4):CD000409.

20. Collins BA, Hawks JW, Davis R. From theory to practice: identifying authentic opinion leaders to improve care. Manag Care. 2000;9(7):56-58, 61-62.

21. NHS Centre for Reviews and Dissemination. Getting evidence into practice. Effective Health Care. 1999;5(1):1-16. http://www.york.ac.uk/inst/crd/EHC/ehc51.pdf. Accessed December 2, 2008.

Download PDF: Pharmaceuticals From Development to Practice: Learnings From an Online Curriculum