"Meaningful" Clinical Quality Measures for Primary Care Physicians

ABSTRACT

Objectives: To systematically solicit recommendations from Meaningful Use (MU) exemplars to inform Stage 3 MU clinical quality measure (CQM) requirements.

Study Design: The study combined an electronic health record (EHR)-based CQM performance assessment with focus groups among primary care practices with high performance (top tertile), or “exemplars.”

Methods: This qualitative exploratory study was conducted in PPRNet, a national primary care practice—based research network. Focus groups among lead physicians from practices in the top tertile of performance on a CQM summary measure were held in early 2014 to learn their perspectives on questions posed by the Office of the National Coordinator related to Stage 3 MU CQMs.

Results: Twenty-three physicians attended the focus groups. There was consensus that CQMs should be evidence-based and focus on high-priority conditions relevant to primary care providers. Participants thought the emphasis of CQMs should largely be on outcomes and that reporting of CQMs should limit the burden on providers. Incorporating patient-generated data and accepting locally developed CQMs were viewed favorably. Participants unanimously concurred that platforms for population management were vital tools for improving health outcomes.

Conclusions: Using a series of focus groups, we solicited Stage 3 MU CQM recommendations from a group of physicians who have already achieved “meaningful use” of their EHR, as demonstrated by their high performance on current MU CQMs. Adhering to the standards deemed to be important to high-performing real-world physicians could ensure that the MU Incentive Programs achieve their ultimate goal to improve outcomes.

Take-Away Points

Through a series of focus groups, we solicited recommendations from high-performing Meaningful Use (MU) “exemplars” on clinical quality measures (CQMs).

Am J Manag Care. 2015;21(10):e583-e590

• The evolution of MU CQMs is not consistent with the exemplars’ recommendations.
• “Exemplars” recommend reverting to at least some core CQMs and ensuring that CQMs focus on high-priority conditions, are evidence and guideline-based, and do not require additional documentation.
• Population management should be incorporated into the MU objectives.

Reporting of clinical quality measures (CQMs) has been a required component of both Stage 1 and Stage 2 of the CMS Electronic Health Record (EHR) “Meaningful Use” (MU) Incentive Programs. These CQMs are intended to promote the capabilities of EHRs to calculate measures that inform providers, and eventually the public, about their clinical performance.^1,2 Several groups, including the American College of Physicians,³ argue that CQMs are the most important part of MU; others have stated that a focus on health outcomes is the critical requirement for reengineering our healthcare system.⁴ CMS notes that CQMs are tools that help us to “measure and track the quality of healthcare services.”¹ Accordingly, MU CQMs are now aligned with other CMS programs, including the Physician Quality Reporting System and the Value-Based Payment Modifier.^5,6

Furthermore, the evolution of MU CQMs supports this notion. For Stage 1 MU, eligible professionals (EPs) were required to report on 3 core CQMs: blood pressure, tobacco status, and adult weight screening and follow-up. EPs also chose to report on 3 additional CQMs from a menu of 38 measures. In 2014, for both Stage 1 and Stage 2 MU, reporting on 9 of 64 CQMs was required. These CQMs are organized by the 6 National Quality Strategy (NQS) domains representing HHS priorities for healthcare quality improvement. There are recommended core sets for both adults and children based on high-priority conditions and a requirement that the selected measures cover at least 3 of the 6 NQS domains.⁷ The Table lists the 6 NQS domains along with CQMs relevant to primary care within these domains.

CQMs for Stage 3 MU, which has been delayed and is now scheduled to begin in 2017, are now under consideration.^8,9 In late 2012, the Office of the National Coordinator for Health Information Technology (ONC) issued a Request for Comments (RFC) for Stage 3 MU which included questions related to CQMs, focused on ensuring that the CQM set improves the “quality of care and experience of care for providers and patients” consistent with the ultimate goal for MU.¹⁰ In February 2013, the Agency for Healthcare Research and Quality (AHRQ), working in partnership with the ONC and CMS, solicited rapid cycle research projects to provide evidence to inform the development of Stage 3 MU objectives.¹¹ The purpose of this study, in response to this request, was to systematically solicit recommendations from high-performing primary care MU exemplars to help inform the Stage 3 MU CQM requirements for EPs. The final rule for Stage 3 MU was released by CMS in October 2015.^8,9,12

METHODS

Setting

This exploratory qualitative study was conducted in Primary (Care) Practices Research Network (PPRNet), a national EHR-based primary care practice-based research network, and an AHRQ Center for Primary Care Practice-Based Research and Learning.¹³ At the time of the study, all PPRNet practices used the same EHR (McKesson Practice Partner, San Francisco, California). PPRNet maintains a longitudinal clinical database, derived from regular EHR extracts from participating practices, which is used for quality reporting and research. Reports provide feedback on the practice, provider, and patient level for more than 60 quality measures encompassing primary and secondary prevention, disease management, and safe medication prescribing and monitoring, as well as summary measures.¹⁴ Twenty-one of the PPRNet measures are comparable to 2014 MU CQMs.⁷

All PPRNet practices whose providers had attested for Stage 1 MU were eligible for the study. Practices were recruited through a series of e-mail messages in the fall of 2013. Enrolled practices that submitted an October 1, 2013, PPRNet data extract were included in the study.

EHR-Based CQM Performance Assessment

The PPRNet approach to CQM performance assessment has been described in detail elsewhere.¹⁵ For this study, we assessed performance for each practice as of October 1, 2013, on the Summary Quality Index (SQUID),¹⁴ the quotient of the number of eligible measures the patient met, and the total number of MU CQMs for which the patient was eligible. Seventy-one PPRNet practices were eligible for performance assessment; the practices with the 27 highest SQUID-CQMs (approximately the top tertile) were deemed exemplar practices.

Focus Groups Among Exemplar Performers

Lead physicians from each of the 27 designated exemplar practices were invited to participate in 1 of 3 focus groups held on consecutive Saturdays in geographically separate cities in late January and early February 2014. Participants provided verbal consent to participate and have the discussions recorded for analysis. Two of the study authors (CL and SO) moderated the focus groups using a detailed slide presentation; each focus group lasted 2.5 hours. Although each of the 3 focus groups was conducted similarly, minor clarifications in the questions were made based on experiences from the first group. During a 15-minute introduction, ground rules for the focus groups were established and an overview of prior and current MU CQM requirements was provided. The introductory presentation refreshed the group about the overall intent of MU CQMs to improve the quality of care.

Questions from the ONC’s RFC for Stage 3 MU CQM were used to guide the discussion. Participants were asked to identify which CQMs (Table) should be a high priority, discuss how to reduce the burden of CQM reporting on providers, and compare the value of process versus outcome measures. The significance of incorporating patient-generated data into CQMs, the appeal of aligning MU CQMs with MU functional objectives, the feasibility and desirability of locally developed measures, and the importance of population management platforms were also explored with participants. All focus group members were encouraged to respond to each question and participate in the general discussion. Two other research team members (LN and AW) observed the groups and took detailed notes in a structured observational template to supplement the audio recordings. The observational template was developed to document the responses, agreements, and contrary views among the participants of each focus group, adapted from a micro-interlocutor approach.¹⁶

A professional transcription service was used to transcribe digital records of all focus groups. Transcripts and focus group observation notes were imported into NVivo International, Pty (QSR International, Doncaster, Australia) by a qualitative researcher (LN) for thematic analysis based on the ONC RFC questions. Two of the study authors (CL and LN) independently reviewed transcripts and notes from all focus groups to identify themes related to MU CQMs. The transcripts were then used to locate specific comments and context for clarification. The 2 qualitative analysts reconciled discrepancies through a process of immersion and crystallization which were, for the most part, consistently interpreted.¹⁷

RESULTS

Lead physicians from 23 of the 27 practices selected as exemplar practices attended the focus groups. Five of these physicians were female, 2 were Hispanic, and all were white; the median age of these physicians was 56 years. Five physicians were general internists, 2 were internal medicine/pediatrics physicians, and the remainder were family medicine physicians. The physicians came from practices in 18 US states, and all practices reported using their EHRs for over 6 years: 9 practices had been using their EHRs for 6 to 10 years, 7 for 11 to 15 years, 5 for 16 to 20 years, and 2 for over 20 years. Analyses from focus group discussions have been organized into 8 themes, discussed in more detail below.

1. CQMs should be evidence-based, focused on high-priority conditions, and relevant for primary care physicians. Participants were asked to provide feedback on measures from each NQS domain. For all domains, there was general consensus in each focus group that CQMs should be evidence-based, noncontroversial, and based on national guidelines when available. They also believed that the measures should be “the ambulatory-sensitive ones—the ones that we can control ought to be measured in every office.” Participants emphasized the need for CQMs to be flexible and rapidly evolve to reflect changes in evidence or new guidelines. For example, many participants were concerned that current CQMs regarding dyslipidemia are not concordant with the recently released American College of Cardiology/American Heart Association (ACC/AHA) Guideline on the Treatment of Blood Cholesterol,¹⁸ and argued that these CQMs should be quickly updated to reflect the new guidelines. One participant stated, “Keep it simple. Whatever you do, use things that have been vetted as indicators or results or processes that are valuable and proven they make a difference, and keep the flexibility.”

Participants generally agreed that CQMs in the Clinical Process/Effectiveness domain should reflect highly prevalent conditions with long-term consequences and for which improved performance on the CQM could have considerable impact on morbidity and mortality, such as hypertension, hyperlipidemia, and diabetes. All participants believed CQMs in the Population/Public Health domain should be a high priority—particularly those with broad public health implications. At one focus group, there was near unanimous consensus that CQMs should be limited to measures only in this domain, while participants in the 2 other focus groups agreed that, although important, CQMs in other domains also reflected high-priority chronic conditions and should be included.

Many argued that adherence to all US Preventive Services Task Force (USPSTF) Grade A and B recommendations and recommendations from the Advisory Committee of Immunization Practices be included as CQMs, while others favored selecting specific USPSTF recommendations with a considerable impact on mortality. Participants universally agreed that CQMs related to hypertension, obesity, and smoking cessation were of utmost importance. One participant stated, “I would put my energy into blood pressure, blood pressure, blood pressure. Smoking, smoking, smoking. Exercise, exercise, exercise.”

Compared to the Clinical Process/Effectiveness domain and the Population/Public Health domain CQMs, there was much less enthusiasm for 2014 CQMs included in the Patient Safety, Efficient Use of Healthcare Resources, Care Coordination, or Patient/Family Engagement domains. In general, these CQMs were not felt to have as much impact on the health of the US population as the initial 2 domains. The evidence supporting several measures included in the Patient Safety domain was questioned. For example, participants noted that there is not clear data to support the CQM, "Screening for falls in the elderly." One participant stated, “That’s so frustrating that you’re going to measure me on something that’s 1, difficult to document and 2, we already have a study that shows it doesn’t do any good.” They were also apprehensive about the measure, "Documentation of all current medications in the EHR." First, some participants argued that this “attestation” measure required “checking boxes” on the part of provider. Second, the utility of this measure in improving care was questioned. One participant argued that listing all over-the-counter medications and herbal supplements in the EHR could have an unintended consequence and result in the provider missing an important prescribed medication due to the complexity of the list.

Participants expressed similar unease over CQMs related to the Efficient Use of Healthcare Resources domain. For example, participants largely agreed with avoiding imaging for low back pain, yet they were concerned about the ability to measure this accurately using EHR data along with the “extra work” of having to document exceptions. Participants argued that while measuring efficient use of resources was important, other types of measures would be more relevant for primary care, such as the proportion of their patients with emergency department (ED) visits or hospitalizations. Participants acknowledged that this type of measure could be difficult to assess using only ambulatory EHR data.

There was also concern that the only Care Coordination domain CQM, "Receipt of specialist report," was a measure of coordination outside of their control and reflected EHR functionality rather than provider performance. Although participants concurred that closing the loop with specialists is important for care management, there was debate about whose responsibility this was and that this measure could unfairly place additional burden on the primary care provider rather than the specialist who was receiving the revenue from providing the service. Also, as one participant stated, “My feeling is that I would rather not try to track every referral, but track what are, for me, the high-risk referrals. I don’t really need to get the report back on every time I send the patient to the dietician or the physical therapist.” Participants again questioned the evidence for whether adhering to this CQM improves quality.

Participants also doubted the clinical utility of CQMs included in the Patient Engagement domain. One participant remarked that “it was not clear how administering functional status assessments to patients makes them more engaged in their care” and, again, did not feel these measures were clearly supported by evidence. There was concern that current EHRs could not yet support this patient-generated data. Several participants suggested other potential patient engagement CQMs that were more useful for primary care physicians, including discussion of end-of-life issues, asthma control, or symptoms of depression.

2. A few core CQMs focused on public health issues should apply to all eligible providers. While participants in one focus group unanimously agreed that there should be a core group of CQMs for all providers, regardless of specialty, that focuses on important public health issues—such as measures related to tobacco abuse and obesity—participants at the other 2 focus groups maintained that while a few CQMs could be applicable to all providers, there should be flexibility with CQMs tailored for both primary care physicians and specialists. According to 1 participant, “Why can’t you have a core like smoking cessation that applies to everybody? Why can’t your dermatologist say stop smoking?”

3. The focus of CQMs should largely be on outcomes. Participants debated whether CQMs should reflect process or outcome measures, although most favored outcome measures in the context of chronic disease management. One participant favoring process measures pointed out that socioeconomic factors were more likely to impact performance on outcome measures. Another participant favoring outcome measures argued, “I think we’ve kind of shown that we can get the numbers on the process measures,” but acknowledged that it was much harder to achieve high performance on outcome measures. There was consensus that CQMs should measure outcomes that are consistent over time; for example, A1C measuring blood glucose control over 3 months was felt to be a more important outcome than a single blood pressure value taken at a single point in time.

4. Reporting of CQMs should limit burden on providers. Although there were mixed opinions on the number of measures that a provider should be expected to calculate, participants in all focus groups agreed on a few important considerations. Participants consistently advocated that the focus of the CQM measure set should be on achieving improved outcomes without requiring additional work by providers. Being forced to “check boxes,” as required by the 2014 core CQM, Documentation of all current medications in the EHR was a common frustration expressed at all focus groups. Participants repeatedly stated that accurate calculation of measures should not require additional steps outside the routine work flow. One participant cited the “tremendous provider burden” currently placed on the provider for current measure calculations. Another participant stated, “I should not require a full-time information technology (IT) person to meet these goals.”

There was general agreement that the measures should be meaningful to providers and beneficial for patients; a few participants believed that the number of measures should differ by specialty. The majority of participants agreed, however, that if CQMs were both meaningful and calculated automatically without additional burden, increasing the number of measures would not be problematic. According to one participant, “If the burden was eliminated where it was automated, I think we wouldn’t have a lot of pushback.” Participants all concurred that functional MU objectives, such as those related to clinical decision support, should be aligned with CQMs.

Participants agreed that improved EHR capabilities, including integrated immunization registries and synchronization between billing and documentation services, could reduce the burden on providers to duplicate documentation for measure requirements. As one participant stated, “There should not be a measure for which you have to manually go get other data just so that you can make your report.” Another participant agreed, “Why do we have to click a box to say that we did medication reconciliation? When we opened the medication list or we made a couple of changes in medications, shouldn’t that be automatically recorded?”

5. Consider performance thresholds for some CQMs. The concept of requiring providers to reach performance thresholds on some CQMs was also discussed. Participants agreed that if thresholds for quality measures were required in future MU requirements, then the number of measures should definitely be limited. One participant suggested that requiring only a select group of 3 or 4 core measures, each with thresholds to reach, could be “more valuable than having lots of measures with no threshold.” Another participant was opposed to a threshold requirement and instead believed those goals should be achieved through pay-for-performance programs rather than the MU programs.

6. EHRs should have capabilities to capture patient-generated data, which could be incorporated into CQMs. Participants gave many examples of current uses of patient-generated data, including home blood pressure measurements, blood glucose logs, and food diaries, and generally agreed that these types of data would be easier to use for clinical care purposes if they could be directly imported into the EHR through a portal. Examples of possible EHR-enabled CQMs using patient-generated data suggested by clinicians were measures related to dementia or cognitive impairment, exercise, tobacco use, or end-of-life care. Many participants agreed that there should be a review queue to ensure clinician recognition of patient-reported data, along with a way to differentiate patient-generated data from office-generated data in the EHR. One participant stated, “If it’s patient-generated data, it’s going to have to come in some kind of format so it doesn’t create an error.” Providers also emphasized that there should be evidence that collecting this type of data improves outcomes before CQMs are developed.

7. Locally developed CQMs can encourage innovation. The majority of participants believed that allowing locally developed CQMs would foster innovation and promote the development of new measures that could be relevant and useful for primary care physicians. However, participants conceded that each EP should only be permitted to submit a single, locally developed CQM, and that there should be some certification process, including a submission form describing the evidence that supports the measure. There were mixed opinions about whether development of local CQMs should be limited to larger organizations such as accountable care organizations or professional societies, with some concern that larger systems or professional societies might not share the same interests as individual providers. According to one provider, “Meaningful use ought to be using the EHR in a way that allows the providers to really figure out what they are doing with their patients and get the feedback on it. It really isn’t sending something off to Congress so that 2 years from now you don’t get a penalty.”

8. Population management tools are vital to improving quality. There was unanimous agreement on the importance of health IT tools for population management. All exemplars participating in these focus groups receive regular performance reports on CQMs, including patient-level registries as part of their routine participation in PPRNet, and many stated that the registries were one of the most useful tools they had for improving the health of the patients in their practices.

Participants felt that tools that assisted with case management and calculated risk scores (ie, FRAX, Framingham, ACC/AHA 10-year atherosclerotic cardiovascular disease estimate) should be present in a basic CQM population management platform. Others suggested adding utilization information, including ED visits and hospitalizations. Participants agreed that there was a business case for their use of these tools, and several described using registries to reach out to patients due for specific services. Others pointed out that the business case for use of these tools is enhanced in a pay-for-performance environment.

DISCUSSION

To our knowledge, this is the first systematic exploration of perspectives on MU from a group of physicians who have already achieved meaningful use of their EHR “to achieve significant improvements in care”² as demonstrated by their high performance on many current MU CQMs. All of the participating physicians came from practices who had adopted EHRs before the start of the MU programs; the majority of our participants represent early adopters who have been using EHRs for over 10 years. These exemplars provided many recommendations for how MU CQMs could be improved to achieve improved outcomes.

Participants advocated that the measure set be limited to CQMs that are evidence-based, flexible, and focused on high-priority conditions likely to have a large beneficial impact on the health of the US population. They proposed requiring a few core CQMs that focus on public health, such as tobacco cessation and obesity counseling, for all eligible professionals, regardless of specialty. They also suggested prioritizing outcome measures for disease-management domains and incorporating performance thresholds for some measures rather than adding measures. MU measures that would promote the use of population management tools, such as registries, to facilitate outreach to patients were encouraged.

Participants also believed allowing the submission of locally developed CQMs could accelerate the development and testing of new CQMs that could help to improve the health of their patients. Although the use of patient-generated data was viewed favorably, participants cautioned that there needed to be ways to assess the accuracy of this data, along with evidence of its impact, before inclusion as MU CQMs. They did advise against the inclusion of CQMs that create an additional documentation burden for physicians.

This advice is consistent with other recent studies reporting increasing physician dissatisfaction with EHRs and concern that the use of EHRs does not result in improved quality of care. Concern has additionally been expressed that the focus of clinical documentation processes, including those required by MU, is inappropriately being shifted to payment and regulatory requirements rather than care delivery.¹⁹ In one recent national survey, nearly half of all physicians stated that patient care was worse since implementing an EHR and nearly two-thirds would not purchase their current EHR system.²⁰

Limitations

This study has several limitations. Focus group participants were all volunteer members of PPRNet, a primary care learning and research organization with a mission to improve healthcare. Similarly, we selected focus group participants who were members of practices with high performance on current MU CQMs. Therefore, the conclusions may not be generalizable to all physicians. In addition, all participants used the same EHR, which could have influenced their perspectives on questions about work flow and added burdens. Finally, our questions related to MU CQMs were limited to topics included in the ONC’s RFC for Stage 3 MU.

To date, the progression of MU CQM requirements from 2011 to 2014 largely does not reflect the recommendations from these exemplars. Participants advocated that CQMs be evidence-based and reflect high-priority conditions; however, many of the CQMs added to the 2014 measure set do not meet these criteria. For example, 2014 CQMs for adults include several new measures that our participants did not feel were evidence-based, such as "Functional status assessment for complex chronic conditions" and "Screening for future fall risk." In fact, while participants advocated for adhering to USPSTF recommendations, the USPSTF has concluded that the likelihood of benefit for routinely performing an in-depth fall risk assessment in adults 65 years or older is small and therefore not recommended.²¹ Other new CQMs, such as "Documentation of current medications in EHR require attestation," which was viewed by participants as creating additional burden. In addition, although participants recommended required core CQMs, during the transition from initial CQMs introduced in 2011 to the current 2014 set, required core CQMs were eliminated.

CONCLUSIONS

In this qualitative exploratory study, we obtained feedback on MU CQMs from primary care exemplar physicians who have demonstrated their ability to achieve high performance using EHRs. CQMs most likely to effectively improve quality in primary care should be limited to those that are evidence-based, focused on high-priority conditions, do not require additional documentation, and facilitate population management. Adhering to these standards deemed to be important to real-world primary care physicians could ensure that the MU Incentive Programs achieve their ultimate goal to “achieve significant improvements in care.”Author Affiliations: Department of Family Medicine (SMO, AMW), and Department of Medicine, Division of General Internal Medicine and Geriatrics (CBL), and College of Nursing (LSN), Medical University of South Carolina, Charleston, SC.

Source of Funding: This study was funded by the Agency for Healthcare Research and Quality, Grant No. 1R18HS022701. Dr Litvin was supported by the Agency for Healthcare Research and Quality Grant No. K08HS018984.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (CBL, LSN, SMO, AMW); acquisition of data (CBL, LSN, SMO, AMW); analysis and interpretation of data (CBL, LSN, SMO, AMW); drafting of the manuscript (CBL, LSN, SMO, AMW); critical revision of the manuscript for important intellectual content (CBL, LSN, SMO, AMW).

REFERENCES

Address correspondence to: Cara B. Litvin MD, MS, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, 135 Rutledge Ave, Charleston, SC 29425. E-mail: litvincb@musc.edu.

1. Clinical quality measures basics. CMS website. http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/ClinicalQualityMeasures.html. Updated September 4, 2015. Accessed November 4, 2015.

2. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010;363(6):501-504.

3. Zaroukian MH. American College of Physicians comment letter on stage 3 meaningful use. ACP website. https://www.acponline.org/acp_policy/letters/health_it_stage_three_meaningful_use_2013.pdf. Published January 14, 2013. Accessed November 4, 2015.

4. Hoffman A, Emanuel EJ. Reengineering US health care. JAMA. 2013;309(7):661-662.

5. Physician Quality Reporting System. CMS website. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/PQRS/index.html. Updated June 12, 2015. Accessed November 4, 2015.

6. Value-Based Payment Modifier. CMS website. http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/PhysicianFeedbackProgram/ValueBasedPaymentModifier.html. Updated September 8, 2015. Accessed November 4, 2015.

7. 2014 clinical quality measures. CMS website. http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/2014_ClinicalQualityMeasures.html. Updated September 24, 2015. Accessed November 4, 2015.

8. Tagalicod R, Reider J. Progress on adoption of electronic health records. CMS website. http://www.cms.gov/eHealth/ListServ_Stage3Implementation.html. Updated December 13, 2013. Accessed April 16, 2014.

9. Health IT Policy Committee: recommendations to the National Coordinator for Health IT. HealthIT.gov website. http://www.healthit.gov/facas/health-it-policy-committee/health-it-policy-committee-recommendations-national-coordinator-health-it. Updated September 18, 2015. Accessed November 4, 2015.

10. HIT Policy Committee: request for comment regarding the stage 3 definition of Meaningful Use of electronic health records (EHRs). HealthIT.gov website. http://www.healthit.gov/sites/default/files/hitpc_stage3_rfc_final.pdf. Accessed April 1, 2014.

11. Evaluation of Meaningful Use. Agency for Healthcare Research and Quality website. http://healthit.ahrq.gov/ahrq-funded-projects/evaluation-of-meaningful-use. Updated October 2015. Accessed November 4, 2015.

12. Joszt L. Proposed rule for stage 3 Meaningful Use submitted. AJMC website. http://www.ajmc.com/newsroom/Proposed-Rule-for-Stage-3-Meaningful-Use-Submitted. Published January 7, 2015. Accessed January 19, 2015.

13. AHRQ Centers for Primary Care Practice-Based Research and Learning. Agency for Healthcare Research and Quality website. http://www.ahrq.gov/professionals/systems/primary-care/rescenters/index.html. Published September 2012. Accessed November 4, 2015.

14. Nietert PJ, Wessell AM, Jenkins RG, Feifer C, Nemeth LS, Ornstein SM. Using a summary measure for multiple quality indicators in primary care: the Summary QUality InDex (SQUID). Implement Sci. 2007;2:11.

15. Ornstein S, Nietert PJ, Jenkins RG, Wessell AM, Nemeth LS, Rose HL. Improving the translation of research into primary care practice: results of a national quality improvement demonstration project. Jt Comm J Qual Patient Saf. 2008;34(7):379-390.

16. Onwuegbuzie AJ, Dickenson WB, Leech NL, Zoran AG. A qualitative framework for collecting and analyzing data in focus group research. Int J Qual Methods. 2009;8(3).

17. Borkan J. Immersion/crystallization. In: Crabtree BF, Miller WL, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications, Inc; 1999:177-194.

18. Stone NJ, Robinson J, Lichtenstein AH, et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25, pt B):2889-2934.

19. Cusack CM, Hripcsak G, Bloomrosen M, et al. The future state of clinical data capture and documentation: a report from AMIA’s 2011 policy meeting. J Am Med Inform Assoc. 2013;20(1):134-140.

20. Verdon DR. Physician outcry on EHR functionality, cost will shake the health information technology sector. Medical Economics website. http://medicaleconomics.modernmedicine.com/medical-economics/news/physician-outcry-ehr-functionality-cost-will-shake-health-information-technol?page=0,0. Published February 10, 2014. Accessed April 18, 2014.

21. Moyer VA; US Preventive Services Task Force. Prevention of falls in community-dwelling older adults: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2012;157(3):197-204. 

Download PDF: "Meaningful" Clinical Quality Measures for Primary Care Physicians