Information Retrieval Pathways for Health Information Exchange in Multiple Care Settings

ABSTRACTObjectives

To determine which health information exchange (HIE) technologies and information retrieval pathways healthcare professionals relied on to meet their information needs in the context of laboratory test results, radiological images and reports, and medication histories.

Study Design

Primary data was collected over a 2-month period across 3 emergency departments, 7 primary care practices, and 2 public health clinics in New York state.

Methods

Qualitative research methods were used to collect and analyze data from semi-structured interviews and participant observation.

Results

The study reveals that healthcare professionals used a complex combination of information retrieval pathways for HIE to obtain clinical information from external organizations. The choice for each approach was setting- and information-specific, but was also highly dynamic across users and their information needs.

Conclusions

Our findings about the complex nature of information sharing in healthcare provide insights for informatics professionals about the usage of information; indicate the need for managerial support within each organization; and suggest approaches to improve systems for organizations and agencies working to expand HIE adoption.

Am J Manag Care. 2014;20(11 Spec No. 17):SP494-SP501

We sought to understand the dimensions of information retrieval pathways that healthcare professionals used to engage in health information exchange (HIE) within the context of laboratory test results, radiological images and reports, and medication histories

• Our findings imply that HIE models need to support both push and pull methods to meet the very diverse needs of health professionals.
• We suggest that the more the data from HIE exchange services are integrated and work flows are harmonized with the electronic health record, the better.
• Our findings indicate that both clinical and nonclinical staff would benefit from organizational support meant to reduce the complex data gathering situation for clinicians and improve productivity.

Improving access to patient information at the point of care figures prominently into many countries’ goals¹ for health system reform and improvement. Clinicians require patient information from disparate sources to overcome patients’ fragmented patterns of care and poor clinical communication during transitions of care. Difficulty in accessing health records, or finding patient information to be completely inaccessible during the delivery of care, creates inefficiencies,^2,3 fosters duplication of ^4,5services fails to support optimal decision making,^4,6and threatens patient safety.^3,7

Health information exchange (HIE) addresses these shortcomings by providing clinicians who are not part of the same organizational entity with the capability to electronically retrieve and share health-related information across regional, community, or organizational boundaries.^8-10 Studies have shown that HIE can lead to numerous benefits such as reduced utilization of healthcare services,¹¹ reduced diagnostic testing,¹² prescription safety,¹³ public health support, and cost savings.^14-16 Healthcare organizations can engage in HIE using different technological architectures for point-to-point communication (ie, the direct exchange of data between electronic health record [EHR] systems from different vendors) between providers, and under various interorganizational collaborations such as services provided through a single intermediary health information organization.^16,17

HIE models vary across the United States, but often involve physicians, hospitals, and other healthcare stakeholders in the community joining a local HIE effort run by a regional health information organization (RHIO)¹⁸ or another model for exchange. These include enterprise HIE efforts (ie, an HIE system privately owned, funded, and developed by a single enterprise or organization)¹⁹ and vendor-mediated exchange.^20,21 In general, RHIOs have emerged as the leading model to facilitate the electronic exchange of information between healthcare organizations.²² A common feature for all HIE models is that information exchange is enabled through the electronic transmissionof data represented by 2 exchange methods—“push” and “pull”.^17,23 Push takes place when clinical data are electronically deposited into a recipient’s system after a sender initiates transfer. Pull is initiated when a user proactively uses a HIE system to retrieve aggregated patient health data stemming from multiple sources across a community.

Existing studies have evaluated the use of pull-based HIE systems,^24-27 and 1 quantitative study examined RHIO-managed HIEs with both push and pull capabilities,¹⁷ however, little qualitative research exists that examines the experience of healthcare professionals using multiple HIE models and push/pull methods at their place of work.

With the myriad options organizations have for HIE, our goal was to understand the dimensions of the information retrieval pathways used by healthcare professionals to engage in HIE. We sought to assess how healthcare professionals met their information needs in the context of laboratory test results, radiological images and reports, and medication histories. Access to these types of data has the greatest potential for impact on patient health, and holds the greatest promise for cost savings,²⁸ improved diagnosing and monitoring of health and diseases,²⁹ detection of diseases at an early stage,³⁰ and reduced frequency of repeat diagnostic testing.³¹

METHODS

Study Design and Setting

We undertook a qualitative multicase study approach in New York state. The state provides a relevant setting to evaluate HIE; it has invested more than $440 million in health information technology and HIE adoption over the last 7 years.³² We consulted with the New York eHealth Collaborative, a private-public nonprofit organization charged with facilitating statewide HIE by coordinating the creation of a network to connect healthcare providers, to identify 3 RHIOs serving distinctly diverse communities; this selection process would maximize variation and allow for comparison.⁵ The RHIOs that matched our criteria for selection agreed to participate in the study and cooperated in identifying care organizations for our study sites. The communities varied in terms of population size and consented patients, and in the number of users (clinicians and staff) with access to their local RHIO’s HIE (Table 1). Each RHIO implemented a different commercial HIE platform; the exchange architectures also differed. Community A and Community C used a federated model, and Community B used a centralized model. In a federated model, an organization locally stores and retains control over the patient information, and it responds when another organization, also a member of the same RHIO HIE effort, requests information. In a centralized model, patient data is stored in a central repository maintained by the RHIO after being collected from the organizations participating in the local RHIO effort.

We provided a brief project overview and obtained verbal and written informed consent from each care organization's clinicians and staff who agreed to participate in this study. The consent form was provided by the Institutional Review Board of Weill Cornell Medical College, which approved this study.

Data Collection

Data was collected during 2-day site visits at each participating care organization. To increase the internal validity of our study, we used 2 sources for evidence: observation and interviews.³⁴ All data collection began in May 2013 and ended in June 2013. RHIO staff helped identify and secure the cooperation of key informants who had experience utilizing various HIE models. We used a snowball sampling procedure in which we first interviewed a key informant; they were then asked to identify and help recruit other potential informants who were familiar with HIE and could provide additional insight about their information retrieval experiences.³⁵ Because these other potential informants came from the key informants' social networks, we believed this would reflect the perspectives of a diverse community of healthcare professionals.³⁶ We also observed participants and recorded the field notes of healthcare professionals engaging in HIE activities.³⁷ Data gathering at each site ended when the point of data saturation was reached—that is, when the interviews and observations did not produce any new information.³⁸ All interviews lasted an average of 30 minutes and were conducted using a semi-structured protocol. This allowed us to delve deeper into a topic concerning information retrieval approaches, to pose additional questions based on subject responses, and to thoroughly understand the answers that were given. All interviews were audio recorded and transcribed.

Data Analysis

From the interviews and observations, it was possible to obtain a narrative of HIE models and exchange methods used in each community. For this analysis, we undertook an iterative, open-coding approach to the interview and observational data.³⁹ To begin, we identified broad themes around the important concepts of information retrieval pathways for the types of information sought, based on our research objective. We completed the initial coding as close as possible to the date of data collection. During a second cycle of coding, we identified new concepts and relationships that were not adequately represented by the initial categories. This open coding was undertaken independently by authors PK and JRV, who were responsible for all phases of the coding. Collaboratively, PK and JRV reviewed and revised the coding schemes to consolidate redundant codes.

RESULTS We interviewed a total of 48 individuals across 3 emergency departments (EDs), 7 primary care practices (PCPs), and 2 local public health departments (LHDs) (Table 1). We observed healthcare practitioner activities for more than 40 hours across all 3 communities. The PCPs included a federally qualified health center, a community health center, solo practices, and large group practices. Each care organization was equipped with an EHR) and had access to their RHIO’s HIE system. The LHDs provided clinical services but did not have an EHR. LHDs were included in this study because HIE can provide them with access to previously hard-to-obtain clinical and demographic data for the purposes of disease surveillance, disaster response, and healthcare service delivery. We obtained the perspective of 3 broad types of professionals: clinicians (physicians, nurses, physician assistants, scribes, pharmacists, and medical assistants); case managers and public health disease investigation staff; and administrative staff (directors, information technology specialists, medical records personnel, quality managers, clerks, and front-desk staff). We used these larger groupings to protect confidentiality.

Information Retrieval Pathways

Based on the observations and interviews, we found that all 3 care settings (EDs, PCPs, and LHDs) relied on a combination of HIE models (RHIO-based and non— RHIO-related exchange services) and exchange methods (“push” and “pull”) to acquire clinical data from external organizations. These 2 dimensions of models and methods created 4 mutually exclusive categories of information retrieval pathways for HIE (Table 2). Based on these combinations of models and methods, we define these 4 categories as:

1. RHIO-pull: clinicians actively use a RHIO-managed Web-query portal to retrieve information from other healthcare organizations participating in the local RHIO effort. Authorized system users “pulled” patient information by logging into the portal and searching for such information. The query-based HIE systems were independent of the practices’ or hospitals’ EHRs and were available in all 3 communities.
2. RHIO-push: RHIOs facilitate the automatic delivery of Continuity of Care Documents (CCDs)—an electronic document exchange standard for clinical documents, or other defined sets of electronic message standards designed to facilitate the data exchange of a clinical document between EHRs from different vendors. This method also included instances of DIRECT project services, if they were provided by the RHIO. DIRECT is a federal project designed to facilitate the sharing of single-patient information between provider EHRs on a point-to-point basis.²³
3. Alternate-pull: clinicians manually retrieve clinical data from external organizations using a non—RHIO-related exchange service. Examples include providers with read-only privileges to the local hospital’s information systems (ie, providers use a login and password to remotely access laboratory and radiology data from another organization).
4. Alternate-push: clinical data are automatically delivered to the healthcare organization using non-RHIO services. Examples include vendor-mediated exchanges in which laboratory reports from specialists are automatically delivered to a PCP’s system. Instances of DIRECT protocol usage that were not RHIO-facilitated were included in this category. These information retrieval pathways are resources in addition to each organization’s internal information systems, such as an EHR, picture archiving and communication system (PACS), or laboratory information system. Table 3 outlines, by care setting, the favored methodologies used by healthcare professionals to retrieve information.

Emergency Departments

For healthcare professionals working in EDs, their first choice for seeking information was using their own EHR and other hospital-based systems. Generally, interviewees thought going to their own systems was “a lot easier” and that their systems were fairly comprehensive and up to date. For example, radiology data was often retrieved through a hospital’s PACS, were because “[it] has everything that the patient has had done.” One Community C clinician noted, “Our EHR is where I get my lab results on a daily basis.”

There were some exceptions. When healthcare professionals needed “historical data,” wanted to “compare patient’s labs prior,” or if all other internal information gathering approaches had been exhausted, they would use RHIO-pull. In the ED, RHIO-pull was used to obtain additional radiology information. A clinician in Community B noted that the ability to retrieve images and reports was particularly useful when she knew the patient had sought care in other hospitals. Similarly, RHIO-pull was the only option to obtain prior laboratory data; it was commonly used for comparing historical laboratory values with current test values. In all communities, RHIO-pull was used to retrieve data for medication reconciliation or in response to patients exhibiting drug-seeking behavior. For example, a Community Clinician reported he would pull data “when I questioned a patient’s use of medications.”

Primary Care Practices

Healthcare professionals at PCPs used a combination of information retrieval pathways for HIE. In general, clinicians preferred to have clinical data automatically transferred into their EHRs from external organizations, via RHIO-push, because it did not interfere with their natural work flow. In comparison, RHIO-pull was less preferred because it led to inconveniences and work flow disruptions due the additional time and effort required to access and find data in the RHIO-managed Web-query portal. One clinician said, “We get CCDs through [RHIO]. We can access [RHIO-managed Web-query portal], although we don’t do that very much just because it really disrupts the work flow.” Some clinicians used both RHIO-push and RHIO-pull depending on the situation. One staff member said “We’ll use the Web portal depending on what we get from the CCD. Sometimes it’s easier to go to the Web portal. Sometimes it’s easier to do the CCD...”

Because RHIO-push used CCDs, it was the primary mechanism used to procure medication and laboratory information from external sources. However, RHIO-push still presented challenges. For example, interviewees said that when data are pushed into their systems, time is required for interpretation. They said, too, that the amount of information delivered to the practice was substantial and “a little intimidating.” Also, because of the format of the CCD and the amount of information each document contained, several interviewees used the RHIO-pull or Alternate-pull. One clinician noted, “Say they had an x-ray done at [local hospital]. We can go directly to [their PACS] and get that a little quicker than you could [if you went] through the entire CCD.” Providers also reported that technical challenges sometimes caused them to use RHIO-pull, because they “cannot parse an x-ray or an x-ray report into the system.” Lastly, regardless of method of delivery, data still had to be checked for accuracy: “[If] a patient says they have an allergy, but the CCD says they don’t, you’re going to listen to the patient.”

PCPs had access to many sources of information outside the efforts of the RHIO-based services and used alternate-pull and alternate-push to engage in HIE. Community A had access to a local hospital’s EHR, a laboratory company’s information systems, a hospital’s PACS, vendor-mediated delivery of clinical documents, and access to Surescripts as part of their EHR. Practices in Community B also had access to vendor-mediated document delivery and to a local hospital’s laboratory information system and PACS. Laboratory reports from specialists were an example of vendor-mediated exchange: one clinician said, “When our patients go to see the specialists, we get cc’d labs that are done.”

Remote log-ins, the process by which clinicians are provided with remote access privileges to another organization’s system by using a user name and password, were prevalent for laboratory and radiology data. One clinician said, “I generally go to the source,” and another added, “If we needed to see the images that the different hospital systems have, a PAC system is set up so that’s where we ended up signing in.” Remote logins did frustrate some clinicians who said, “[External organizations] kind of lock everything down where you need 18 steps to get to where you need to go.” In general, when Alternate-pull proved difficult, PCPs would resort to pulling the data from the RHIO-managed Web-based query system. A clinician said, “If I’ve done blood work on somebody, and they’re supposed to send it...sometimes it doesn’t always get sent...So I can go on [RHIO-managed Web-based query system] and look up the results.”

Public Health

Healthcare professionals in public health settings relied on the RHIO-pull and Alternate-pull methods for information retrieval. Clinicians regularly used RHIO-pull to retrieve information not available in their own internal systems. For example, a tuberculosis nurse emphasized: “The first thing I usually do is go into the RHIO and get some background on each of the cases that I’ve gotten a report on. I’m looking at chest x-ray results, CAT [computed tomography] scan results, biopsy results, recent clinic notes from a physician [explaining] why they got the test.” Additionally, accessing the RHIO-pull supported efficient work. A case manager said, “It allows us to access radiology reports instead of waiting for the provider...” and another added, “I do obtain those lab reports regularly, and [it] is a huge help to be able to get them as soon as they’re available.”

Alternate-pull methods were predominantly in the form of remote logins to other organizations. This included access to the local hospital and a local radiology group. One nurse said, “Most of the time we can access their imaging online,” and another said, “I will see which lab it came from and I will ordinarily go to the [local hospital EHR].” For imaging, clinicians relied on the Alternate-pull for getting information. One nurse stated, “We’re pretty good about getting the reports and the x-rays sent over from the hospitals. I think the reports are faxed and then the courier brings over the chest x-rays,” and another added, “We have to call and ask them to get us a disk over here or we pick up the disk.”

DISCUSSION

Healthcare professionals used a complex combination of information retrieval pathways for HIE to obtain clinical information from external organizations. The choice for each approach was setting- and information-specific, but was also highly dynamic across users and their information needs.

Across each clinical site, several general features of information retrieval approach were consistent. First, the organization’s own internal information system (ie, its EHR) was the primary source of information. Second, when the EHR was the focal technology, that resulted in more reliance on “push” methods for the automatic delivery of information, whether it was through RHIO-push or Alternate-push. Data being pushed were not without technical challenges (ie, too much data), but the information was still accessible and stored within the EHR. Third, pull methods were used when the primary information retrieval method failed to meet clinicians’ information needs. Of the 4 information retrieval pathways for HIE, RHIO-pull required the most change to workflows.

These findings imply that HIE models need to support both push and pull methods to meet the very diverse needs of health professionals.^17,23 Although levels of usage of each HIE model varied, each type of exchange served a purpose to meet the user’s information needs. This has further relevance for those measuring and evaluating exchange activities. The multiple mechanisms and reasons for exchange suggest somewhat of a substitution effect among systems (eg, Alternate-pull used instead of RHIO-pull). As a result, measuring individual system usage may be inadequate or misleading about the level of HIE adoption. Composite measures of all exchange activity may be more informative.

As multiple HIE models appear to be the reality for the foreseeable future, attempts to centralize data will continue to be a challenge. To make data management easier for providers and organizations, those working to foster the exchange (eg, RHIOs, vendors, and government agencies) can support numerous technological innovations. Given that the EHR is the primary source of information, we suggest that the more the data from the exchange systems are integrated and work flows are harmonized with the EHR, the better.

The choice of primary and secondary information sources among different sites suggests implementation priorities for those fostering exchange. Based on this study, push methods would address many of the work flow and information needs of primary care settings, particularly when considering patient information like medication and laboratory results that have discrete data elements capable of being parsed. The nature of care in the ED suggests using RHIO-pull may be the best match. Public health agencies have less advanced internal information systems, but they possess a voracious appetite for information and their work flows are based on prolonged data collection and investigation.⁴⁰ HIE systems using RHIO-pull fit the public health work flow, but only if they contain a broad amount of information sources. Remote portal access to local hospitals will support public health work, but will not necessarily help efficiency if users have to repeatedly search multiple systems institution by institution.

In addition, of interest to the leadership of health organizations, these findings indicate that both clinical and nonclinical staff would benefit from organizational support. First, the sheer number of available information sources does not, in and of itself, create increases in productivity.⁴¹ Multiple systems with different log-ins, use cases, and designs can place strains on the organization and on individuals.⁴²

As more demands are being placed on the workforce for technology skills and competencies,⁴³ managers need to ensure that staff members have sufficient computer skills to navigate multiple systems and avenues when challenges arise. One possibility to address the complex data-gathering situation for clinicians would be to introduce medical scribes. By assuming most of the data-gathering and documentation responsibilities, scribes tend to increase physician productivity.⁴⁴ Second, the multiple information retrieval pathways indicates that professionals have complex mental models of determining where the data are available and under what circumstances they expect to find it. With the increased availability of externally created information, healthcare organizations are going to need to develop formalized methods for retaining and using this knowledge. These strategies can include creating directories, developing formal learning collaboratives, sharing best practices, holding discussion groups, or specific trainings.^45-47

Study Limitations

The primary limitation of this study is that it is confined to experiences in New York state; the differences in infrastructure, policies, and maturity of HIE systems may limit the generalizability of our findings. Additionally, while our sample included both urban and rural settings and organizations of different types and sizes, we cannot speak to the experiences of specialty care providers or use of HIE systems for inpatient care.

CONCLUSION

The exchange of health information is a critical strategy to improve patient safety and the quality of care. Healthcare professionals make use of a variety of technologies and methods to support their information needs. Understanding clinicians’ information retrieval pathways for HIE can assist in improving the design and functionality of HIE systems and aid their integration into organizational work flows. These findings provide insights for informatics professionals about the usage of information, indicate the need for managerial support within each organization, and suggest approaches to improve systems for organizations and agencies working to expand HIE adoption.

Acknowledgments

We would like to thank the health information exchange efforts, the public health departments, and study participants for their cooperation in the data collection. This project was funded by the New York eHealth Collaborative as a part of the State HIE Cooperative Agreement Program evaluation. The Institutional Review Board of Weill Cornell Medical College approved this study.

Author Affiliations:

University of Copenhagen, Denmark (PK); Center for Healthcare Informatics & Policy, Department of Healthcare Pol- icy and Research, Weill Cornell Medical College, New York, NY (RK, JRV).

Source of Funding:

This project was funded by the New York eHealth Collaborative as a part of the State HIE Cooperative Agreement Program evaluation.

Author Disclosures:

Drs Kierkegaard, Vest, and Kaushal report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. Dr Vest has plans to submit a grant proposal with Rochester RHIO as the study site in the future, but there is no funding for them other than as a data provider.

Authorship Information:

Concept and design (PK, RK, JRV); acquisi- tion of data (PK, JRV); analysis and interpretation of data (PK, RK, JRV); drafting of the manuscript (PK, RK, JRV); critical revision of the manuscript for important intellectual content (PK, RK, JRV); statistical analy- sis (PK, JRV); obtaining funding (RK, JRV); administrative, technical, or logistic support (RK).

Address correspondence to:

Patrick Kierkegaard, PhD, University of Copenhagen, Njalsgade 128, Bygning 24, 5. sal, 2300 København S, Denmark. E-mail: patrick.kierkegaard@di.ku.dk.

REFERENCES

1. Vest J. Health information exchange: national and international ap- proaches.

Adv Health Care Manag.

2012;12:3-24.

To Err Is Human: Building a Safer Health System.

2. Committee on Quality of Health Care in America; Kohn LT, Corrigan JM, Donaldson MS, eds. Washington, DC: National Academy Press; 2000.

JAMA

3. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. . 2007;297(8):831-841.

JAMA

4. Elder NC, Hickner J. Missing clinical information: the system is down. . 2005;293(5):617-619.

JAMA

5. Smith PC, Araya-Guerra R, Bublitz C, et al. Missing clinical informa- tion during primary care visits. . 2005;293(5):565-571.

BMC Health Serv Res

6. Burnett S, Deelchand V, Franklin BD, Moorthy K, Vincent C. Missing clinical information in NHS hospital outpatient clinics: prevalence, causes and effects on patient care. . 2011;11:114.

CMAJ

7. Tam VC, Knowles SR, Cornish PL, Fine N, Marchesano R, Etchells EE. Frequency, type and clinical importance of medication his- tory errors at admission to hospital: a systematic review. . 2005;173(5):510-515.

Evolution of State Health Information Exchange/A Study of Vision, Strategy, and Progress.

8. Rosenfeld S, Koss S, Caruth K, Fuller G. Rockville, MD:The Agency for Healthcare Research and Quality; 2006.

Report to the Office of the National Coordinator for Health InformationTechnology on Defining Key Health InformationTechnologyTerms.

9.The National Alliance for Health InformationTechnology. Washington, DC: Office of the National Coordinator for Health Information Technology; 2008.

Health Information Exchange Roadmap: The Landscape and the Path Forward.

10. National eHealth Collaborative. Washington, DC: National eHealth Collaborative; 2012.

J Biomed Inform

11. Hripcsak G, Kaushal R, Johnson KB, et al.The United Hospital Fund meeting on evaluating health information exchange. . 2007;40(6Suppl):S3-S10.

12. Bailey JE, Wan JY, Mabry LM, et al. Does health information exchange reduce unnecessary neuroimaging and improve quality

J Gen Intern Med

of headache care in the emergency department? . 2013;28(2):176-183.

J Biomed Inform

13. Kaelber DC, Bates DW. Health information exchange and patient safety. . 2007;40(6Suppl):S40-S45.

J Biomed Inform

14. Frisse ME, Holmes RL. Estimated financial savings associated with health information exchange and ambulatory care referral. . 2007;40(6 suppl):S27-S32.

Health Aff (Millwood).

15. Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B.The value of health care information exchange and interoperability. 2005;Suppl Web Exclusives:W5-10-W5-18.

J Am Med Inform Assoc

16. Frisse ME, Johnson KB, Nian H, et al.The financial impact of health information exchange on emergency department care. . 2012;19(3):328-333.

AMIA Annu Symp Proc.

17. Campion TR Jr, Ancker JS, Edwards AM, Patel VN, Kaushal R; HI-TECH Investigators. Push and pull: physician usage of and satisfaction with health information exchange. 2012;77-84.

J Med Syst

18. Solomon MR. Regional health information organizations: a vehicle for transforming health care delivery? . 2007;31(1):35-47.

Harness the Power of Enterprise HIE.

19. Harris Healthcare Solutions. Melbourne, FL: Harris; 2012.

Appl Clin Inform

20.WindenTJ, Boland LL, Frey NG, Satterlee PA, Hokanson JS. Care everywhere, a point-to-point HIE tool: utilization and impact on patient care in the ED. . 2014;5(2):388-401.

.

21. A practical guide to understanding HIE, assessing your readiness and selecting HIE options in Minnesota. website http://www.health.state.mn.us/divs/hpsc/ohit/hieguidance/index.html. Updated July 2013. Accessed March 23, 2014

J Med Internet Res

22. Simon SR, Evans JS, Benjamin A, Delano D, Bates DW. Patients’ attitudes toward electronic health information exchange: qualitative study. . 2009;11(3):e30.

J Am Med Inform Assoc

23. Kuperman GJ. Health-information exchange: why are we doing it, and what are we doing? . 2011;18(5):678-682.

J Am Med Inform Assoc

24. Johnson KB, Unertl KM, Chen Q, et al. Health information exchange usage in emergency departments and clinics: the who, what, and why. . 2011;18(5):690-697.

J Am Med Inform Assoc

25. Vest JR, Zhao H, Jasperson J, Gamm LD, Ohsfeldt RL. Factors motivating and affecting health information exchange usage. . 2011;18(2):143-149.

J Am Med Inform Assoc

26. Unertl KM, Johnson KB, Lorenzi NM. Health information exchange technology on the front lines of healthcare: workflow factors and patterns of use. . 2012;19(3):392-400.

Am J Manag Care

27. Kaelber DC, Waheed R, Einstadter D, Love TE, Cebul RD. Use and perceived value of health information exchange: one public healthcare system’s experience. . 2013;19(10 Spec No.): SP337-SP343.

Am J Manag Care

28. Kern LM, Wilcox A, Shapiro J, Dhopeshwarkar RV, Kaushal R. Which components of health information technology will drive financial value? .18(8):438-445.

Lab Med

29. Wians FH Jr. Clinical laboratory tests: which, why, and what do the results mean? . 2009;40(2):105-113.

Radiology

30. Gunderman RB.The medical community’s changing vision of the patient: the importance of radiology. . 2005;234(2):339-342.

31. Healthcare Information & Management Systems Society Health Information Exchange Best PracticesTask Force. Health Information Exchanges: Similarities and Differences. HIMSS HIE Common Practices Survey Results White Paper. Chicago, IL: HIMSS; 2009.

J Am Med Inform Assoc

32. Phillips AB, Wilson RV, Kaushal R, Merrill JA; HITECH investigators. Implementing health information exchange for public health reporting: a comparison of decision and risk management of three regional health information organizations in New York state. . 2014;21(e1):e173-e177.

Fourth Generation Evaluation.

33. Guba EG, LincolnYS. Thousand Oaks, CA: SAGE Publications, Inc; 1989.

MIS Quarterly

34. Dubé L, Paré G. Rigor in information systems positivist case research: current practices, trends, and recommendations. . 2003;27(4):597-635.

35. Crabtree BF, MillerWL. Doing qualitative research:Thousand Oaks, CA: Sage Publications, Inc; 1999.

Nurs Health Sci.

36. Sadler GR, Lee HC, Lim RS, Fullerton J. Recruitment of hard-to- reach population subgroups via adaptations of the snowball sampling strategy. 2010;12(3):369-374.

Ethnography: Principles in Practice.

37. Atkinson P, Hammersley M. 3rd ed. NewYork, NY: Routledge; 2007.

Qual Health Res.

38. Morse JM.The significance of saturation. 1995; 5(2):147-149.

The Discovery of Grounded Theory; Strategies for Qualitative Research.

39. Glaser BG, Strauss AL. Chicago, IL: AldineTransaction; 1968.

Online J Public Health Inform

40. Vest JR, Issel LM, Lee S. Experience of using information systems in public health practice: findings from a qualitative study. . 2014;5(3):227.

Commun ACM

41. Brynjolfsson E.The productivity paradox of information technology. . 1993;36(12):66-77.

J Inf Sci

42. Bawden D, Robinson L.The dark side of information: overload, anxiety and other paradoxes and pathologies. . 2009;35(2):180-191.

AMIA Annu Symp Proc.

43. Morrison F, Zimmerman J, Hall M, Chase H, Kaushal R, Ancker JS. Developing an online and in-person HIT workforce training program using a team-based learning approach. 2011; 63-71.

Clinicoecon Outcomes Res

44. Bank AJ, Obetz C, Konrardy A, et al. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. . 2013;5:399-406.

MIS Quarterly

45. Alavi M, Leidner DE. Review: knowledge management and knowledge management systems: conceptual foundations and research issues. . 2001;25(1):107-136.

J Health Care Finance

46. Guptill J. Knowledge management in health care. . 2005;31(3):10-14.

J Med Sys

47. Yeager VA, Walker D, Cole E, Mora AM, Diana ML. Factors related to health information exchange participation and use. . 2014;38(8):78.