Understanding Physicians’ Work via Text Analytics on EHR Inbox Messages

ABSTRACT

Objectives: To develop a text analytics methodology to analyze in a refined manner the drivers of primary care physicians’ (PCPs’) electronic health record (EHR) inbox work.

Study Design: This study used 1 year (2018) of EHR inbox messages obtained from the Epic system for 184 PCPs from 18 practices.

Methods: An advanced text analytics latent Dirichlet allocation model was trained on physicians’ inbox message texts to identify the different work themes managed by physicians and their relative share of workload across physicians and clinics.

Results: The text analytics model identified 30 different work themes rolled up into 2 categories of medical and administrative tasks. We found that 50.8% (range across physicians, 34.5%-61.9%) of the messages were concerned with medical issues and 34.1% (range, 23.0%-48.9%) focused on administrative matters. More specifically, 13.6% (range, 7.1%-22.6%) of the messages involved ambiguous diagnosis issues, 13.2% (range, 6.9%-18.8%) involved condition management issues, 6.7% (range, 1.9%-13.4%) involved identified symptoms issues, 9.5% (range, 5.2%-28.9%) involved paperwork issues, and 17.6% (range, 9.3%-27.1%) involved scheduling issues. Additionally, there was significant variability among physicians and practices.

Conclusions: This study demonstrated that advanced text analytics provide a reliable data-driven methodology to understand the individual physician’s EHR inbox management work with a significantly greater level of detail than previous approaches. This methodology can inform decision makers on appropriate workflow redesign to eliminate unnecessary workload on PCPs and to improve cost and quality of care, as well as staff work satisfaction.

Am J Manag Care. 2022;28(1):e24-e30. https://doi.org/10.37765/ajmc.2022.88817

Takeaway Points

Electronic health records (EHRs) have transformed the daily work of primary care physicians (PCPs). Decision makers currently have limited ability to understand physicians’ actual EHR inbox work composition.

• This study demonstrates that text analytics allow understanding of specific work themes managed by each PCP through inbox messages with a greater level of detail than previous approaches.
• A significant part of PCPs’ EHR inbox work is in fact concerned with administrative matters and non–clinically complex work.
• This methodology is a starting point for managers to identify opportunities to redesign workflow processes and practices and to improve quality of care and work satisfaction.

Electronic health records (EHRs) have transformed the daily work of primary care physicians (PCPs) and are now ubiquitous in most practices. However, in spite of promised benefits, PCPs now find that they spend more than half of their work on the EHR,^1-3 with a significant fraction dedicated to administrative tasks.^1,4 The hope was that more sophisticated EHR systems would reduce work burden on care team members, but this has not been the case: The dissemination of EHR systems reduced the work burden on support staff while at the same time increasing PCPs’ work burden.⁵ With this, EHR impact on physicians’ wellness has become a great concern for most health care organizations.^6-9 Moreover, as more of PCPs’ work is now conducted outside well-documented in-person clinic visits, there is limited identification of the scope of tasks they manage and how this varies across PCPs. Partially motivated by these circumstances, there have been time motion studies and studies that leverage EHR log data to better understand various aspects of physicians’ EHR work.^1,6,10,11 Core measures of EHR usage have been developed, such as “work outside of work.”¹² Studies have also focused on quantifying the work allocation among different tasks (eg, clinical face time with patients).¹¹

Given that inbox message management comprises more than 25% of physician work,¹ and these messages have become the primary mode of communication among physicians, team members, and patients,¹³ it is important to understand specific drivers of that work at a more granular level, to improve efficiency and quality while decreasing work burden. A few studies have focused on the work component of managing EHR inbox messages, but most were conducted when the dissemination of EHR systems was still evolving and the tasks that they were used to conduct were relatively less complex.^14-16 Other studies focused on quantifying the number of messages received relative to generic categories (eg, system-generated notifications) without considering the specific tasks managed by physicians.^17,18 These studies highlighted that receiving more than the average of system-generated inbox messages was associated with higher probability of burnout.¹⁷ However, these types of statistics do not lend themselves to directly inform related hospital policies. To the best of our knowledge, few studies look at the specific content of inbox messages,^19,20 and none provides an automated scalable methodology to analyze the specific work managed through EHR inbox communications, which is believed to be a critical part of the PCP’s clinical work.¹³ It is therefore essential to not only quantify the overall level of workload related to inbox messages management by PCPs, but also to understand in a refined manner the specific work themes and related drivers of communications between the PCP and team members and patients.

The purpose of this study is to develop a data-driven scalable approach to identify salient work themes (topics) in the EHR messages managed by PCPs. This can help in differentiating clinically complex work of PCPs from work that could be handled by other care team members. Specifically, messages related to the management of complex clinical conditions and patient health need the PCP involvement, whereas messages related to “simpler” clinical tasks (eg, routine preventive work) or administrative tasks (eg, scheduling) are better managed by other team members. The value of this methodological approach is that it allows analyzing physician work composition at a great level of detail. This provides critically important diagnostic tools to help inform policies that address compensation of PCPs, as well as redesign of related workflow processes to improve quality, cost of care, and staff work satisfaction.

One challenge to developing detailed understanding of PCPs’ work is that it requires massive manual review of messages or otherwise relying on qualitative reporting by PCPs. This paper leveraged advanced text analytics to develop a data-driven and computer-enabled methodology to conduct such analysis. The methodology and tools developed in this study allow unprecedented understanding of the substance and complexity of work themes managed by PCPs through EHR inbox messages. Moreover, the ability to understand heterogeneity in work composition across PCPs and practices is critical to be able to identify opportunities to improve the quadruple aim: quality, cost, patient satisfaction, and physician satisfaction.

METHODS

Study Setting and Population

This work included retrospective analysis of all the EHR messages managed by 184 PCPs who worked in 18 primary care practices at a large academic medical center in Boston, Massachusetts, from March 1, 2018, through March 1, 2019. Table 1 displays characteristics of this cohort. Note that the academic medical center has been using the Epic EHR system since 2016.

Data Sources

The analysis used a self-constructed data set that integrates 2 main sources: (1) EHR inbox message data and (2) PCP patient panel data.

In the academic medical center, the Epic EHR system is the primary messaging system for communication within the practice team and with patients. Inbox messages include messages from patients and care team members, as well as messages generated automatically by the EHR system. Only messages from patients and care team members contain text information that is not protocol driven. More specifically, those inbox messages that are not protocol driven include communications with patients as well as communications among care team members that are initiated by a patient’s message, by a patient’s phone call, by the need to transfer an electronic note related to a patient, or as part of the coordination of an action required for a given patient. The forthcoming analysis focuses on this specific category of patient-initiated inbox messages. Such patient-initiated communications correspond to 36% of inbox messages received by the 184 PCPs included in the study. The remaining EHR protocol-driven messages can be divided into system-generated notifications, lab results notifications, and prescription-related notifications (authorization and signing), which represent, respectively, 26%, 9%, and 29% of received inbox messages.

The second main data source used in the analysis is an internal registry of patient assignments to the panel of individual PCPs. This defines the specific patient population for which each PCP cares.

Using the PCP patient panel data, each message from the inbox message data regarding a given patient was associated with a PCP. Only inbox messages concerning patient-initiated communication were selected in the final data set. This final data set captured a total of 1,279,712 messages.

Analysis

Latent Dirichlet allocation model. A latent Dirichlet allocation (LDA) model was trained on the text messages.²¹ The LDA model is a generative probabilistic Bayesian model for discovering the topics that occur in a corpus of text documents. The number of desired topics to discover is given as input to the model. After training the model on all text messages, each identified topic was characterized by the respective frequency of each word from the corpus vocabulary. The LDA model then identifies, for each message, what specific topics are discussed in the message and what relative portion of the message is dedicated to each topic (eg, the model can determine that a message consists of 60% of topic A and 40% of topic B). Because of uncertainty in the modeling, there can be a small percentage of the message topics mixture that corresponds to a nonspecific topic (ie, the part of the message not explained by the LDA-identified topics).

The number of topics given as input to the LDA model was selected manually, using the common practice of expert assessment.^22,23 The detailed methodology can be found in eAppendix Exhibit 1 (eAppendix available at ajmc.com).

Work theme (topic) identification. After selecting the best LDA model, a manual review was performed, with the help of 2 physicians, on each of the topics identified by the LDA model to label the work theme captured in the respective topic. Manual review involved interpretation of the most frequent words within a topic. A general theme was selected based on this review. The methodology used to evaluate external accuracy of the algorithm predictions for a given message is detailed in eAppendix Exhibit 2.

To understand the output of the model and the role of the different topics, some message analysis was performed and is described in detail in eAppendix Exhibit 3.

Theme (topic) taxonomy. The overall aim of this methodological step was to provide a hierarchical taxonomy of the LDA-identified topics and their respective work themes, so that it would be possible to analyze the share of PCPs’ work concerned with more general work categories. The higher level of the proposed taxonomy included administrative and medical issues. This fundamentally distinguishes between clinical messages on which the physician’s input is important compared with administrative messages, which likely could be handled by other care team members. A second more granular level was proposed to divide the administrative category and the medical category into subcategories. After discussion with 2 physicians, the medical category was divided into the following subcategories: identified symptoms, ambiguous diagnosis, condition management, tests and exams, and clinical decision-making referral. Similarly, the administrative category was divided into the following subcategories: paperwork, scheduling, administrative referral, and prescription. It should be emphasized that other subcategories are possible here, as the purpose of this second level of the taxonomy is only to provide a higher-level analysis of the work composition.

Each previously identified topic was then assigned to one of the categories of the proposed taxonomy described above.

PCP-level analysis. All statistics calculated at the PCP level considered all the patient-initiated messages where the PCP was involved either as sender or receiver of the message. The share of messages concerned with a given topic was calculated for each PCP, by summing over all messages the respective percentage that this specific topic represented (ranging from 0%-100%) and then calculating the mean. Following this analysis, it was possible to characterize, for each PCP, what the EHR messages–related workload was across the different work categories from the taxonomy.

Practice-level analysis. Practice-level analysis was performed by aggregating the statistics regarding the share of work per topic among all doctors from a given practice.

RESULTS

LDA Model

Using human assessment of topic model validity as explained in the Methods section, the best number of topics selected was 30. For models with a smaller number of topics, some topics were missing, and other topics seemed to combine what would preferably be better differentiated themes. For models with more than 30 topics, additional topics were assessed to be less meaningful in terms of discerning a unique theme.

Work Theme (topic) Identification

The list of the top 10 most frequent words for all topics is displayed in eAppendix Exhibit 4 and these are displayed as word clouds in eAppendix Exhibit 5 for a subset of topics. For example, the 5 most frequent words for the nutrition topic are food, diet, weight, eat, and cholesterol. Sample messages and their topic mixture are presented in eAppendix Exhibit 6.

The accuracy obtained with external manual validation was equal to 81% (84% and 78% specifically for each of the 2 experts), which corresponds to the average of the cumulative weight of the topics identified correctly. This indicates that most of the topics identified by the algorithm are correct.

Theme (topic) Taxonomy

Each identified work theme (topic) was first assigned to one of the granular subcategories. For example, the following topics were assigned to the subcategory condition management: medication, nutrition, patient-initiated care, chronic cardiovascular and diabetes condition, miscellaneous chronic condition, joint procedure, and surgical procedure. The topic assignment to one of the high-level categories (ie, administrative and medical) was then determined by the chosen subcategory. The complete resulting assignments of topics to categories are presented in Figure 1.

PCP-Level Analysis

On average, of the patient-initiated messages sent and received by physicians, 34.1% (range, 23.0%-48.9%; SD, 5.3%) involved administrative issues, whereas 50.8% (range, 34.5%-61.9%; SD, 5.3%) involved medical issues. The large range indicates significant variability among PCPs. The other 15.0% (range, 11.3%-20.5%; SD, 1.7%) were concerned with nonspecific topics (ie, the part of messages not explained by the LDA-identified topics).

Results at the subcategory level are presented in Table 2. We found that 13.6% (range, 7.1%-22.6%; SD, 3.1%) of the patient-initiated messages sent and received by physicians involved ambiguous diagnosis issues, 13.2% (range, 6.9%-18.8%; SD, 2.1%) involved condition management issues, 6.7% (range, 1.9%-13.4%; SD, 1.4%) involved identified symptoms issues, 9.5% (range, 5.2%-28.9%; SD, 2.3%) involved paperwork issues, and 17.6% (range, 9.3%-27.1%; SD, 3.2%) involved scheduling issues.

Practice-Level Analysis

Figure 2 reports the mean share of messages concerned with administrative and medical topics across different practices. The mean across all PCPs from a given practice differed greatly from one practice to another; for example, it ranged from 28% to 42% for the administrative topic category.

DISCUSSION

Demand for non–face-to-face care is increasing for primary care physicians, and handling EHR inbox messages represents a large share of such non–face-time work.^1,13 Moreover, the handling of these messages is a source of significant work burden that has been identified as being associated with burnout.¹⁷ It is therefore critical to improve the level of understanding of EHR inbox management work burden by exploring the specific drivers of communications between PCPs and their support team and patients. The contributions of this study are 2-fold. First, this study develops an advanced text analytics approach to identify detailed work composition and themes managed by PCPs in their EHR inbox messages that can be implemented at scale with minimal work. Second, the new methodology offers important new insights on PCPs’ EHR inbox work composition and how it differs across PCPs and practices.

As already mentioned, the analysis highlights several important concrete insights regarding PCPs’ work composition. First, it highlights the great complexity of PCPs’ work, which typically requires simultaneous management of multiple tasks that may be very different in nature from one another, often including a blend of clinical and administrative issues. For example, handling an acute patient problem may require clinical diagnosis of the problem, as well as scheduling an appointment by the doctor. This underscores the importance of understanding the specific work topics managed by PCPs and their corresponding complexity, and not merely monitoring the volume of work.

Related to that, the analysis highlights that of all the time PCPs spent managing EHR inbox messages, a significant fraction was spent handling issues classified as administrative topics. Specifically, more than 34% of the messages managed by the PCPs in the study were concerned with administrative topics. It is likely that some of these administrative messages could be handled by other team members. For example, scheduling and paperwork issues mostly require the involvement of administrative staff, and although signing off on a prescription may require the involvement of a PCP, administrative issues related to prescriptions (eg, prescription sent to wrong pharmacy) could be addressed by nurses or medical assistants. Previous research already highlighted the significant share of administrative and clerical work (eg, related to order entry, billing, and coding)¹ as a possible factor associated with burnout.^7,24 The fact that more than 30% of patient-initiated EHR inbox messages are concerned with administrative work further describes and quantifies the nonclinical work conducted by physicians and raises the question of whether it has any impact on physician burnout. Although this paper does not provide an answer to this question, it provides important data to stimulate further relevant research. In particular, more research should be dedicated to understanding the possible link between this administrative share of messages and physicians’ and patients’ well-being and, more specifically, what the “optimal” share between clinical and administrative work is.

The broad range and variable complexity of clinical tasks that are managed by PCPs raises another question of whether the PCP is always the best team member to handle them, or whether some of them could also be delegated to nonphysician clinicians. For example, some of the messages related to the condition management category (eg, nutrition, routine preventive counseling) could probably be delegated to nurses or medical assistants. This suggests that health care organizations should consider the broad range of work themes currently managed by PCPs when redesigning operational processes. This insight is aligned with the results of other studies that suggested, for example, a shift from physician-centric models to shared-care models by expanding the roles of nurses and medical assistants to meet patients’ clinical needs with more efficient use of PCPs’ time.²⁴

The observed variability across practices regarding the administrative share in the messages managed by PCPs supports the hypothesis that practice-level drivers can have an impact on PCPs’ work composition. Practice-level factors such as standardized workflow processes and staffing schemes are modifiable by leaders, and future research could help in determining which of those factors play an important role in affecting work composition.

The previous discussion suggests that the methodology and tools developed in this study enable the first step to go beyond merely monitoring PCPs’ work volume and to understand the details of the work currently managed by PCPs. Such diagnostic tools are a starting point for future research to study workflows and their impact on work composition. They can allow health care systems and policy makers to identify opportunities to redesign workflow processes and practices and shape the respective payment schemes accordingly. Moreover, the newly developed approach could be used to accurately measure and assess the impact of these efforts after changes took effect. The complexity and variability of the different work themes handled by the PCPs also suggest that a “one-size-fits-all” solution would probably fall short of success and that different workflow redesigns should be considered.

These are all key enablers to obtain the quadruple aim of improving cost and quality of health care, as well as patient and physician satisfaction.

Limitations

This study has several limitations. First, the methodology was used on PCPs within a single institution; it would be interesting to observe the generalizability of the obtained results to other hospitals, and to physicians from other specialties. Second, EHR data were extracted from the Epic EHR system. Other EHR systems may yield different insights regarding work composition, because of, for example, different inbox message formats and templates that may affect communication workflows. Third, the algorithm identifying work themes based on text can be sensitive to patient health literacy, as level of education is known to be correlated with how patients use internet-based patient portals.^25,26 Lastly, this study only provides descriptive analysis with respect to the content of managed inbox messages and does not analyze other potentially important factors related to workflow processes (eg, use of pooled email queues or forwarding messages).

CONCLUSIONS

Handling EHR inbox messages is an important driver of work burden for physicians. This study illustrates that advanced text analytics provide a reliable data-driven methodology to identify the work themes managed by each PCP through their EHR inbox with significantly greater level of detail. This methodology is a critical first step to inform decision makers on the redesign of workflow processes to eliminate unnecessary workload on PCPs and improve cost and quality of care.

Acknowledgments

The funding of the first author was partially supported by the Susan Hockfield – Thomas Byrne Fellow in Clinometrics. The authors would like to thank Joshua Metlay, MD, PhD, chief of the Division of General Internal Medicine, Massachusetts General Hospital, and Kerri Palamara, MD, director of the Center for Physician Well-Being, Massachusetts General Hospital, for their help and consultation on the project. The authors would also like to thank the editor and the anonymous reviewers for their constructive comments that helped in substantially improving the paper substance and exposition.

Author Affiliations: Operations Research Center (CE) and Sloan School of Management (RL), Massachusetts Institute of Technology, Cambridge, MA; Division of General Internal Medicine (SAE) and Pulmonary/Critical Care Division (WJO), Massachusetts General Hospital and Harvard Medical School, Boston, MA.

Source of Funding: This work was funded by a Massachusetts General Hospital grant (grant no. 220150) through the Susan Hockfield – Thomas Byrne Fellow in Clinometrics.

Author Disclosures: Ms Escribe was funded by a grant from Massachusetts General Hospital. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (CE, SAE, WJO, RL); acquisition of data (CE, SAE, WJO, RL); analysis and interpretation of data (CE, SAE, WJO, RL); drafting of the manuscript (CE, SAE, WJO, RL); critical revision of the manuscript for important intellectual content (CE, SAE, WJO, RL); statistical analysis (CE, SAE, RL); and obtaining funding (WJO, RL).

Address Correspondence to: Retsef Levi, PhD, Sloan School of Management, Massachusetts Institute of Technology, Operations Research Center, 100 Main St, E62-416, Cambridge, MA 02142. Email: retsef@mit.edu.

REFERENCES

1. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med. 2017;15(5):419-426. doi:10.1370/afm.2121

2. Young RA, Burge SK, Kumar KA, Wilson JM, Ortiz DF. A time-motion study of primary care physicians’ work in the electronic health record era. Fam Med. 2018;50(2):91-99. doi:10.22454/FamMed.2018.184803

3. Tai-Seale M, Olson CW, Li J, et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood). 2017;36(4):655-662. doi:10.1377/hlthaff.2016.0811

4. Erickson SM, Rockwern B, Koltov M, McLean RM; Medical Practice and Quality Committee of the American College of Physicians. Putting patients first by reducing administrative tasks in health care: a position paper of the American College of Physicians. Ann Intern Med. 2017;166(9):659-661. doi:10.7326/M16-2697

5. Howard J, Clark EC, Friedman A, et al. Electronic health record impact on work burden in small, unaffiliated, community-based primary care practices. J Gen Intern Med. 2013;28(1):107-113. doi:10.1007/s11606-012-2192-4

6. Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27(4):531-538. doi:10.1093/jamia/ocz220

7. Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016;91(7):836-848. doi:10.1016/j.mayocp.2016.05.007

8. Babbott S, Manwell LB, Brown R, et al. Electronic medical records and physician stress in primary care: results from the MEMO study. J Am Med Inform Assoc. 2014;21(e1):e100-e106. doi:10.1136/amiajnl-2013-001875

9. Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26(2):106-114. doi:10.1093/jamia/ocy145

10. Gilchrist V, McCord G, Schrop SL, et al. Physician activities during time out of the examination room. Ann Fam Med. 2005;3(6):494-499. doi:10.1370/afm.391

11. Sinsky C, Colligan L, Li L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med. 2016;165(11):753-760. doi:10.7326/M16-0961

12. Sinsky CA, Rule A, Cohen G, et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc. 2020;27(4):639-643. doi:10.1093/jamia/ocz223

13. McMahon LF Jr, Rize K, Irby-Johnson N, Chopra V. Designed to fail? the future of primary care. J Gen Intern Med. 2021;36(2):515-517. doi:10.1007/s11606-020-06077-6

14. Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis. Am J Med. 2012;125(2):209.e1-e7. doi:10.1016/j.amjmed.2011.07.029

15. Murphy DR, Reis B, Kadiyala H, et al. Electronic health record-based messages to primary care providers: valuable information or just noise? Arch Intern Med. 2012;172(3):283-285. doi:10.1001/archinternmed.2011.740

16. McDonald CJ, McDonald MH. Electronic medical records and preserving primary care physicians’ time: comment on “electronic health record-based messages to primary care providers.” Arch Intern Med. 2012;172(3):285-287. doi:10.1001/archinternmed.2011.1678

17. Tai-Seale M, Dillon EC, Yang Y, et al. Physicians’ well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood). 2019;38(7):1073-1078. doi:10.1377/hlthaff.2018.05509

18. Murphy DR, Meyer AND, Russo E, Sittig DF, Wei L, Singh H. The burden of inbox notifications in commercial electronic health records. JAMA Intern Med. 2016;176(4):559-560. doi:10.1001/jamainternmed.2016.0209

19. Heisey-Grove D, Rathert C, McClelland LE, Jackson K, DeShazo JP. Patient and clinician characteristics associated with secure message content: retrospective cohort study. J Med Internet Res. 2021;23(8):e26650. doi:10.2196/26650

20. Heisey-Grove DM, McClelland LE, Rathert C, Tartaglia A, Jackson K, DeShazo JP. Associations between patient health outcomes and secure message content exchanged between patients and clinicians: retrospective cohort study. J Med Internet Res. 2020;22(10):e19477. doi:10.2196/19477

21. Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res. 2003;3:993-1022.

22. Hall D, Jurafsky D, Manning CD. Studying the history of ideas using topic models. In: Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2008:363-371. Accessed February 19, 2021. https://www.aclweb.org/anthology/D08-1038

23. Chang J, Boyd-Graber J, Wang C, Gerrish S, Blei DM. Reading tea leaves: how humans interpret topic models. In: NIPS ’09: Proceedings of the 22nd International Conference on Neural Information Processing Systems. Citeseer; 2009:288-296.

24. Sinsky CA, Willard-Grace R, Schutzbank AM, Sinsky TA, Margolius D, Bodenheimer T. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med. 2013;11(3):272-278. doi:10.1370/afm.1531

25. Sarkar U, Karter AJ, Liu JY, et al. Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access. J Am Med Inform Assoc. 2011;18(3):318-321. doi:10.1136/jamia.2010.006015

26. Fareed N, Swoboda CM, Jonnalagadda P, Huerta TR. Persistent digital divide in health-related internet use among cancer survivors: findings from the Health Information National Trends Survey, 2003-2018. J Cancer Surviv. 2021;15(1):87-98. doi:10.1007/s11764-020-00913-8