Relationships Between Patient Portal Activation and Patient Satisfaction Scores Among CG-CAHPS and HCAHPS Respondents

ABSTRACT

Objectives: Patient portals are health information technology tools that offer patients access to their personal health information and a means to communicate with health care providers, but little is known about their impact on patient satisfaction. Identifying factors that increase patient satisfaction may improve patient care and can protect health care providers from financial penalties. Our study sought to investigate how patient portals are associated with patient satisfaction in both inpatient and outpatient settings.

Study Design: Retrospective, pooled cross-sectional study.

Methods: Data from the Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) and Hospital CAHPS (HCAHPS) patient satisfaction surveys were linked to inpatient and outpatient portal account status at a large academic medical center. Using fractional logistic regression, we investigated the relationship between patient satisfaction survey scores and patient portal activation.

Results: Patients with an activated outpatient portal account reported higher patient satisfaction across a subset of the CG-CAHPS dimensions, and patients with an activated inpatient portal account reported higher patient satisfaction across a subset of the HCAHPS dimensions, compared with patients without the respective portal accounts.

Conclusions: Our study suggests the potential for patient portals to enhance patient satisfaction, especially in areas such as care coordination and care transitions. In both inpatient and outpatient settings, portal use may improve the patient-centeredness of care. Our findings indicate important considerations for both health care organizations and their patients to promote patient portal use as a means of improving patient satisfaction, especially in the context of potential impacts on reimbursement and reputation.

Am J Manag Care. 2022;28(1):25-31. https://doi.org/10.37765/ajmc.2022.88813

Takeaway Points

This study sought to investigate how patient portals in both inpatient and outpatient settings are associated with patient satisfaction as measured by Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) and Hospital CAHPS (HCAHPS) surveys.

• Patients with an activated outpatient portal account reported higher patient satisfaction in 2 dimensions of the CG-CAHPS survey (communication with their doctor and care coordination), and patients with an activated inpatient portal account reported higher patient satisfaction in 3 dimensions of the HCAHPS survey (communication with their nurse, care transition, and discharge), compared with patients without respective portal accounts.
• The highest marginal gains in patient satisfaction scores for patients with an activated portal account were for care coordination for the outpatient portal and care transition for the inpatient portal.
• Future work should evaluate the impact of patient portal functions and their use as mediating factors of the effect of patient portals on patient experience.

Patient satisfaction has become an important measure as the need for health care to be more patient centered has been increasingly emphasized. Identified as a key element of care quality by the Institute of Medicine,¹ patient-centered care encourages providers to include patients as partners, engaging and empowering patients to actively participate in their health care.² With the shift to this focus on patient-centered care, patient experience has become an important aspect of health care quality, in which patient satisfaction with their care is used as one measure of the patient experience.³

To improve patient satisfaction, health care providers are increasingly focused on increasing the patient-centeredness of care, and patient-centered health information technologies such as patient portals are one such tool that may be beneficial. Patient portals are offered as websites or mobile applications and are tethered to a patient’s electronic health record (EHR) to provide patients with access to their personal health information and a means to manage their health care and communicate with their health care providers. Through these functions, patient portals satisfy CMS’ Promoting Interoperability Program (formerly the Meaningful Use program) requirements to improve quality of care by supporting health information exchange between health care providers and their patients.^4,5

Although they were first implemented in the outpatient setting, patient portals are now increasingly implemented in the inpatient setting, with functions tailored to support patients during hospitalization. Outpatient and inpatient portals together have the potential to benefit patients across care settings. Multiple benefits of patient portals have been reported in previous studies, including improved patient knowledge about their health conditions, increased patient engagement, and enhanced patient-provider communication,^6-8 but the impact of patient portals on patient satisfaction is not well understood.

Our study aimed to address this gap in knowledge by exploring how outpatient and inpatient portals might be associated with patient satisfaction, as measured by Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) and Hospital CAHPS (HCAHPS) survey responses. The CG-CAHPS survey⁹ and the HCAHPS survey¹⁰ were developed by the Agency for Healthcare Research and Quality¹¹ and CMS¹² to collect standardized perspectives of patient satisfaction based on patients’ outpatient and inpatient experiences, respectively. These surveys assess patient satisfaction across several dimensions of the patient experience, such as communication with doctors, nurses, and staff; care coordination and care transitions; access to and responsiveness of care; and facility conditions.

Understanding the association between patient portal use and patient satisfaction, as measured by the CG-CAHPS and HCAHPS surveys, has important implications for the delivery of managed care. First, patient satisfaction measures are interpreted to reflect the quality of care received by the patient because they are based on patients’ perceived needs and expectations.^13-15 In addition, public reporting of patient satisfaction scores by CMS provides comparable data among health care providers that can affect their reputation and influence patient recruitment and retention.¹⁶ Furthermore, patient satisfaction is included as a factor in performance assessments that affect reimbursement payments from CMS.¹⁷ Understanding whether and how patient portals are associated with patient satisfaction can help direct health care organizations to use these tools in ways that positively affect patient experience.

METHODS

Study Setting

This study was conducted at The Ohio State University Wexner Medical Center, a large Midwestern academic medical center (AMC) that includes 6 hospitals and 53 ambulatory care locations. At this AMC, outpatients are offered electronic access to their health care information through an outpatient portal, MyChart (MC; Epic Systems). Inpatients at this AMC are offered Android tablets during their hospital stay; through these tablets, patients have access to an inpatient portal, MyChart Bedside (MCB; Epic Systems).

MC includes functions that allow patients to access their personal health information (eg, medical history, test results), view and schedule appointments, and message their providers. MCB includes functions that are tailored to the hospital setting; these functions allow patients to access their personal health information (eg, test results, vital signs), view an anticipated schedule for their medications and/or procedures, view educational materials, view names and pictures of their care team members, order meals, send requests, and message their care team (see eAppendix A for a full list of portal functions [eAppendices available at ajmc.com]).

Data and Study Sample

Retrospective data were obtained for patient encounters from our AMC’s Information Warehouse (IW) in accordance with the Honest Broker protocol; this protocol is recognized by The Ohio State University’s Institutional Review Board (IRB) as a means to use deidentified data without requiring formal IRB approval. Data were obtained for patient encounters between January 2019 and January 2020 for patients aged between 18 and 110 years. Our study sample consisted of 50,314 outpatient encounters and 7783 inpatient encounters (ie, hospital stays). Our study sample was focused on encounters that have CG-CAHPS or HCAHPS responses from our AMC’s patient population.

MC and MCB Account Activation

Using Epic System’s EHR log files for MC and MCB, we identified variables that captured patient activation of their portal accounts. We defined patients with an activated MC or activated MCB account as those patients who registered their portal account online with an activation code provided by the medical center and successfully logged into the portal system at least once (see eAppendix A for additional details about MC and MCB activation). Among all patients during our study period, approximately 53% had activated MC accounts and 46% had activated MCB accounts.

Patient Satisfaction Surveys: CG-CAHPS and HCAHPS

The IW provided us with the CG-CAHPS and HCAHPS survey scores for the patient encounters of our study population with and without activated MC and MCB accounts during the study period. CG-CAHPS and HCAHPS scores were collected on a rolling basis at the AMC and were aggregated for our study time period. Of note, both the CG-CAHPS and HCAHPS surveys have inclusion and exclusion criteria that are described in great detail in the literature.^18,19

The CG-CAHPS and HCAHPS surveys contained questions that were aggregated into composite measures related to 4 and 9 dimensions of patient care, respectively. Each survey also collected 2 global measures that assessed the rating and recommendation of either the provider or the hospital (see eAppendix B for a complete list of survey measures and dimensions). The measures for pain management in the hospital were excluded because scores for this measure were not available from the AMC during the study period because of an institutional decision to not collect this information. During our study period, CG-CAHPS and HCAHPS response rates at the AMC among all patients were approximately 14% and 19%, respectively.

We used top-box scores for each patient satisfaction dimension as the dependent variables for our study. The top-box score for each individual question was calculated as the fraction of respondents who selected the most positive response. Top-box scores within each dimension were aggregated by summing the top-box scores for all questions associated with that dimension and dividing by the total number of questions answered for that specific dimension. This resulted in the top-box proportion for each dimension, which was used as the dependent variable for the fractional logistic regression model. For example, a top-box proportion for Doctor Communication of 0.90 is constructed based on 2700 responses of “always” of a possible 3000 responses to any of the questions under this dimension.

Data Analysis

To develop our analytic data set, we merged data from multiple data sources using a unique identifier variable for patient encounters. Using this identifier, we linked patient demographic/clinical information, patient satisfaction scores, and activated MC/MCB account status for patients by encounter. We subsequently split our analytic data into 2 separate sets, one for the MC and CG-CAHPS analysis and the other for the MCB and HCAHPS analysis. Our first data set contained 50,314 eligible patient encounters with CG-CAHPS scores for 1 or more dimensions. Of these encounters, we had 36,798 (73.1%) encounters with activated MC accounts. Our second data set contained 7783 eligible patient encounters with HCAHPS scores, with 3729 (47.9%) of those encounters with activated MCB accounts. For both of our analytical data sets, only CG-CAHPS and HCAHPS responses that were reported after portal activation were retained.

We used patient demographic and clinical characteristics as covariates to obtain adjusted estimates from our empirical models. These covariates, including race, age, patient self-reported health status, health insurance status, gender, and survey mode, were obtained from the CG-CAHPS or HCAHPS survey or the EHR (see eAppendix C for additional details about the selection of covariates used in our empirical model). We also included a covariate that indicated multiple survey responses because respondents answering multiple surveys may have systematic differences in how they responded to the surveys compared with 1-time respondents.

Our study analysis included univariate and multivariate approaches. For the latter, we used fractional logistic regression model estimations on our separate data sets (see eAppendix C for more information about this analytic approach). Our analysis was at the patient encounter level and included patients with multiple encounters; hence, we estimated robust standard errors that clustered and accounted for the unique variance by patients. All statistical analyses, including our univariate tests (analysis of variance for continuous variables and χ² for categorical and binary variables), were performed using Stata 15 software (StataCorp LLC). For our multivariate analysis, we corrected for multiplicity by performing Bonferroni corrections to maintain a family-wise error rate (FWER) of 0.05. We indicate results that have P values below the adjusted α levels for each analytical set.

RESULTS

Table 1 and Table 2 provide descriptive statistics about demographic and clinical characteristics for the patient encounters in our CG-CAHPS and HCAHPS data sets. Patients with activated MC and activated MCB accounts were most often White, 65 years or older, and female and had good self-reported health status and commercial insurance. Between patients with activated patient portal accounts (MC or MCB) and their counterparts with no activated patient portal accounts, there were significant differences by all the demographic and clinical characteristic variables used in our analysis. Notably, for both data sets, the groups without activated portal accounts had much older patients and more patients on Medicare than the respective groups with activated portal accounts. Survey mode and multiple survey response differed significantly across groups within each data set. We compared our analytical samples with the overall outpatient and inpatient encounters at our AMC by race, age, gender, and health insurance status. There were statistically significant differences based on these factors for both analytical samples, with the exception of gender in the inpatient sample. Both the inpatient and outpatient samples were more likely to have White, older, and Medicare-covered patient encounters; the outpatient sample was also likely to have more male patient encounters (results available on request).

Table 3 reports the unadjusted and adjusted odds ratios for (1) the estimation of activated MC account status (reference: no MC account) on the 6 CG-CAHPS top-box dimensions and (2) the estimation of activated MCB account status (reference: no MCB account) on the 10 HCAHPS top-box dimensions. We also report the marginal effects for our adjusted models, which can be interpreted as the percentage change in the dependent variable (at the mean) associated with the switch from no patient portal to activated patient portal status.

With respect to the results in panel A, the odds of higher top-box proportions for doctor communication were 1.24-fold higher among patients with activated MC accounts compared with those without activated MC accounts. This was a 1.33% higher proportion of top-box scores for doctor communication associated with activated MC accounts. Similarly, the odds of higher top-box proportions for care coordination were 1.20-fold higher among patients with activated MC accounts compared with those without activated MC accounts. This was a 4.22% higher proportion of top-box scores for care coordination associated with activated MC accounts.

Panel B in Table 3 shows that patients with an activated MCB account had significantly higher odds of reporting higher top-box proportions for the HCAHPS dimensions, including nurse communication, care transition, and discharge. Notable relationships included care transition (7.24% higher). The other outcomes (ie, nurse communication and discharge) had higher proportions of top-box scores that ranged from 2.16% to 3.94%.

We reran our analysis for all the outcomes related to MC and MCB account status for encounters with only the most recent CG-CAHPS or HCAHPS scores (ie, without controlling for multiple responses). Our results were consistent for all but 1 outcome; the HCAHPS Discharge dimension was no longer significant at the FWER adjusted α. We discuss our findings based on our original models.

DISCUSSION

Our study found that patients with an active patient portal account had higher odds of reporting top-box scores on many dimensions of the CG-CAHPS or HCAHPS survey. Patients with an activated outpatient portal account had significantly higher odds of reporting top-box responses to dimensions of the CG-CAHPS survey regarding communication with their doctor and care coordination compared with patients without an activated outpatient portal account. Patients with an activated inpatient portal account had significantly higher odds of top-box responses to dimensions of the HCAHPS survey regarding nurse communication, care transitions, and discharge compared with patients without an activated inpatient portal account.

In many ways, functions of patient portals may be associated with different dimensions of the CG-CAHPS and HCAHPS surveys, as suggested by the findings of our study. For instance, patient portals support patient-provider communication through secure messaging in both inpatient and outpatient settings. In addition, patient portals can assist with the processes of care coordination, care transitions, and discharge from the hospital by providing access to personal health information and educational materials. Notably, the highest marginal gains in top-box scores in our study were for care coordination (4.22%) for the outpatient portal and care transition (7.24%) for the inpatient portal. By facilitating patient and provider communication regarding medications and test results, as well as providing patients tools to help them understand and manage their health conditions, patient portals may be able to support these dimensions of patient experience, as suggested by the association between these dimensions of patient satisfaction and patient portal use.

Interestingly, the dimensions of patient satisfaction that were not positively associated with activated patient portal status were those that were unlikely to be related to patient portal use. For example, activated outpatient patient portal status was not correlated with increased satisfaction with office staff, with whom the patients would not be expected to interact through the portal. Furthermore, portal status was not associated with increased satisfaction with access to care. Although the portal does offer a means to schedule appointments, the questions in this domain ask about phoning the provider’s office and the promptness of receiving appointments or care, which are extraneous to patient portal use. For activated inpatient portal accounts, there was no correlation with dimensions related to the hospital facility (ie, cleanliness and quietness of the hospital environment) or with responsiveness of care (eg, receiving help with the call button or getting to the restroom), neither of which were related to portal use.

Additional support for the role of patient portals comes from analyses showing that patient satisfaction was correlated with the provisioning of tablet computers in the hospital environment. As inpatient portals are commonly offered to patients through hospital-provided tablet computers, tablet access may influence patient satisfaction. However, prior work has shown that tablet access alone was not associated with higher scores on nurse communication or care transitions,²⁰ suggesting that functions of the portal beyond tablet access may have contributed to the associations of these dimensions with patient satisfaction.

As patient satisfaction measures are increasingly being incorporated in studies of patient portal use, few studies have utilized CG-CAHPS or HCAHPS surveys as a generalizable approach to measure the correlation between patient portals and patient satisfaction. Previous studies have primarily utilized a subset of CAHPS questions, and to our knowledge, their results have been both limited and variable.^21-24 For example, in one study that used only CAHPS survey questions related to communication, higher self-reported outpatient portal use was correlated with high patient satisfaction and the belief that the portal improved communication.²² In another study, use of an outpatient portal in the inpatient environment was found to have minimal association with satisfaction when evaluated with partial responses to the HCAHPS survey questions regarding self-management, medications, and overall hospital rating.²³

Our study contributes to the existing literature by analyzing all CG-CAHPS and HCAHPS survey dimensions, which, as comprehensive measures, have consequences for health care organizations due to their impact on perceived quality of care, patient recruitment and retention, and reimbursement. Our analyses provide insight into how patient portals may positively affect patients’ experiences, both in ambulatory and hospital care settings. Future research should explore the causality of patient portal use on patient satisfaction scores, as well as identify the mediating factors of patient portals and their use on patient experience.

Limitations

Our study has several limitations. First, our primary study predictors focused on activated patient portal accounts and did not measure overall portal use or use of specific portal functions. Future extensions of this work should incorporate use measures to enable evaluation of their correlation with dimensions of satisfaction. Second, our study was focused on patients of a single hospital system, which limits our ability to generalize across organizations. Third, the functions offered on the portals at our AMC may vary from those offered at other medical centers. Although portal platforms used by other hospital systems may also be different, the wide implementation of Epic Systems’ portal platforms across US hospitals, as well as the similarity of functions among different portal platforms, supports the generalizability of our results. Fourth, our samples of encounters with and without activated portal accounts differ by notable demographic characteristics. Although we control for these differences in our adjusted model and follow standard risk adjustment procedures for estimating patient satisfaction scores, we acknowledge that there may be unobserved factors that cannot be included in our models. Finally, the potential exists for a simultaneity effect, in which patients who decided to open a portal account are those who are highly satisfied with the provider or hospital, and this could result in an upward bias in our findings. Our study, however, took measures in study design and our analytical models to mitigate this potential effect, giving us confidence in the findings that we report.

CONCLUSIONS

Patient portals are a promising tool that can promote patient-centered care by enabling patients to participate in their health care. Our study provides preliminary evidence that patient portal use is correlated with patients’ positive perceptions of their care experiences. Given the importance of patient satisfaction scores in the context of reimbursement and reputation, our results encourage further research to understand how health care providers and institutions can consider the opportunity to promote patient portal use as a means of improving patient satisfaction.

Acknowledgments

The authors would like to thank Inal Elbeyli, senior systems consultant at their institution.

Author Affiliations: The Center for the Advancement of Team Science, Analytics, and Systems Thinking (CATALYST) (NF, SRM, PJ, ASM), Department of Biomedical Informatics (NF, PJ), and Department of Family Medicine (ASM), College of Medicine, The Ohio State University, Columbus, OH; The Ohio State University Wexner Medical Center (SV), Columbus, OH.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (NF, SRM, SV, ASM); acquisition of data (NF, ASM); analysis and interpretation of data (NF, SRM, SV, PJ, ASM); drafting of the manuscript (NF, SRM, SV, PJ, ASM); critical revision of the manuscript for important intellectual content (NF, SRM, SV, PJ, ASM); statistical analysis (NF, ASM); provision of patients or study materials (NF, ASM); obtaining funding (NF, ASM); administrative, technical, or logistic support (NF, ASM); and supervision (ASM).

Address Correspondence to: Naleef Fareed, PhD, CATALYST, College of Medicine, The Ohio State University, 460 Medical Center Dr, Ste 502, Columbus, OH 43210. Email: Naleef.Fareed@osumc.edu.

REFERENCES

1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. The National Academies Press; 2001.

2. Davis K, Schoenbaum SC, Audet AM. A 2020 vision of patient-centered primary care. J Gen Intern Med. 2005;20(10):953-957. doi:10.1111/j.1525-1497.2005.0178.x

3. Cleary PD. Evolving concepts of patient-centered care and the assessment of patient care experiences: optimism and opposition. J Health Polit Policy Law. 2016;41(4):675-696. doi:10.1215/03616878-3620881

4. Blumenthal D, Tavenner M. The “Meaningful Use” regulation for electronic health records. N Engl J Med. 2010;363(6):501-504. doi:10.1056/NEJMp1006114

5. Ahern DK, Woods SS, Lightowler MC, Finley SW, Houston TK. Promise of and potential for patient-facing technologies to enable Meaningful Use. Am J Prev Med. 2011;40(5 suppl 2):S162-S172. doi:10.1016/j.amepre.2011.01.005

6. Han HR, Gleason KT, Sun CA, et al. Using patient portals to improve patient outcomes: systematic review. JMIR Hum Factors. 2019;6(4):e15038. doi:10.2196/15038

7. Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to Meaningful Use: a systematic review. J Med Internet Res. 2015;17(2):e44. doi:10.2196/jmir.3171

8. Dendere R, Slade C, Burton-Jones A, Sullivan C, Staib A, Janda M. Patient portals facilitating engagement with inpatient electronic medical records: a systematic review. J Med Internet Res. 2019;21(4):e12779. doi:10.2196/12779

9. CAHPS Clinician & Group Survey. Agency for Healthcare Research and Quality. Updated August 2021. Accessed April 20, 2020. https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html

10. About the survey. Hospital Consumer Assessment of Healthcare Providers and Systems. Accessed April 20, 2020. https://hcahpsonline.org/#AboutTheSurvey

11. The CAHPS program. Agency for Healthcare Research and Quality. Updated October 2018. Accessed April 20, 2020. https://www.ahrq.gov/cahps/about-cahps/cahps-program/index.html

12. Patient satisfaction surveys. NEJM Catalyst. January 2018. Accessed April 20, 2020. https://catalyst.nejm.org/doi/full/10.1056/CAT.18.0288

13. Tsai TC, Orav EJ, Jha AK. Patient satisfaction and quality of surgical care in US hospitals. Ann Surg. 2015;261(1):2-8. doi:10.1097/SLA.0000000000000765

14. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients’ perception of hospital care in the United States. N Engl J Med. 2008;359(18):1921-1931. doi:10.1056/NEJMsa0804116

15. Kupfer JM, Bond EU. Patient satisfaction and patient-centered care: necessary but not equal. JAMA. 2012;308(2):139-140. doi:10.1001/jama.2012.7381

16. Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2009;67(1):27-37. doi:10.1177/1077558709341065

17. HCAHPS fact sheet. CMS. October 2019. Accessed April 20, 2020. https://hcahpsonline.org/globalassets/hcahps/facts/hcahps_fact_sheet_october_2019.pdf

18. Elliott MN, Lehrman WG, Goldstein E, Hambarsoomian K, Beckett MK, Giordano LA. Do hospitals rank differently on HCAHPS for different patient subgroups? Med Care Res Rev. 2009;67(1):56-73. doi:10.1177/1077558709339066

19. Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44(2, pt 1):501-518. doi:10.1111/j.1475-6773.2008.00914.x

20. Vink S, Fareed N, MacEwan SR, McAlearney AS. An exploration of the association between inpatient access to tablets and patient satisfaction with hospital care. Perspect Health Inf Manag. 2019;16(Fall):1i.

21. Nagykaldi Z, Aspy CB, Chou A, Mold JW. Impact of a wellness portal on the delivery of patient-centered preventive care. J Am Board Fam Med. 2012;25(2):158-167. doi:10.3122/jabfm.2012.02.110130

22. Neuner J, Fedders M, Caravella M, Bradford L, Schapira M. Meaningful Use and the patient portal: patient enrollment, use, and satisfaction with patient portals at a later-adopting center. Am J Med Qual. 2015;30(2):105-113. doi:10.1177/1062860614523488

23. Aljabri D, Dumitrascu A, Burton MC, et al. Patient portal adoption and use by hospitalized cancer patients: a retrospective study of its impact on adverse events, utilization, and patient satisfaction. BMC Med Inform Decis Mak. 2018;18(1):70. doi:10.1186/s12911-018-0644-4

24. Sorondo B, Allen A, Fathima S, Bayleran J, Sabbagh I. Patient portal as a tool for enhancing patient experience and improving quality of care in primary care practices. EGEMS (Wash DC). 2017;4(1):1262. doi:10.13063/2327-9214.1262