Misalignment Between EHR Penicillin Allergy Flags, Allergist Assessments Signals Need for Improved Documentation

Details documented in the allergy module of the electronic health record (AM-EHR), as well as high-risk flags related to penicillin allergy labels (PALs), may be unreliable, according to a research letter published in JAMA Network Open.1

Assessing Agreement Between EHR Documentation and Allergist Findings

The researchers emphasized that evaluating PALs is a public health priority because these labels are common, frequently inaccurate, and associated with preventable adverse outcomes driven by inappropriate antibiotic prescribing.2 Current guidelines recommend classifying patients with PALs as high or low risk for true allergy, with patients with low-risk reaction histories able to safely undergo direct penicillin challenges and receive other β-lactam antibiotics without prior testing.

Although previous research has shown that reaction descriptions in the AM-EHR are often incomplete and may misclassify intolerances as allergies, these records are still frequently used to assess reaction characteristics and associated risk.1 Given this, the researchers evaluated agreement between AM-EHR penicillin reaction descriptions and allergist-obtained histories, high-risk reactions identified in the AM-EHR and by allergists, and AM-EHR high-risk flags and penicillin allergy testing results.

EHR penicillin allergy flags do not often match allergist assessments, highlighting the need for improved evaluation processes. | Image Credit: Zerbor - stock.adobe.com

This retrospective cohort study included patients seen for consultative visits at a US university-affiliated allergy and immunology clinic between August 1, 2023, and April 30, 2024, who had an allergist-documented PAL. Data were extracted from the Epic AM-EHR as recorded before the visit. The researchers assessed agreement between primary reaction symptoms documented by allergists and those recorded in the AM-EHR; for patients with multiple reaction symptoms, the most severe was used for comparison.

They also compared allergists’ risk assessments with the high-, moderate-, and low-risk flags in the AM-EHR. The researchers performed descriptive analyses, with agreement calculated using percentage agreement and the Cohen κ; a 2-sided P < .05 was considered statistically significant.

Demonstrating Inaccuracies in High-Risk Penicillin Allergy Flags

The study population consisted of 230 patients with a median (IQR) age of 37 (30-60) years, the majority of whom were female (83.0%; n = 191) and had PALs documented in the AM-EHR (97.4%; n = 224). Among the patients with documented PALs, 50.9% had primary reaction symptoms in the AM-EHR that aligned with allergists’ assessments. The researchers highlighted higher agreement when identifying cutaneous symptoms. For example, agreement for any rash was 74.7% (95% CI, 69.1%-80.4%), with a Cohen κ of 0.39 (95% CI, 0.25-0.52).

In contrast, agreement between AM-EHR reaction risk and allergists’ assessments was 41.7% (95% CI, 35.3%-48.2%), with a Cohen κ of 0.01 (95% CI, –0.04 to 0.06). Cross-tabulated data for additional high-risk symptoms, such as anaphylaxis, demonstrated discordance between allergist and AM-EHR documentation. Specifically, while 107 patients were flagged as high risk in the AM-EHR, only 1 was classified as such by an allergist.

Additionally, 119 patients underwent penicillin allergy testing during the study period. One patient tested positive but had no high-risk flags in the AM-EHR. Meanwhile, 60 of the 118 patients who tested negative had high-risk flags documented.

Advancing Penicillin Allergy Documentation and Clinical Decision-Making Support

The researchers acknowledged several limitations, including the single-center, retrospective design and the predominantly White and female study population. Conversely, strengths included the use of allergist assessments and testing, as well as the study’s contribution to existing evidence that AM-EHR allergy documentation is often incomplete, inconsistent with patient-reported history, and prone to misclassifying intolerances as allergies.

The researchers concluded by expressing confidence in their findings and identifying areas for further research.

“Our findings suggest that details in the AM-EHR and high-risk flags regarding PALs are unreliable,” the authors concluded. “…These findings reinforce the need for implementation studies, quality initiatives, education, and cross-specialty guidelines to improve AM-EHR documentation and clinical decision support for PALs.”

References

1. Novotny S, Elmansy L, Kwah J, et al. High-risk penicillin reaction flags in the medical record. JAMA Netw Open. 2025;8(12):e2549081. doi:10.1001/jamanetworkopen.2025.49081
2. Khan DA, Banerji A, Blumenthal KG, et al. Drug allergy: a 2022 practice parameter update. J Allergy Clin Immunol. 2022;150(6):1333-1393. doi:10.1016/j.jaci.2022.08.028