Swaminathan Kandaswamy 1, Julia K W Yarahuan 1 2, Elizabeth A Dobler 3 4, Matthew J Molloy 5 6, Lindsey A Knake 7 8, Sean M Hernandez 9 10, Anne A Fallon 11, Lauren M Hess 12 13, Allison B McCoy 14, Regine M Fortunov 12 15, Eric S Kirkendall 9 16, Naveen Muthu 1 2, Evan W Orenstein 1 2, Adam C Dziorny 11 17, Juan D Chaparro 18 19
Abstract
Objective: To assess the prevalence of recommended design elements in implemented electronic health record (EHR) interruptive alerts across pediatric care settings.
Materials and methods: We conducted a 3-phase mixed-methods cross-sectional study. Phase 1 involved developing a codebook for alert content classification. Phase 2 identified the most frequently interruptive alerts at participating sites. Phase 3 applied the codebook to classify alerts. Inter-rater reliability (IRR) for the codebook and descriptive statistics for alert design contents were reported.
Results: We classified alert content on design elements such as the rationale for the alert’s appearance, the hazard of ignoring it, directive versus informational content, administrative purpose, and whether it aligned with one of the Institute of Medicine’s (IOM) domains of healthcare quality. Most design elements achieved an IRR above 0.7, with the exceptions for identifying directive content outside of an alert (IRR 0.58) and whether an alert was for administrative purposes only (IRR 0.36). IRR was poor for all IOM domains except equity. Institutions varied widely in the number of unique alerts and their designs. 78% of alerts stated their purpose, over half were directive, and 13% were informational. Only 2%-20% of alerts explained the consequences of inaction.
Discussion: This study raises important questions about the optimal balance of alert functions and desirable features of alert representation.
Conclusion: Our study provides the first multi-center analysis of EHR alert design elements in pediatric care settings, revealing substantial variation in content and design. These findings underline the need for future research to experimentally explore EHR alert design best practices to improve efficiency and effectiveness.
Links
Affiliations
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA 30322, United States.
- Division of Hospital Medicine, Children’s Healthcare of Atlanta, Atlanta, GA 30329, United States.
- Department of Clinical Informatics, Ann and Robert H Lurie Children’s Hospital of Chicago, Chicago, IL 60611, United States.
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, United States.
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH 45221, United States.
- Division of Hospital Medicine and Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229, United States.
- Department of Pediatrics, Division of Neonatology, University of Iowa, Iowa City, IA 52242, United States.
- Stead Family Children’s Hospital, Iowa City, IA 52242, United States.
- Department of General Internal Medicine, Wake Forest School of Medicine, Winston-Salem, NC 27101, United States.
- Primary Care, Miami Veteran’s Affairs, Miami, FL 33125, United States.
- Division of Pediatric Hospital Medicine, Department of Pediatrics, University of Rochester Medical Center, Rochester, NY 14642, United States.
- Department of Pediatrics, Baylor College of Medicine, Houston, TX 77030, United States.
- Pediatric Hospital Medicine, Texas Children’s Hospital, Houston, TX 77030, United States.
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37232, United States.
- Division of Neonatology, Texas Children’s Hospital, Houston, TX 77030, United States.
- Department of Pediatrics, Wake Forest School of Medicine, Winston-Salem, NC 27101, United States.
- Division of Critical Care Medicine, Golisano Children’s Hospital at Strong, Rochester, NY 14642, United States.
- Division of Clinical Informatics, Nationwide Children’s Hospital, Columbus, OH 43205, United States.
- Department of Pediatrics, The Ohio State University College of Medicine, Columbus, OH 43210, United States.
