Generative AI’s performance on emergency medicine boards questions: an observational study

Simple item page
dc.contributor.authorKajitani, Stenen
dc.contributor.authorPastrak, Milaen
dc.contributor.authorGoodings, Anthonyen
dc.contributor.authorNguyen, Audreyen
dc.contributor.authorDrewek, Austinen
dc.contributor.authorLafree, Andrewen
dc.contributor.authorMurphy, Adrianen
dc.contributor.editorMehta, Shobhaen
dc.contributor.editorCronin, Pádraigen
dc.date.accessioned2025-10-28T13:55:06Z
dc.date.available2025-10-28T13:55:06Z
dc.date.issued2025en
dc.description.abstractBackground: The evolving field of medicine has introduced ChatGPT as a potential assistive platform, though its use in medical board exam preparation remains debated [1-2]. This study aimed to evaluate the performance of a custom-modified version of ChatGPT-4, tailored with emergency medicine board exam preparatory materials (Anki deck), compared to its default version and previous iteration (3.5) [3]. The goal was to assess the accuracy of ChatGPT-4 answering board- style questions and its suitability as a tool for medical education. Methods: A comparative analysis was conducted using a random selection of 598 questions from the Rosh In-Training Exam Question Bank [4]. The subjects of the study included three versions of ChatGPT: the Default, a Custom, and ChatGPT-3.5. Accuracy, response length, medical discipline subgroups, and underlying causes of error were analyzed. Results: Custom ChatGPT-4 did not significantly improve accuracy over Default (p>0.05), but both significantly outperformed ChatGPT-3.5 (p<en
dc.description.statusNot peer revieweden
dc.description.versionPublished Versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.citationKajitani, S. Pastrak, M., Goodings, A., Nguyen, A., Drewek, A., Lafree, A. and Murphy, A. (2025) 'Generative AI’s performance on emergency medicine boards questions: an observational study', UCC Student Medical Journal, 5, p. 113. https://doi.org/10.33178/SMJ.2025.1.39en
dc.identifier.doi10.33178/SMJ.2025.1.39en
dc.identifier.endpage113en
dc.identifier.issn2737-7237
dc.identifier.journalabbrevUCC SMJ
dc.identifier.journaltitleUCC Student Medical Journalen
dc.identifier.startpage113en
dc.identifier.urihttps://hdl.handle.net/10468/18116
dc.identifier.volume5
dc.language.isoenen
dc.publisherUCC Medical Research and Technology Societyen
dc.rights© 2025, the Author(s). This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.en
dc.rights.urihttps://creativecommons.org/licenses/by-nc/4.0
dc.sourceBatch uploaden
dc.subjectGenerative AIen
dc.subjectEmergency medicine boards questionsen
dc.titleGenerative AI’s performance on emergency medicine boards questions: an observational studyen
dc.typeConference itemen
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Generative+AI's+performance+on+emergency+medicine+boards+questions_+observational+study.pdf
Size:
124.13 KB
Format:
Adobe Portable Document Format
Description:
Published Version
Download
Collections
UCC Student Medical Journal Vol. 5 (2025)