Comparison of AI-assisted and human-produced podcasts derived from cochrane plain language summaries: Protocol for a randomized noninferiority trial (Health Information Effectiveness Trials-2)
Loading...
Files
Published Version
Published Version
Date
2026-05-19
Authors
O'Byrne, Isabel
Pope, Johanna
Byrne, Paula
Dahly, Darren
Judge, Conor
O'Donnell, Martin
Krewer, Finn
Li, Mengqi
Saif-Ur-Rahman, K. M.
Conway, Tom
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier Inc.
Published Version
Abstract
Background and objectives Podcasts can make health evidence easier to follow, but it is unclear whether artificial intelligence (AI)-assisted production can match human production when both use the same audio format. This trial will evaluate whether a vetted AI workflow can match human communicators on comprehension, quality, safety, accessibility, and trust when both deliver podcasts derived from the same evidence base. Methods We will run a randomized, two-arm, noninferiority trial comparing AI-assisted podcasts with human-produced podcasts. Adults (≥18 years; English-proficient) will be recruited from the general public via Prolific, an online research participant recruitment platform, and randomly allocated 1:1 to listen to three short episodes (6–8 minutes each) based on the same Cochrane Plain Language Summaries. The AI arm uses Wondercraft AI in a human-in-the-loop workflow; the human arm features experienced communicators working to an identical brief. In both arms, content is limited to the plain language summary, with authorship masked for participants and expert raters. The primary outcome is comprehension, measured by a 10-item test per episode, with the primary analysis using the participant-level mean score across the three episodes, aligned with the “Understanding” dimension of the Quality, Understanding, Expression, Safety, and Trust framework. Secondary outcomes include format accessibility (listenability), quality of information, perceived trust, and safety. Noninferiority margins are prespecified; for comprehension, the margin is 1 point on the 10-item scale. If noninferiority is shown, we will also assess superiority. We plan to recruit 458 participants. Differences between arms will be estimated using appropriate repeated-measures models, with two-sided 95% confidence intervals. Results This is a study protocol; results will be reported on completion of the trial.
Description
Keywords
Artificial intelligence , Cochrane reviews , Health communication , Noninferiority trial , Plain language summaries , Podcasts , QUEST framework , Systematic reviews , [CoMHOffice] , [Medicine]
Citation
O'Byrne, I, Pope, J, Byrne, P, Dahly, D, Judge, C, O'Donnell, M, Krewer, F, Li, M, Saif-Ur-Rahman, K M, Conway, T, Thomas, J, Sheils, G, Burke, N, O'Byrne, M, O'Byrne, A & Devane, D 2026, 'Comparison of AI-assisted and human-produced podcasts derived from cochrane plain language summaries: Protocol for a randomized noninferiority trial (Health Information Effectiveness Trials-2)', Journal of Clinical Epidemiology, vol. 195, 112272, pp. 1-11. https://doi.org/10.1016/j.jclinepi.2026.112272