Stylometric comparisons of human versus AI-generated creative writing

Thumbnail Image
Files
Date
2024
Authors
O'Sullivan, James
Journal Title
Journal ISSN
Volume Title
Publisher
Published Version
Research Projects
Organizational Units
Journal Issue
Abstract
This study employs stylometry to compare the creative writing styles of humans and large language models (LLMs) such as GPT-3.5, GPT-4, and Llama 70b. Using a dataset of short stories generated in response to predefined narrative prompts, the analysis focuses on stylistic differences, measured through Burrows’ Delta, a widely used metric in computational literary studies. Clustering techniques reveals clear distinctions between human and AI-generated texts, findings which suggest that LLMs, while increasingly sophisticated in mimicking human creativity, remain distinguishable as machine-generated texts.
Description
Keywords
Stylometry , Creative writing styles , Large language models (LLMs) , GPT-3.5 , GPT-4 , Llama 70b , Stylistic differences , Burrows’ Delta
Citation
O'Sullivan, J. (2024) 'Stylometric comparisons of human versus AI-generated creative writing', pp. 1-7.
Link to publisher’s version
URI
https://hdl.handle.net/10468/16884
Collections
Digital Arts and Humanities - Journal Articles
Copyright
© 2024, James O'Sullivan.