Villanova University study: AI fiction scores higher, but human labels win trust

Villanova University has published a new study in Judgment and Decision Making with a result plenty of writers probably won’t enjoy: in blind ratings, machine-written short stories scored higher than human-written ones on both absorbing and quality. The responses came from more than 1,600 readers.

ChatGPT Download

A follow-up test with more than 900 participants found that readers could identify the machine-written story at about the rate you’d expect from random guessing, though text generator experience were better at spotting it. Lead researcher Dr. Deena Skolnick Weisberg said one reason may be that simpler, easier-to-process prose helps these stories move more smoothly on the page.

That changed once labels were added. When a story was described as human-written, it got higher scores no matter who wrote it. That lines up with a 2024 Bynder survey in which 56% of readers picked the machine-written article when reading blind, and with separate 2026 findings suggesting that news produced by a person using software was seen as more trustworthy than copy generated entirely by a machine.

If you follow AI writing, this study is worth reading because it gets at the same issue now showing up in publishing disputes, including the Commonwealth Short Story Prize controversy over entries flagged as machine-generated.

You can read the study in Judgment and Decision Making.

Grammarly Download