Keywords
ChatGPT, generative AI, automation bias
Abstract
This paper adopts an automation bias lens to cast doubt on the growing claims that GenAI is a transformational tool for legal industry. In this context, automation bias refers to the well-known psychological phenomenon in which human decision-makers unwittingly defer to automated processes, flowing from overreliance on the accuracy of the automation. Even well-meaning individuals aiming to keep a “human-in-the-loop” of any automated decision can fall prey to this well-documented phenomenon, trusting the technology more than it deserves to be trusted. Champions for current GenAI technologies often do acknowledge limitations within the existing landscape of tools, like GenAI’s tendency to hallucinate, but argue that the technology can still be useful with robust human supervision, and that, regardless, forthcoming improved versions will soon correct any lingering issues. By contrast, I argue that this viewpoint is overly optimistic: it is far more likely that unthinking use of today’s GenAI systems will muddy the waters of human expertise, which has serious consequences in the legal realm. Using GenAI to produce legal knowledge risks the creation and propagation of sourceless information, undermining law’s key social systematizing function.
Recommended Citation
Katie Szilagyi, “Regenerating Justice: ChatGPT and the Legal Minefield of Generative AI” (2025) 22:2 CJLT 109.
Included in
Computer Law Commons, Intellectual Property Law Commons, Internet Law Commons, Privacy Law Commons, Science and Technology Law Commons