WHY GENERATIVE AI HALLUCINATES: WHAT CREATIVE PROFESSIONALS NEED TO UNDERSTAND
Doug Woodham | September 2026
Large language models can produce remarkably useful work while also inventing facts with striking confidence. These hallucinations arise from the same predictive machinery that makes generative AI so powerful. For creative professionals using these tools, the crucial question is when plausibility is enough, and when truth matters.
Generative AI can be brilliant one moment and confidently wrong the next. The same system that sharpens a paragraph, suggests an unexpected visual direction, or produces a useful first draft can invent a book, misattribute a quotation, or cite a source that does not exist.
This is not simply a defect waiting to be engineered away. Large language models generate language by predicting what plausibly comes next, based on statistical patterns learned from enormous bodies of human-made material. They are good at producing answers that sound right. Sounding right, however, is not the same as being right.
For creative professionals, that distinction matters because different kinds of work tolerate error very differently. Some tasks reward variation and surprise; others depend on factual accuracy. Understanding why generative AI can nevertheless be so valuable requires looking at hallucinations from several angles: why they happen, why they are so persuasive, why better models have not made them disappear, and why businesses have embraced the technology anyway. The question that ultimately matters is when generative AI’s willingness to invent is a strength, and when it becomes a liability.
What AI Hallucinations Are and Why They Happen
A hallucination is the mind’s quiet rebellion: the conjuring of voices, visions, or sensations without external cause. Computer scientists borrowed the term for a different phenomenon, the moment when a language model confidently invents information.
The term entered mainstream discussion after the release of ChatGPT in late 2022, when millions of people discovered that polished, authoritative-sounding answers could be built on information that simply did not exist. What made these failures so unsettling was not simply that the systems were wrong; it was the fluency and confidence with which the errors arrived.
Large language models hallucinate because they generate language through prediction. Given a prompt, the system repeatedly estimates what is most likely to come next, drawing on patterns learned from vast amounts of text. As explained in How Generative AI Works: What Creative Professionals Need to Understand, this is the basic mechanism behind large language models. Their output reflects probability, not independent verification.
That distinction is easy to miss because probability and truth often overlap. A model trained on enormous amounts of text will frequently produce the correct answer because it fits the patterns the system has learned. But when the evidence is weak or absent, the same machinery can generate a plausible answer anyway. A nonexistent book title, for example, can sound exactly like one an author might have written. The model is doing what it was built to do: produce language that fits the sequence. It does not stop to ask whether the book actually exists. Experienced users have learned to treat their output much like the claims of a confident stranger at a bar: engaging, occasionally insightful, but often wrong.
Fluency is therefore the model’s comparative advantage; verification is not. Hallucinations do not sit apart from the system’s strengths. They arise from the same generative flexibility that allows a model to combine learned patterns into expressions it has never encountered in precisely that form. That capacity is what makes invention possible. It is also what allows the model to invent when the user expected it to retrieve a fact.
Why Fluent Errors Are So Easy to Trust
The danger becomes clearest when fluency is mistaken for evidence. In June 2025, the Chicago Sun-Times published its annual summer reading list. Fifteen well-known authors were featured, including Isabel Allende and Andy Weir. Ten of the recommended books, however, did not exist. Titles such as Tidewater Dreams, described as Allende’s foray into climate fiction, and The Last Algorithm, attributed to Weir, were pure invention, complete with plausible plot summaries and fabricated blurbs. The bylined author had used an AI system to draft the piece, and no one appears to have verified the results before publication. The prose was confident and convincing, but the books were fictional. That they made it into print was a failure not only of technology but of editorial judgment. The paper blamed a third-party content provider before removing the article.
The problem is not confined to journalism. A study titled Large Legal Fictions examined how large language models answer questions about American case law. Testing four widely used models across more than 800,000 legal queries, the authors found that even the best-performing model hallucinated at least 58 percent of the time. The systems routinely misstated basic facts about cases and legal authorities. As lawyers increasingly use AI to save time and reduce costs, a fabricated precedent can find its way into a memo, brief, or court filing with consequences far beyond an imaginary summer reading list.
That is what makes these errors especially dangerous. Nonsense invites skepticism. A polished fabrication invites trust. The same cues people normally use to judge competence, including specificity and orderly reasoning, can all be present when the underlying claim is false.
Why Better AI Models Do Not Eliminate Hallucinations
Developers have poured vast resources into reducing hallucinations. ChatGPT, for instance, has used reinforcement learning from human feedback, in which people rank model responses to steer the system toward more useful outputs. Developers have also introduced retrieval tools that allow models to consult external sources and ground answers in information that can be checked. Both approaches can improve accuracy, yet neither changes the underlying fact that language models generate answers probabilistically rather than independently verify what is true. Indeed, the New York Times reported in 2025 that independent researchers found rising hallucination rates in some newer reasoning models.
Model providers tend to present hallucinations as a technical limitation that will diminish with better models, more data, and stronger safeguards. That may prove partly true. But admitting that an expensive system can confidently fabricate information is hardly reassuring to enterprise customers. The problem matters most where factual accuracy has high economic value and mistakes are costly. In finance and medicine, a plausible-sounding error can have serious consequences. These are also among the markets AI companies most want to serve.
Other fields are more forgiving. In advertising, entertainment, and some forms of journalism, plausibility may be enough. A weak slogan or unusable draft can simply be discarded. In those settings, generative AI can still create value even when some of its output is wrong.
Why Other Forms of AI Do Not Hallucinate in the Same Way
Large language models have become the public face of artificial intelligence, but they are only one branch of it. Other systems are built for very different purposes, often in settings where being measurably right matters much more.
Consider drug discovery. Scientists increasingly use AI to sift through enormous molecular datasets and identify promising compounds earlier in the process. Molecules can be represented as graphs, with atoms as nodes and chemical bonds as edges, allowing graph neural networks to learn relationships within those structures. These systems can then predict properties such as toxicity and assess how compounds may interact with biological targets. Their performance is ultimately judged against laboratory results, not by whether their output sounds persuasive. In this domain, structural accuracy matters more than linguistic fluency.
Agriculture offers another illustration. As William Boston reported in the Wall Street Journal, some farmers are experimenting with tractors equipped with cameras, sensors, and AI that can steer themselves while deciding where to apply fertilizer or control weeds. Other systems combine aerial imagery with field data to help farmers apply treatments more precisely. The value of these systems lies in seeing conditions across thousands of acres and responding efficiently, not in sounding human.
These systems make mistakes too. A molecular model can make a bad prediction, and an agricultural system can misread conditions in a field. But in both cases, the error can be tested against an external reality.
That is the important distinction. Large language models produce language in forms people instinctively associate with intelligence and competence. When they fail, the error can arrive wrapped in the very signals that normally inspire trust. Other forms of AI are built chiefly to extend human perception and analysis into domains too complex or subtle to navigate unaided. The story of artificial intelligence is therefore not only about machines that seem human. It is also about machines that allow humans to see and measure the world more clearly.
Why Businesses Invest in Generative AI Despite Hallucinations
Why, then, have some of the world’s largest technology companies invested billions in systems known to fabricate information? That is one of the central paradoxes of the generative AI boom. Large language models are fast, fluent, and often useful, yet they can produce confident falsehoods. Rather than treating that weakness as disqualifying, companies have embedded the technology across a growing range of products.
Part of the answer lies in how differently investors have treated large language models from conventional software. Venture capital traditionally favored products that could be tested cheaply before large sums were committed. Large language models reversed that pattern. Companies such as OpenAI and Anthropic raised and spent billions on computing infrastructure and training while the technology was still improving rapidly and its commercial uses were still being discovered.
The wager was that these systems were not simply products but platforms. A single model could perform a wide range of tasks through the same interface, suggesting a foundational layer of a new computing cycle. In a race of that kind, scale mattered more than perfection. Reliability could improve later. More important, perfect accuracy is not necessary for every economically valuable task. A creative director asking for campaign ideas does not need every suggestion to be usable. When output is cheap to produce and easy to evaluate, even an imperfect system can save substantial amounts of human effort.
That helps explain why hallucinations have not stopped adoption. Generative AI can produce large volumes of usable output at very low marginal cost. Much of it may be discarded or revised. The economics can still work if what remains is valuable enough. For businesses, the relevant question is not whether the system is perfectly reliable, but whether the value of its useful output exceeds the cost of generating and checking it.
That calculation varies sharply by task. A fabricated legal precedent may be disastrous; a bad advertising slogan can simply be thrown away. The uneven cost of being wrong helps explain both the commercial appeal of generative AI and why its impact on creative work is likely to vary so much.
When Plausibility Is Enough, and When Truth Matters
Creative work makes the trade-off unusually clear. A novelist can invent freely; a journalist cannot invent a source. What counts as a failure depends on the task.
Generative AI is most useful where variation and experimentation have value and a knowledgeable person can judge the result. The calculation changes when factual authority is part of the product. A nonfiction writer checking a historical claim or an editor verifying a quotation cannot treat plausible invention as harmless variation. The job is not to choose among possibilities, but to establish what is true.
Hallucination is therefore a useful clue to where generative AI fits. It is most valuable when asked to generate possibilities that a knowledgeable person can assess, and most hazardous when asked to supply facts the user cannot independently verify. That is the practical dividing line: know when the machine is being asked to invent, and when it is being asked to know.
About Doug Woodham
Doug Woodham is an economist, former Partner at McKinsey & Company, former President of Christie’s Americas, and the author of two books about art and creative markets. At McKinsey, he helped co-found the firm’s Business Technology Office and advised companies on strategy, technology, and organizational performance. He holds a Ph.D. in economics from the University of Michigan, where his training included econometrics and the study of how innovation reshapes firms and markets. He writes about how artificial intelligence is changing the economics of creative work, drawing on his experience in economics, technology, business strategy, and creative industries.
