Ask ChatGPT, Claude, or Gemini a question about something that never happened, and you’ll often get a detailed, confident answer anyway. The model won’t say “I don’t know” — it will fabricate names, dates, and facts that sound plausible but are completely wrong.
This isn’t a bug. It’s how large language models work. Understanding why helps you use AI tools more effectively and catch mistakes before they matter.
Language Models Predict, They Don’t Know
When you ask an AI model a question, it doesn’t search a database or check its facts. It predicts the most statistically likely next word, then the next, based on patterns in its training data.
If you ask about a real event the model saw thousands of times during training — like the Apollo 11 moon landing — it predicts accurate details because those patterns are strong. But if you ask about something obscure, recent, or fictional, the model still predicts words. It just predicts based on weaker patterns, adjacent topics, or pure statistical noise.
The result looks like knowledge. The model uses proper nouns, dates, confident phrasing. But it’s generating text that sounds like an answer, not retrieving information it actually has.
Why Models Don’t Just Say “I Don’t Know”
You’d think a well-designed AI would recognize uncertainty and admit it. Some do, sometimes. But most models are trained to be helpful and produce complete responses. Saying “I don’t know” too often makes them seem less useful.
There’s also a technical problem: the model doesn’t have a reliable way to measure whether it “knows” something. It can estimate confidence in its next-word predictions, but that’s not the same as knowing whether a fact is true. A model can be very confident in a completely false statement if the false statement follows common linguistic patterns.
OpenAI, Anthropic, and Google have all added refusal training — teaching models to decline some requests — but distinguishing “I don’t have this information” from “I shouldn’t answer this” is still inconsistent. Models err toward answering.
What This Looks Like in Practice
Ask Claude to summarize a book that doesn’t exist, and it will often generate a plausible plot, themes, and even fake critical reception. Ask ChatGPT about a nonexistent software feature, and it may give you step-by-step instructions that sound real but don’t work.
The more niche your question, the higher the risk. AI models perform well on common knowledge — capital cities, famous historical events, widely documented software — because the training data is dense and repetitive. But ask about a small-town newspaper article, an obscure API method, or a person who isn’t famous, and the model starts guessing.
This also explains why models sometimes contradict themselves. If you ask the same question two ways, the model might predict different answers based on how you phrase it, because it’s not retrieving a stored fact — it’s generating text each time.
How to Work Around It
You can’t fix this behavior, but you can reduce the damage:
- Verify anything important. Treat AI output as a draft that needs fact-checking, especially for dates, names, quotes, and technical details.
- Ask the model to cite sources. Models with search features (ChatGPT Search, Gemini with Search, Perplexity) can pull from the web and link to pages. This doesn’t guarantee accuracy, but it gives you something to verify.
- Test with questions you already know the answer to. If you’re using AI for research in a domain you understand, throw in a question where you know the right answer. If the model gets it wrong, you know to be skeptical of everything else.
- Use explicit uncertainty prompts. Adding “If you’re not sure, say so” or “Only answer if you have high confidence” sometimes helps, though it’s not foolproof.
Some people assume that paying for a premium model — ChatGPT Plus, Claude Pro, Gemini Advanced — solves this. It doesn’t. Larger, more expensive models are better at many tasks, but they still predict text the same way. They’re often more confident in their fabrications because they generate more fluent, detailed responses.
The Bottom Line
AI models don’t know what they don’t know because they don’t “know” anything in the way humans do. They predict language patterns. When the pattern is strong and accurate, the output is useful. When it’s weak or noisy, the output is convincing nonsense.
That doesn’t make these tools useless — it makes them tools that require judgment. Treat AI like a very fast, very confident intern who sometimes makes things up. Check the work.
Want one useful AI idea in your inbox every day? Subscribe to the One Two Three AI newsletter and keep up with what actually matters.