Ask ChatGPT or Claude to explain quantum physics, and you’ll get a clear, detailed answer. Ask it to multiply 8,347 by 6,291, and there’s a decent chance it’ll get it wrong. This isn’t a bug—it’s a fundamental quirk of how large language models work.
Understanding why AI fails at tasks that seem trivial while succeeding at complex ones helps you use these tools more effectively and know when to trust their output.
LLMs Predict Words, Not Numbers
Large language models don’t actually calculate. They predict the next most likely token (roughly a word or part of a word) based on patterns in their training data. When you ask for 2 + 2, the model has seen that sequence thousands of times and reliably predicts “4” because that’s what always follows in the training data.
But ask for 8,347 × 6,291, and the model probably never encountered that exact multiplication problem during training. It’s forced to guess based on patterns—and patterns aren’t the same as computation. The model might produce something that looks like a plausible answer (a five-digit number, properly formatted), but the actual digits could be wrong.
This is why GPT-4, Claude 3.5 Sonnet, and Gemini can all explain how multiplication works, write Python code to perform it, or solve a word problem involving multiplication—but might fail when asked to do the raw arithmetic directly.
What Complex Problems AI Actually Handles Well
So why can AI explain calculus, debug code, or draft a legal memo? Because those tasks rely on patterns, context, and relationships between concepts—exactly what language models excel at.
When you ask an AI to explain the chain rule in calculus, it’s drawing on thousands of explanations it’s seen. It recognizes the structure of a good explanation: start with the concept, provide the formula, give an example, explain the intuition. Same with debugging code—the model has seen millions of code examples and error patterns. It can spot that a missing semicolon or incorrect indentation matches patterns it associates with common bugs.
Writing a business email? The model recognizes the tone, structure, and phrasing patterns of professional communication. These are all tasks where being “roughly right based on patterns” produces genuinely useful results.
The Workaround: Tools and Chain-of-Thought
Modern AI assistants increasingly solve the math problem by not doing math themselves. ChatGPT, Claude, and Gemini can now call external tools—essentially running Python code or using a calculator behind the scenes when they detect a computation request.
If you’re using an AI that supports code execution (like ChatGPT with Advanced Data Analysis enabled or Claude with analysis tools), you’ll see it write a small script to compute the answer rather than attempting to predict digits. This hybrid approach combines the model’s pattern-matching strength with actual computational tools.
Chain-of-thought prompting also helps. When you ask an AI to “show your work” or “break this down step by step,” you force it to generate intermediate steps. For some problems, this improves accuracy because each small step is more likely to match training patterns than jumping straight to a final answer.
When to Trust AI Outputs
Use this mental model: trust AI for tasks where approximate pattern-matching is valuable, and verify when precision matters.
- Trust more: drafting, brainstorming, explaining concepts, finding patterns in text, suggesting approaches, reformatting data
- Trust less: precise calculations, counting (words, items, characters), exact dates or citations, sequential logic that requires perfect accuracy at each step
- Always verify: anything you’ll rely on professionally, financial calculations, medical or legal information, factual claims with sources
The paradox isn’t really a paradox once you understand what these models are. They’re incredibly sophisticated pattern-matching systems, not calculators with a chat interface bolted on. That limitation also defines their strength—they find patterns and connections humans might miss, explain complex topics clearly, and handle ambiguous creative tasks that traditional software can’t touch.
Want to understand AI better and use it more effectively? Subscribe to the One Two Three AI newsletter and get one useful AI idea in your inbox every day—no hype, just practical insights you can actually use.