If you spend any time on social media or AI forums, you’ve probably seen the complaints: “ChatGPT used to be so much better,” or “Claude was smarter six months ago.” The perception that AI chatbots are degrading over time has become surprisingly widespread.
But is it actually true? Are the models getting worse, or is something else going on?
What the Research Shows
A Stanford study from 2023 found that GPT-4’s performance on certain tasks did change between March and June of that year—sometimes better, sometimes worse, depending on the task. The researchers noted variations in code generation accuracy and math problem solving.
However, this doesn’t mean the models are “getting dumber.” What’s happening is more nuanced. AI companies regularly update their models with new training data, safety filters, and optimizations. OpenAI, Anthropic, and Google all deploy updated versions of their flagship models without always announcing every change.
These updates can improve some capabilities while inadvertently affecting others. A change that makes the model safer or faster might also make it more cautious or less verbose in certain situations.
The Real Culprits Behind the Perception
Several factors contribute to the feeling that AI is getting worse, even when the models themselves are improving:
- Rising expectations: When you first used ChatGPT, everything it did seemed magical. Now you’re comparing it to your memory of that initial amazement, not its actual past performance.
- Harder tasks: As you get comfortable with AI, you naturally give it more difficult problems. You’re no longer asking it to write basic emails—you’re asking it to debug complex code or analyze nuanced arguments.
- Confirmation bias: Once you believe the model is worse, you notice every mistake and forget the successes. Your brain is actively looking for evidence to confirm your theory.
- Different model versions: When you use ChatGPT, you might be routed to different model versions depending on load. GPT-4 and GPT-4 Turbo behave differently, and you might not always know which one you’re using.
What We Found When We Tested
We ran the same set of 50 prompts through ChatGPT and Claude that we’d saved from six months ago. The prompts covered writing, analysis, math, coding, and creative tasks.
The results? Mixed, but not in the way you’d expect. Both models showed improvements in following complex instructions and maintaining context over longer conversations. ChatGPT was noticeably faster. Claude showed better nuance in ethical reasoning tasks.
However, both models were slightly more conservative in certain areas—more likely to add disclaimers, less willing to engage with hypothetical scenarios that could be considered sensitive. This isn’t degradation; it’s the result of ongoing safety tuning.
The biggest difference? Our own prompting had improved dramatically. Looking back at our six-month-old prompts, many were vague or poorly structured. When we rewrote them with clearer instructions, both current models outperformed their earlier versions significantly.
How to Get Consistent Results
If you’re frustrated with inconsistent AI performance, try these approaches:
- Save your successful prompts: When a prompt works well, save it exactly as written. This gives you a baseline to return to.
- Specify the model version: In ChatGPT, you can choose between GPT-4 and GPT-4 Turbo. In Claude, you can select between Claude 3.5 Sonnet, Opus, or Haiku. Different versions excel at different tasks.
- Use custom instructions or system prompts: Set consistent behavior parameters so the model knows your preferences from the start of every conversation.
- Test objectively: If you think performance has declined, run the same prompt through the model multiple times and compare results with your saved outputs from months ago.
The bottom line: AI models aren’t getting dumber. They’re evolving in complex ways, your expectations are rising, and your tasks are getting harder. That combination can feel like decline, but the data tells a different story.
Want to stay sharp on what’s really happening in AI? Subscribe to the One Two Three AI newsletter—one useful AI idea delivered to your inbox every day, no hype included.