Most people now know that ChatGPT, Claude, and Gemini can “see” images. You can upload a photo of a receipt, a diagram, a plant, or a screenshot, and the model will tell you what it is, explain it, or answer questions about it.
But here’s a question most users never consider: should you always upload the image, or are there times when describing it in words works better?
The answer isn’t obvious. Vision models are powerful, but they’re not always the fastest, cheapest, or most accurate option. Sometimes a careful text description gets you better results—and sometimes it’s the only option that works at all.
When Uploading the Image Wins
If the image contains details you can’t easily put into words, upload it. This includes:
- Charts, graphs, and data visualizations: AI can read axes, parse trends, and extract numbers from a bar chart or line graph faster than you can type them out.
- Screenshots with UI elements: If you’re troubleshooting software, debugging code, or asking for design feedback, the model needs to see buttons, labels, and layout.
- Handwriting, receipts, and forms: OCR (optical character recognition) built into vision models works well for printed and handwritten text, saving you transcription time.
- Photos of physical objects: Identifying plants, diagrams, broken appliances, or architectural details requires visual context a description can’t capture.
Uploading is also faster when the image itself is the task—like asking Claude to critique a design mockup or having Gemini extract text from a scanned document.
When Describing in Text Works Better
Vision processing costs more tokens than text. If you’re on a rate-limited free plan or paying per API call, a text description can be cheaper and faster. But cost isn’t the only reason to skip the upload.
Abstract or conceptual images: If you’re asking about “a photo of a sunset” or “a generic office space,” the model doesn’t need pixel data. A short description like “a corporate meeting room with a whiteboard and six chairs” works just as well and processes instantly.
Privacy-sensitive content: If the image contains faces, personal information, or proprietary data you don’t want uploaded to a cloud service, describe it instead. Most AI providers state they don’t train on user uploads, but describing keeps the data entirely off their servers.
When you want to control what the model focuses on: Vision models sometimes latch onto irrelevant details—a background object, a watermark, a shadow. If you describe only the relevant parts, you guide the model’s attention and avoid distractions.
When the image quality is poor: Blurry photos, low-resolution scans, and images with bad lighting often confuse vision models. If you can see what matters but the image is messy, typing it out may be clearer.
Hybrid Approach: Upload and Describe
The best results often come from doing both. Upload the image, then add a sentence or two telling the model what to focus on.
For example:
- “This is a photo of my garden. Can you identify the plant in the center with the yellow flowers?”
- “I’m attaching a screenshot of an error message. The relevant part is the red text in the middle.”
- “Here’s a chart from a report. I need you to extract the data for Q3 only.”
This combination works because it gives the model both the raw visual data and your intent. It reduces ambiguity and speeds up the response.
One More Thing: When Vision Isn’t Available
Not all AI models support vision, and not all versions of the same model do. OpenAI’s API, for instance, requires you to explicitly enable vision for GPT-4 models. Some lightweight or fine-tuned models strip out vision capabilities entirely to save compute.
If you’re using an older model, a custom deployment, or a third-party tool that wraps an AI API, check whether vision is even supported. If it’s not, description is your only option—and it works fine for most tasks.
The rule of thumb: if the image contains information you can’t easily or accurately describe in a sentence or two, upload it. If you can describe it clearly and quickly, consider skipping the upload. And when precision matters, do both.
Want one useful AI idea delivered to your inbox every day? Subscribe to the One Two Three AI newsletter and keep up with what actually works in AI.