agent-vision-toolkit: Give Your Text-Only Model Eyes

There's a wall every text-only coding agent hits: it can't see images.

Tell your DeepSeek-powered agent to "build this page from the mockup," and it stares blankly. Hand it an error screenshot, and it says "I can't process images." Your options used to be: switch to a pricier multimodal model, or give up on vision tasks.

agent-vision-toolkit (1.2k stars, MIT) offers a third path, and its core insight is worth stealing: an agent's vision capability doesn't have to live in the model — it can live in the harness.

The idea

Don't expect the model to "understand" an image. Use tools to translate the image into text the model can understand, then feed it in.

Two components:

1. Vision tool CLIs — image Q&A, long-screenshot OCR, frontend UI restoration, GUI automation, pixel diff, foreground extraction;
2. A vision-skills skill — teaches the agent when to reach for which tool.

Any agent that can invoke a shell (Codex, Claude Code, Pi, OpenCode) gains vision the moment it installs these tools. Your DeepSeek is still your DeepSeek — it just has eyes now.

The tools that matter

• Long-screenshot OCR — extract text from those tall error/log/chat screenshots that text-only models choke on;
• Frontend UI restoration — mockup in, code out;
• GUI automation — see the screen, locate elements, click, read the result;
• Image Q&A — "what's in this image," "what's different between these two."

The real payoff is cost

Strong multimodal models are expensive, and for private deployments the VRAM cost is a hard constraint. This toolkit lets you keep the cheap text-only model while adding vision. It turns "seeing" from a model capability into a tool capability, which opens up the whole model-selection space — you no longer have to buy a bigger model for a single vision requirement.

The honest caveat

Tools convert images into structured text; the model still reasons over text, not pixels. For structured vision — extracting text, restoring layout, locating elements — it's enough. For genuine visual aesthetics or complex image reasoning, use a native multimodal model. It's also been verified end-to-end in real Codex + DeepSeek sessions, and in Claude Code, Pi, Oh My Pi, and OpenCode.

I've localized the README and core docs to Chinese: https://github.com/yangshun2005/agent-vision-toolkit-cn

If you find this project useful, a star on the original repo supports the author's ongoing maintenance.