kaixinbaba/dsh-vision-recognizer

dsh-vision-recognizer

Keep DeepSeek as the conversation brain, attach images anyway, and switch the image-recognition provider any time from Settings → Plugins. A vision plugin for DeepSeek Harness.

It registers a new provider route (default vision-recognizer, shown as DeepSeek + 识图 in the model picker) that wraps the real DeepSeek adapter: it declares image input (so the attachment preflight and the read_image gate admit images) and, in the request stream, transcribes every attached image to text through the vision model you select, then delegates the text-only conversation to DeepSeek. DeepSeek still answers; recognition is an add-on.

attached image ──▶ vision-recognizer route ──▶ vision-model transcription (OCR + layout + detail)
                     │                          │
                     ▼                          ▼
              DeepSeek answers ◀── text-only conversation (image replaced by [图片转译] text)

Features

• One-click install: dsh plugin --profile web add dsh-vision-recognizer — no build scripts, no sharp approval (no native dependencies at all).
• Configure from Settings → Plugins → Vision: pick a provider, enter an API key, override model / endpoint / token cap / timeout / marker. Saved changes take effect immediately, no restart.
• 15+ providers, domestic and international: OpenAI, Anthropic Claude, Google Gemini, OpenRouter, Azure OpenAI, Ollama (local), plus Alibaba DashScope, QwenCloud (Intl), Zhipu GLM, Baidu Qianfan, iFlytek Spark, Moonshot Kimi, Tencent Hunyuan, Volcengine Doubao, SiliconFlow. Any OpenAI-compatible endpoint works via the custom provider.
• Two wire protocols: OpenAI-compatible (/chat/completions) and native Anthropic Messages — Claude works out of the box.
• No hangs: local/anonymous endpoints get a hard 20s timeout cap, HTTP 429 fails fast, failed endpoints cool down for 60s; without a key and without local Ollama it fails fast with actionable guidance.
• Fallback chain: after the primary model fails, each fallbackModels entry is tried in order (each may target a different vendor); only after all fail does the request fail, listing every attempt.
• Content-hash cache: the same image is transcribed at most once per process (in-process, capped at 200).
• Zero-config local path: autoLocalOllama (default on) probes http://localhost:11434 and prepends a running Ollama to the chain — images never leave your machine.

Quick start

dsh plugin --profile web add dsh-vision-recognizer

Slow npm registry? dsh plugin --profile web add dsh-vision-recognizer --registry=https://registry.npmmirror.com

Install from a local checkout (development):

dsh plugin --profile web add file:/path/to/dsh-vision-recognizer

Use the file: prefix (copies the package into node_modules). A bare add . or add link:… makes pnpm symlink the package, in which case the plugin’s schemastery dependency resolves from the source checkout and is not found — a general pnpm symlink-install gotcha, not a bug in the plugin.

Restart dsh web, then:

1. Pick DeepSeek + 识图 in the model selector;
2. Open Settings → Plugins → Vision, choose a provider, enter an API key, save;
3. Paste an image into any conversation → you should see the [图片转译] marker followed by a DeepSeek answer.

With no key and no local Ollama, a turn fails fast in a few seconds with guidance — that is the intended anti-hang behavior.

Supported providers

Model ids drift over time; the defaults are starting points — override Model in the settings UI. Key resolution order: key entered in the UI → the provider env var → $VISION_API_KEY / $DASHSCOPE_API_KEY.

Configuration storage

Config saved from the UI is written to $DSH_HOME/vision-recognizer.json and merged over the bundle defaults at startup. cordis.patch.yml only carries factory defaults; a user cordis.patch.yml override still works as the composition-time fallback.

⚠️ patch semantics: the bundle’s - insert: appends this row to the entry list. Writing a second - insert: with the same id in your own cordis.patch.yml would register the adapter twice (undefined behavior). To override individual keys, write a single top-level - id: dsh-vision-recognizer entry; better yet, use the Settings UI.

Implementation notes (for plugin authors)

Only stable rc.6 public interfaces are used:

• ctx.llm.registration(innerProvider).adapter — fetch the wrapped adapter;
• ctx.llm.registerAdapter([providerId], proxyAdapter) — register the new route;
• proxying resolveModel overrides inputModalities to ['text', 'image'];
• proxying stream transcribes image blocks ({ type: 'image', attachment }, bytes read via ctx.get('attachments').readImage(ref)), then yield* forwards the inner adapter’s stream;
• the settings UI rides the settings.plugins.tab slot plus custom webServer routes, persisting config to its own JSON file (independent of the api-proxy settings allowlist).

Privacy

Transcription sends image bytes (base64, HTTPS) to the vision endpoint you configure — image data leaves your machine unless the endpoint is local (e.g. Ollama). Nothing beyond the harness’s own attachment storage persists any image. For sensitive images, use your own endpoint or a local model, or don’t install this plugin.

License

MIT