wulusai2333/mimo-vision

mimo-vision · Native vision plugin for DSH

mimo-vision is a native plugin for DeepSeek Harness (DSH), package name mimo-vision. It registers a describe_image tool that sends an image to a mimo-v2.5-series multimodal model and returns the text description to the main model — a “vision bridge” built for main models (e.g. deepseek-v4-flash) that have no vision input of their own.

It is not a standalone process: it is a first-class citizen of DSH’s “everything is a plugin” model. apply does exactly one thing — registers the capability as a first-class dsh tool. Dependencies, files, credentials, and subprocesses all go through dsh’s defined capability seams; uninstalling cleans up cleanly.

Implementation paradigm: a direct landing of DSH’s native plugin primitives

• Registration is reversible by construction: apply(ctx) contains a single ctx.tools.register(defineTool(...)). register() returns a disposer; when the plugin fiber disposes, the tool is unregistered and its schema is automatically withdrawn from the system prompt. There is no leftover cleanup code — clean uninstall is a structural guarantee, not hand-written cleanup.
• inject declares dependencies: export const inject = ['tools', 'fs', 'credentials'] follows the pure Cordis effect spec — activation only happens once the seams are in place. ctx.get('subprocess') in transcode.ts is an optional capability with a default fallback, used at execution time, not activation time. This is “declared dependencies”, not “probed dependencies”.
• All capabilities go through dsh seams: file reads use ctx.fs.resolve/stat/readBytes + ctx.emit('fs/observed'); credentials use ctx.credentials.resolve(credentialRef(...)) with no hand-rolled parsing; subprocess (transcoding) uses ctx.subprocess. The one exception: transcode temp files are written via node:fs to the system temp directory (see Known Limitations below).
• Uninstallable / composable: uninstall is pure — the disposer runs and everything is reclaimed: no disk writes, no timers, no long-lived connections to manually wind down.

Tool

Supported image formats:

• Sent natively: PNG / JPEG / GIF / WebP / BMP (all verified decodable by the vision model)
• Auto-transcoded: SVG / TIFF (.tif) / HEIC (.heif) / PSD / ICO / EXR / JP2 / JXL / AVIF — when ImageMagick is installed locally, these are transcoded to PNG before sending, downscaled to a ≤2048px long edge (saves tokens)
• Any other extension, or a transcode-format when ImageMagick is not installed, returns an explicit error (never sent silently)

Usage example (tell the model): Use describe_image to describe D:\photos\cat.png, focusing on what breed of cat it is.

How it works

1. Key resolution: ctx.credentials.resolve takes the first non-empty of OPENCODE_GO_API_KEY → OPENCODE_API_KEY (DSH credential layering: process env > ~/.dsh/.credentials.yaml > .env).
2. Read the image: ctx.fs.resolve (relative paths resolve against the session workspace cwd) → ctx.fs.readBytes (20 MiB cap); non-native formats (SVG/TIFF/HEIC…) are transcoded to PNG via ImageMagick (through the ctx.subprocess seam) and downscaled to a 2048px long edge → base64.
3. Routing: the free Zen route (mimo-v2.5-free) is tried first; on failure (non-2xx / timeout) it falls back to the paid Go route (mimo-v2.5) once per request; allowPaid: false disables the paid fallback.

Design decisions are documented in adr/0002-dsh-native-plugin.md (which supersedes the earlier MCP approach, ADR-0001).

Quick install (npm-installed DSH, recommended)

For npx @deepseek-ai/dsh web or a globally installed DSH. The repo ships prebuilt artifacts (lib/index.js) — no build toolchain required.

Prerequisite: Node ^22.19 || >=24, and DSH already able to start.

Step 0 · Install and activate (one command)

mimo-vision declares dsh.bundle (see the dsh field in package.json), so dsh plugin add reconciles it as a bundle layer of the profile — installing the package, mounting the layer, and activating the tool happen in one step:

# from the npm registry (recommended once this package is published)
dsh plugin --profile web add mimo-vision

# from GitHub (source install, prebuilt lib/)
dsh plugin --profile web add github:wulusai2333/mimo-vision

If the repo is not on its default branch, use github:wulusai2333/mimo-vision#<branch-or-tag>. This path relies on the dependency closure that DSH maintains in ~/.dsh/profiles/node_modules (symlinking all @deepseek-ai/* seams to the dsh install tree), so the plugin’s runtime import "@deepseek-ai/dsh-tools" etc. resolves to the same instance DSH uses — singleton-safe, register()/inject semantics unchanged. GitHub-source installs require this repo to commit a prebuilt lib/ (kept in sync with src/): pnpm installs the source with lib/ included, no build runs, no allowBuilds needed.