Letter2025/dsh-model-failover

dsh-model-failover

Two-level model circuit breaker with failover for DeepSeek Harness. When a model (or a whole provider) starts failing repeatedly, the plugin opens a circuit and routes the next model request to a configured fallback — no core changes, installable with dsh plugin.

What it does

• Model circuit — a provider/model route opens after modelCircuitThreshold failures inside burstWindowMs.
• Platform circuit — a provider opens when platformCircuitThreshold distinct models under it are open at once (a platform outage usually takes down every model on it).
• Failover — the next request goes to the first healthy fallback in fallbacks; the switch is recorded by the loop itself (request/header change) and optionally announced as a user-visible message.
• Recovery probes — an open model circuit is probed after modelCooldownMs with a tiny real call; a successful probe closes the circuit, a failed one extends the cooldown.
• Composes with llm-retry — per-request backoff retries stay owned by the bundled llm-retry policy; the breaker observes the failures that escape it, so transient blips that recover after a retry never trip a circuit.

Install

dsh plugin --profile web add dsh-model-failover

Then configure fallbacks (the only field you must set) and, if needed, the thresholds in your profile cordis.patch.yml — see the plugin row in cordis.patch.yml for the full default config.

A companion skill, configure-model-failover, walks the agent through setting the fallback models (AI probes the current model config, writes the fallback override, then asks you to confirm). It is installed automatically with the plugin: the package ships skills/configure-model-failover/SKILL.md and the plugin registers it as a bundled skill whenever the skills service is present — no copy step needed. For a standalone install (profile without the plugin), copy it into ~/.dsh/skills/configure-model-failover/ (user-level skills are picked up live).

How it works

The plugin decorates two agent-loop waterfalls (both official extension points, no core changes):

request ──> agent/request ──> primary (mock/m1) ──> fail ×2 ──> circuit open
                                                                │
next request ──> agent/request ──> primary open ──> fallback (mock2/m2) ✔
                                                                │
                                     probe after cooldown ──> success ──> circuit closed

Configuration

Events

Plugin-defined (emit) events, typed via the @deepseek-ai/cordis augmentation in src/types.ts:

• model-failover/circuit-opened — {provider, model, level: 'model' | 'platform'}
• model-failover/circuit-closed — {provider, model, level: 'model'}
• model-failover/failover — {from, to, agentId}
• model-failover/probe — {provider, model, ok, message?}

Known Limitations and Deferred Work

• Process-local state — circuit state lives in memory and resets on plugin reload (like every harness registry). Cross-instance sharing is deferred.
• Agent-loop calls only — agent/request covers the main conversation loop. Auxiliary calls (session-title, compaction, hand-built ctx.llm.stream) are not routed.
• retryPolicy.mode: 'always' — the bundled llm-retry never delegates a failure to this breaker in that mode, so failover stays idle by design (the operator chose unbounded retries).
• No context-window adaptation — a fallback with a smaller context window may hit CONTEXT_WINDOW_EXCEEDED; set stripReasoningEffort and pick compatible fallbacks.
• No platform probe — the platform circuit recovers by cooldown expiry; only model circuits are probed.

License

MIT