xiaozhe7772222/dsh-api-key-pool

---

English

中文

---

English

✨ Features

Feature	Description
🔄 Round-robin rotation	Multiple keys per provider, rotated per request for load balancing
🛡️ Automatic failover	Key fails with 401 / 403 / 429 → automatically marked unhealthy, next key takes over
⏱️ Cooldown & recovery	Failed keys enter exponential-backoff cooldown (30s default), auto back in rotation when expired
🌐 Web management panel	DSH Web UI → Settings → Plugins → API Key Pool: add/remove keys, view health, reset cooldowns
🔒 Key masking	REST API only exposes masked keys (sk-abc****1234), never full secrets
💾 Persistent config	Keys added via Web UI persist to pool-config.json, survive restarts

📦 Installation

# Clone the repo
git clone https://github.com/xiaozhe7772222/dsh-api-key-pool.git
cd dsh-api-key-pool

# Copy into your DSH profile's plugins directory
mkdir -p ~/.dsh/profiles/web/plugins/dsh-api-key-pool
cp -r lib package.json cordis.patch.yml ~/.dsh/profiles/web/plugins/dsh-api-key-pool/

Then declare the bundle in ~/.dsh/profiles/web/package.json:

{
  "dsh": {
    "profile": {
      "bundles": [
        {
          "name": "dsh-api-key-pool",
          "platform": ["web"],
          "optional": false
        }
      ]
    }
  }
}

Restart DSH:

npx @deepseek-ai/dsh web

⚙️ Configuration

Option 1: Static config via cordis.patch.yml

- insert:
    - id: api-key-pool
      name: dsh-api-key-pool
      inject: [llm, webServer]
      config:
        pools:
          provider-a:
            apiKeyEnv: PROVIDER_A_API_KEY
            keys:
              - sk-your-first-key
              - sk-your-second-key
              - sk-your-third-key
            cooldownMs: 30000
          provider-b:
            apiKeyEnv: PROVIDER_B_API_KEY
            keys:
              - sk-key2-1
              - sk-key2-2
            cooldownMs: 60000
        defaultCooldownMs: 30000

Field	Type	Description
pools.<provider>.apiKeyEnv	string	Env var name for this provider’s API key (must match apiKeyEnv in settings.yaml)
pools.<provider>.keys	string[]	List of API keys to rotate
pools.<provider>.cooldownMs	number	Cooldown per key after failure (ms)
defaultCooldownMs	number	Global default cooldown (ms), default 30000

Option 2: Web management panel (runtime)

After starting DSH, open Settings → Plugins → API Key Pool:

• View each provider’s key health (green = healthy, orange = cooling)
• Paste a new key and click Add
• Click ✕ to remove a key
• Click Reset cooldown to clear cooling state for a provider
• Click Refresh to reload status

Keys added via the panel persist to pool-config.json automatically.

🧠 How it works (v0.3.0)

LLM request
   │
   ▼
┌──────────────────────────────────────────────────┐
│  agent/request waterfall (this plugin v0.3.0)      │
│  v0.3.0 fix #1: event payload has NO provider      │
│  → call next() first, read provider from the       │
│    returned LlmCallConfig                          │
│  ┌──────────────────────────────────────────────┐ │
│  │ 1. pickKey(provider) — round-robin healthy key│ │
│  │ 2. inject process.env[apiKeyEnv]             │ │
│  │ 3. credentials resolve env first → takes effect│ │
│  └──────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────┘
   │
   ▼
Provider API call
   │
   ├── Success → markSuccess (reset failCount)
   │
   └── Fail 429 (e.g. SenseNova 429001) / 401 / 403 / 5xx ...
        │
        ▼
┌──────────────────────────────────────────────────┐
│  agent/request-error waterfall (this plugin)      │
│  v0.3.0 fix #2: no-error-left-behind             │
│    ANY 4xx/5xx status, or message containing      │
│    rate/limit/quota/exhaust/timeout/auth → retry  │
│  v0.3.0 fix #3: return { kind: 'retry' }          │
│    agent loop re-issues the request (not just     │
│    mark-failed-and-quit)                          │
│  ┌──────────────────────────────────────────────┐ │
│  │ 1. markFailed(currentKey) → cooldown         │ │
│  │ 2. return { kind: 'retry' }                  │ │
│  └──────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────┘
   │
   ▼  agent loop retries → agent/request again
   pickKey skips the cooling key → picks next key
   │
   ├── key2 again 429 → cooldown → retry → key3 → OK ✅
   │
   └── all keys cooling → keeps retrying (wait for cooldown)

Key design: DSH resolves credentials in the order process.env > .credentials.yaml > .env. This plugin leverages that — it temporarily rewrites process.env in the agent/request waterfall for hot key switching with zero config file changes. Since v0.3.0, a failed key not only enters cooldown but also asks the agent loop to retry with the next key immediately (previously it only marked the key and the request died).

🔌 REST API

The plugin registers routes on the DSH web server.

GET /dsh-api-key-pool/pools

Returns all pools with masked keys, health states, and env names:

{
  "pools": {
    "provider-a": {
      "apiKeyEnv": "PROVIDER_A_API_KEY",
      "keys": ["sk-your-first-key"],
      "maskedKeys": ["sk-you****-key"],
      "states": {
        "sk-you****-key": { "failCount": 0, "cooldownUntil": 0 }
      }
    }
  },
  "defaultCooldownMs": 30000
}

POST /dsh-api-key-pool/pools

action	body	Description
add	{ action, provider, key }	Add a key
remove	{ action, provider, key }	Remove a key
update	{ action, provider, keys: [] }	Replace the full key list
reset	{ action, provider }	Reset all cooldown states for a provider

Example:

curl -X POST http://127.0.0.1:3080/dsh-api-key-pool/pools \
  -H 'Content-Type: application/json' \
  -d '{"action":"add","provider":"provider-a","key":"sk-new-key"}'

🗂️ Project structure

dsh-api-key-pool/
├── package.json          # Package metadata + DSH bundle declaration
├── cordis.patch.yml      # Static pool config (first-install example)
├── pool-config.json      # Runtime persistence (written by Web UI, gitignored)
├── lib/
│   ├── index.js          # Server: rotation logic + agent waterfalls + REST API
│   └── client.js         # Client: Web settings panel card
├── README.md
├── CHANGELOG.md
└── LICENSE

🏷️ Topics

deepseek-harness · dsh · api-key · api-key-rotation · key-pool · failover · load-balancing · round-robin · circuit-breaker · llm · openai · plugin · cordis

📄 License

MIT License — free to use, modify, distribute. Never commit your real API keys to a public repo.

---

中文文档

🚀 解决在 DeepSeek Harness 框架上面使用中转站模型或者第三方模型限流问题

✨ 功能特性

功能	说明
🔄 自动轮换	同一供应商配置多个 Key，请求时依次轮换，均衡负载
🛡️ 失败切换	Key 返回 401 / 403 / 429 时自动标记故障，切换到下一个健康 Key
⏱️ 冷却恢复	故障 Key 按指数退避进入冷却期（默认 30s 起步），到期自动回到轮换池
🌐 Web 管理面板	DSH Web UI → 设置 → 插件配置 → API Key 池：直接增删 Key、查看健康状态、重置冷却
🔒 Key 脱敏	REST 接口对外只返回脱敏后的 Key（sk-abc****1234），不泄露完整密钥
💾 配置持久化	Web 面板添加的 Key 持久化到 pool-config.json，重启后依然有效

📦 安装

# 克隆仓库
git clone https://github.com/xiaozhe7772222/dsh-api-key-pool.git
cd dsh-api-key-pool

# 拷贝到 DSH profile 的 plugins 目录
mkdir -p ~/.dsh/profiles/web/plugins/dsh-api-key-pool
cp -r lib package.json cordis.patch.yml ~/.dsh/profiles/web/plugins/dsh-api-key-pool/

然后在 ~/.dsh/profiles/web/package.json 的 dsh.profile.bundles 中声明：

{
  "dsh": {
    "profile": {
      "bundles": [
        {
          "name": "dsh-api-key-pool",
          "platform": ["web"],
          "optional": false
        }
      ]
    }
  }
}

重启 DSH：

npx @deepseek-ai/dsh web

⚙️ 配置说明

方式一：cordis.patch.yml（静态配置）

- insert:
    - id: api-key-pool
      name: dsh-api-key-pool
      inject: [llm, webServer]
      config:
        pools:
          provider-a:
            apiKeyEnv: PROVIDER_A_API_KEY
            keys:
              - sk-your-first-key
              - sk-your-second-key
              - sk-your-third-key
            cooldownMs: 30000
          provider-b:
            apiKeyEnv: PROVIDER_B_API_KEY
            keys:
              - sk-key2-1
              - sk-key2-2
            cooldownMs: 60000
        defaultCooldownMs: 30000

字段	类型	说明
pools.<provider>.apiKeyEnv	string	该供应商的 API Key 对应的环境变量名（需与 settings.yaml 中 apiKeyEnv 一致）
pools.<provider>.keys	string[]	要轮换的多个 API Key
pools.<provider>.cooldownMs	number	单个 Key 失败后的冷却时长（毫秒）
defaultCooldownMs	number	全局默认冷却时长（毫秒），默认 30000

方式二：Web 管理面板（运行时）

启动 DSH 后，打开 设置 → 插件配置 → API Key 池：

• 查看各供应商 Key 健康状态（绿点 = 健康，橙点 = 冷却中）
• 输入框粘贴新 Key，点击「添加」
• 点击 ✕ 移除不想用的 Key
• 点击「重置冷却」清空某个供应商的冷却状态
• 点击「刷新」重新加载状态

Web 面板添加的 Key 会自动持久化到 pool-config.json，重启后依然有效。

🧠 工作原理（v0.3.0）

LLM 请求
   │
   ▼
┌──────────────────────────────────────────────────┐
│  agent/request 瀑布（本插件 v0.3.0）               │
│  v0.3.0 修复1：事件 payload 没有 provider 字段     │
│  → 先调 next()，从返回的 LlmCallConfig 里取 provider│
│  ┌──────────────────────────────────────────────┐ │
│  │ 1. pickKey(provider) 轮询选健康 Key           │ │
│  │ 2. 注入 process.env[apiKeyEnv]                │ │
│  │ 3. 凭据解析优先读 env → Key 立即生效          │ │
│  └──────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────┘
   │
   ▼
供应商 API 调用
   │
   ├── 成功 → markSuccess（重置失败计数）
   │
   └── 失败 429（如商汤 429001）/ 401 / 403 / 5xx ...
        │
        ▼
┌──────────────────────────────────────────────────┐
│  agent/request-error 瀑布（本插件拦截）            │
│  v0.3.0 修复2：宁多勿漏                           │
│    任何 4xx/5xx 状态码，或消息含 rate/limit/quota/ │
│    exhaust/timeout/auth 等关键词 → 均判定可重试    │
│  v0.3.0 修复3：返回 { kind: 'retry' }              │
│    agent loop 收到后重新发起请求（不再直接退出！）  │
│  ┌──────────────────────────────────────────────┐ │
│  │ 1. markFailed(当前key) → 进入冷却             │ │
│  │ 2. return { kind: 'retry' }                  │ │
│  └──────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────┘
   │
   ▼  agent loop 重试 → 再次进入 agent/request
   pickKey 跳过冷却中的 key → 选中下一个 key
   │
   ├── key2 也 429 → 标记冷却 → retry → key3 → 成功 ✅
   │
   └── 全部 key 冷却 → 继续重试（等冷却），日志预警

关键设计：DSH 的凭据解析优先级是 process.env > .credentials.yaml > .env，本插件利用这一机制，在 agent/request 瀑布中临时改写 process.env，实现零配置文件改动的 Key 热切换。v0.3.0 起，失败的 Key 不仅进入冷却，还会让 agent loop 立刻换下一个 Key 重试（此前只是标记了故障但请求直接失败，等于白轮换）。

🔌 REST API

插件在 DSH webServer 上注册以下路由：

GET /dsh-api-key-pool/pools

返回所有池的 Key 列表（脱敏）、健康状态、环境变量名：

{
  "pools": {
    "provider-a": {
      "apiKeyEnv": "PROVIDER_A_API_KEY",
      "keys": ["sk-your-first-key"],
      "maskedKeys": ["sk-you****-key"],
      "states": {
        "sk-you****-key": { "failCount": 0, "cooldownUntil": 0 }
      }
    }
  },
  "defaultCooldownMs": 30000
}

POST /dsh-api-key-pool/pools

action	body	说明
add	{ action, provider, key }	添加一个 Key
remove	{ action, provider, key }	移除一个 Key
update	{ action, provider, keys: [] }	整体替换 Key 列表
reset	{ action, provider }	重置该供应商全部冷却状态

示例：

curl -X POST http://127.0.0.1:3080/dsh-api-key-pool/pools \
  -H 'Content-Type: application/json' \
  -d '{"action":"add","provider":"provider-a","key":"sk-new-key"}'

🗂️ 项目结构

dsh-api-key-pool/
├── package.json          # 包元数据 + DSH bundle 声明
├── cordis.patch.yml      # 静态池配置（首次安装示例）
├── pool-config.json      # 运行时持久化（Web 面板写入，已 gitignore）
├── lib/
│   ├── index.js          # 服务端：轮换逻辑 + agent 瀑布 + REST API
│   └── client.js         # 客户端：Web 设置面板卡片
├── README.md
├── CHANGELOG.md
└── LICENSE

🏷️ 标签 (Topics)

deepseek-harness · dsh · api-key · api-key-rotation · key-pool · failover · load-balancing · round-robin · circuit-breaker · llm · openai · plugin · cordis

📄 许可证

MIT License — 自由使用、修改、分发。请勿将你的真实 API Key 提交到公开仓库。