Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1c59906b49 | ||
|
|
19f666e7b1 | ||
|
|
dbcd9827f6 | ||
|
|
b689180717 | ||
|
|
f385c2254b | ||
|
|
9607a38792 | ||
|
|
239ddc767c | ||
|
|
057abaa95a | ||
|
|
dd864eed22 | ||
|
|
c1ff675d80 | ||
|
|
9ef1af3ae1 |
@@ -3,6 +3,56 @@
|
|||||||
All notable changes to this project are documented here.
|
All notable changes to this project are documented here.
|
||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||||
|
|
||||||
|
## [0.2.43] - 2026-08-08
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
|
||||||
|
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
|
||||||
|
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
|
||||||
|
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
|
||||||
|
configured = open mode, as before.
|
||||||
|
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
|
||||||
|
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
|
||||||
|
when the server becomes healthy, so a local model whose chat template can drive tool
|
||||||
|
calling serves agent (tools) traffic with zero manual configuration. Manual override:
|
||||||
|
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
|
||||||
|
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
|
||||||
|
remaining credit is a KPI card, red under $5.
|
||||||
|
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
|
||||||
|
per harness, an install command chip when the harness is missing, and a paste-your-key
|
||||||
|
field that substitutes the real key into every snippet.
|
||||||
|
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
|
||||||
|
Caddy TLS fronting, security checklist, backup list.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
|
||||||
|
route's chain has no model with the required capabilities (tools), the gateway escalates
|
||||||
|
through medium→high chains and then the rest of the pool instead of returning
|
||||||
|
"No healthy model available". Explicit pins still fail loudly.
|
||||||
|
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
|
||||||
|
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
|
||||||
|
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
|
||||||
|
no history-expansion `!`), and the install chips are readable in the light theme.
|
||||||
|
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
|
||||||
|
into the sidecar totals file — per-account all-time and month spend survive compaction
|
||||||
|
and restarts.
|
||||||
|
|
||||||
|
### Breaking
|
||||||
|
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
|
||||||
|
will now be rejected with 401 — either remove the keys from the config or start sending
|
||||||
|
`Authorization: Bearer <key>`.
|
||||||
|
|
||||||
|
## [0.2.42] - 2026-08-08
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **The continuous complexity score is visible**: the benchmark's routing table and the
|
||||||
|
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
|
||||||
|
see what different prompts actually weigh and tune the boundary sliders from real traffic.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
|
||||||
|
«Рабочий стол».
|
||||||
|
|
||||||
## [0.2.41] - 2026-08-08
|
## [0.2.41] - 2026-08-08
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
Generated
+1
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "cortiq-gateway"
|
name = "cortiq-gateway"
|
||||||
version = "0.2.41"
|
version = "0.2.43"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"arc-swap",
|
"arc-swap",
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "cortiq-gateway"
|
name = "cortiq-gateway"
|
||||||
version = "0.2.41"
|
version = "0.2.43"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
rust-version = "1.75"
|
rust-version = "1.75"
|
||||||
license = "Apache-2.0"
|
license = "Apache-2.0"
|
||||||
|
|||||||
@@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
|
|||||||
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
|
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
|
||||||
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
|
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
|
||||||
|
|
||||||
|
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🖥️ Admin console
|
## 🖥️ Admin console
|
||||||
|
|||||||
@@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
|||||||
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
|
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
|
||||||
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
|
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
|
||||||
|
|
||||||
|
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🖥️ Панель управления
|
## 🖥️ Панель управления
|
||||||
|
|||||||
+122
@@ -0,0 +1,122 @@
|
|||||||
|
# Deploying cortiq-gateway for a team / office
|
||||||
|
|
||||||
|
One gateway on a small server, API keys per department, local models for the
|
||||||
|
cheap tiers, cloud models for the hard ones. This page covers the ops side.
|
||||||
|
|
||||||
|
## 1. Run as a service
|
||||||
|
|
||||||
|
### Linux (systemd)
|
||||||
|
|
||||||
|
```ini
|
||||||
|
# /etc/systemd/system/cortiq-gateway.service
|
||||||
|
[Unit]
|
||||||
|
Description=Cortiq Gateway
|
||||||
|
After=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
User=cortiq
|
||||||
|
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||||
|
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||||
|
Restart=always
|
||||||
|
RestartSec=3
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl enable --now cortiq-gateway
|
||||||
|
```
|
||||||
|
|
||||||
|
### macOS (launchd)
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||||
|
<plist version="1.0"><dict>
|
||||||
|
<key>Label</key><string>com.cortiq.gateway</string>
|
||||||
|
<key>ProgramArguments</key><array>
|
||||||
|
<string>/usr/local/bin/cortiq-gateway</string>
|
||||||
|
<string>--no-port-search</string>
|
||||||
|
</array>
|
||||||
|
<key>RunAtLoad</key><true/>
|
||||||
|
<key>KeepAlive</key><true/>
|
||||||
|
</dict></plist>
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||||
|
```
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
Task Scheduler → Create Task → "Run whether user is logged on or not",
|
||||||
|
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
|
||||||
|
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||||
|
|
||||||
|
### Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||||
|
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. TLS and the outside world
|
||||||
|
|
||||||
|
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
|
||||||
|
and bind the gateway to localhost:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
# gateway.toml
|
||||||
|
listen = "127.0.0.1:9000"
|
||||||
|
```
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
server {
|
||||||
|
listen 443 ssl;
|
||||||
|
server_name ai.example.com;
|
||||||
|
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://127.0.0.1:9000;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_buffering off; # SSE streaming
|
||||||
|
proxy_read_timeout 3600s; # long generations
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
|
||||||
|
same with automatic certificates.)
|
||||||
|
|
||||||
|
## 3. Security checklist
|
||||||
|
|
||||||
|
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
|
||||||
|
the console warns that it is open to the network.
|
||||||
|
- **API keys**: create one per department/developer on the keys page. With at
|
||||||
|
least one key configured the model APIs require Bearer / x-api-key —
|
||||||
|
requests without a valid key get 401.
|
||||||
|
- **Rate limits**: per-key requests/min → 429 with Retry-After.
|
||||||
|
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
|
||||||
|
account's month-to-date spend crosses it.
|
||||||
|
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
|
||||||
|
|
||||||
|
## 4. Backups
|
||||||
|
|
||||||
|
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
|
||||||
|
|
||||||
|
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
|
||||||
|
- `secrets.toml` — provider API keys (chmod 600);
|
||||||
|
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
|
||||||
|
cumulative per-key totals and month-to-date budget spend.
|
||||||
|
|
||||||
|
Copy those three and you can rebuild the instance anywhere.
|
||||||
|
|
||||||
|
## 5. Observability
|
||||||
|
|
||||||
|
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
|
||||||
|
OpenRouter balance (red under $5).
|
||||||
|
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
|
||||||
|
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
# Развёртывание cortiq-gateway для команды / офиса
|
||||||
|
|
||||||
|
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
|
||||||
|
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
|
||||||
|
|
||||||
|
## 1. Запуск как сервис
|
||||||
|
|
||||||
|
### Linux (systemd)
|
||||||
|
|
||||||
|
```ini
|
||||||
|
# /etc/systemd/system/cortiq-gateway.service
|
||||||
|
[Unit]
|
||||||
|
Description=Cortiq Gateway
|
||||||
|
After=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
User=cortiq
|
||||||
|
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||||
|
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||||
|
Restart=always
|
||||||
|
RestartSec=3
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl enable --now cortiq-gateway
|
||||||
|
```
|
||||||
|
|
||||||
|
### macOS (launchd)
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||||
|
<plist version="1.0"><dict>
|
||||||
|
<key>Label</key><string>com.cortiq.gateway</string>
|
||||||
|
<key>ProgramArguments</key><array>
|
||||||
|
<string>/usr/local/bin/cortiq-gateway</string>
|
||||||
|
<string>--no-port-search</string>
|
||||||
|
</array>
|
||||||
|
<key>RunAtLoad</key><true/>
|
||||||
|
<key>KeepAlive</key><true/>
|
||||||
|
</dict></plist>
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||||
|
```
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
|
||||||
|
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
|
||||||
|
--no-port-search`. Либо [NSSM](https://nssm.cc):
|
||||||
|
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||||
|
|
||||||
|
### Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||||
|
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. TLS и внешний мир
|
||||||
|
|
||||||
|
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
|
||||||
|
привяжите к localhost:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
# gateway.toml
|
||||||
|
listen = "127.0.0.1:9000"
|
||||||
|
```
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
server {
|
||||||
|
listen 443 ssl;
|
||||||
|
server_name ai.example.com;
|
||||||
|
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://127.0.0.1:9000;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_buffering off; # SSE-стриминг
|
||||||
|
proxy_read_timeout 3600s; # долгие генерации
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
|
||||||
|
с автоматическими сертификатами.)
|
||||||
|
|
||||||
|
## 3. Чек-лист безопасности
|
||||||
|
|
||||||
|
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
|
||||||
|
него консоль предупреждает, что открыта сети.
|
||||||
|
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
|
||||||
|
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
|
||||||
|
запросы без валидного ключа получают 401.
|
||||||
|
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
|
||||||
|
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
|
||||||
|
траты аккаунта за месяц превысят лимит.
|
||||||
|
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
|
||||||
|
|
||||||
|
## 4. Бэкапы
|
||||||
|
|
||||||
|
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
|
||||||
|
|
||||||
|
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
|
||||||
|
- `secrets.toml` — ключи провайдеров (chmod 600);
|
||||||
|
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
|
||||||
|
тоталы по ключам и месячные траты для бюджетов.
|
||||||
|
|
||||||
|
Скопируйте эти три файла — и инстанс поднимается где угодно.
|
||||||
|
|
||||||
|
## 5. Наблюдаемость
|
||||||
|
|
||||||
|
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
|
||||||
|
сложности, баланс OpenRouter (красный ниже $5).
|
||||||
|
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
|
||||||
|
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
|
||||||
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
|||||||
get(get_protocols).put(put_protocols),
|
get(get_protocols).put(put_protocols),
|
||||||
)
|
)
|
||||||
.route("/admin/api/harnesses", get(get_harnesses))
|
.route("/admin/api/harnesses", get(get_harnesses))
|
||||||
|
.route("/admin/api/balance", get(get_balance))
|
||||||
.route("/admin/api/settings", get(get_settings).put(put_settings))
|
.route("/admin/api/settings", get(get_settings).put(put_settings))
|
||||||
.route("/admin/api/keys", get(list_keys).post(create_key))
|
.route("/admin/api/keys", get(list_keys).post(create_key))
|
||||||
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
|
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
|
||||||
@@ -1008,6 +1009,51 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
|
|||||||
ok(json!({ "ok": true }))
|
ok(json!({ "ok": true }))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─────────────────────────────── balance ───────────────────────────────────
|
||||||
|
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
|
||||||
|
// (null) when no OpenRouter key is configured — the card hides itself.
|
||||||
|
|
||||||
|
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
|
||||||
|
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
|
||||||
|
std::sync::OnceLock::new();
|
||||||
|
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
|
||||||
|
if let Some((at, v)) = *cache.lock().unwrap() {
|
||||||
|
if at.elapsed() < std::time::Duration::from_secs(300) {
|
||||||
|
return ok(json!({ "openrouter_usd": v }));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let key = state
|
||||||
|
.secrets
|
||||||
|
.resolve("OPENROUTER_API_KEY")
|
||||||
|
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
|
||||||
|
let Some(key) = key.filter(|k| !k.is_empty()) else {
|
||||||
|
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
|
||||||
|
};
|
||||||
|
let client = reqwest::Client::builder()
|
||||||
|
.timeout(std::time::Duration::from_secs(8))
|
||||||
|
.build()
|
||||||
|
.unwrap_or_default();
|
||||||
|
let resp = client
|
||||||
|
.get("https://openrouter.ai/api/v1/credits")
|
||||||
|
.bearer_auth(&key)
|
||||||
|
.send()
|
||||||
|
.await;
|
||||||
|
let balance = match resp {
|
||||||
|
Ok(r) if r.status().is_success() => {
|
||||||
|
let v: serde_json::Value = r.json().await.unwrap_or_default();
|
||||||
|
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
|
||||||
|
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
|
||||||
|
Some(total - used)
|
||||||
|
}
|
||||||
|
_ => None,
|
||||||
|
};
|
||||||
|
let Some(balance) = balance else {
|
||||||
|
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
|
||||||
|
};
|
||||||
|
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
|
||||||
|
ok(json!({ "openrouter_usd": balance }))
|
||||||
|
}
|
||||||
|
|
||||||
// ──────────────────────────── harness detection ────────────────────────────
|
// ──────────────────────────── harness detection ────────────────────────────
|
||||||
// Which coding harnesses are present on the machine the gateway runs on, so the
|
// Which coding harnesses are present on the machine the gateway runs on, so the
|
||||||
// API-keys view can mark tabs "installed" / "already configured for cortiq".
|
// API-keys view can mark tabs "installed" / "already configured for cortiq".
|
||||||
@@ -1357,6 +1403,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
|
|||||||
"key_masked": mask_key(&k.key),
|
"key_masked": mask_key(&k.key),
|
||||||
"account": k.account,
|
"account": k.account,
|
||||||
"rate_per_min": k.rate_per_min,
|
"rate_per_min": k.rate_per_min,
|
||||||
|
"monthly_budget_usd": k.monthly_budget_usd,
|
||||||
"allow_models": k.allow_models,
|
"allow_models": k.allow_models,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
@@ -1372,6 +1419,8 @@ struct NewKey {
|
|||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
rate_per_min: u32,
|
rate_per_min: u32,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
|
monthly_budget_usd: Option<f64>,
|
||||||
|
#[serde(default)]
|
||||||
allow_models: Vec<String>,
|
allow_models: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1390,6 +1439,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
|
|||||||
account: b.account,
|
account: b.account,
|
||||||
rate_per_min: b.rate_per_min,
|
rate_per_min: b.rate_per_min,
|
||||||
allow_models: b.allow_models,
|
allow_models: b.allow_models,
|
||||||
|
monthly_budget_usd: b.monthly_budget_usd,
|
||||||
});
|
});
|
||||||
state.reload(cfg)?;
|
state.reload(cfg)?;
|
||||||
// show the full key only once
|
// show the full key only once
|
||||||
@@ -2147,6 +2197,7 @@ fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<
|
|||||||
o1_window: None,
|
o1_window: None,
|
||||||
o1_sink: None,
|
o1_sink: None,
|
||||||
skip_mtp: false,
|
skip_mtp: false,
|
||||||
|
caps: Vec::new(),
|
||||||
});
|
});
|
||||||
Some(id)
|
Some(id)
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-1
@@ -55,7 +55,7 @@ impl SemanticCache {
|
|||||||
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
|
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
|
||||||
Arc::new(Self {
|
Arc::new(Self {
|
||||||
enabled: cfg.enabled,
|
enabled: cfg.enabled,
|
||||||
threshold: cfg.threshold,
|
threshold: cfg.threshold as f32,
|
||||||
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
|
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
|
||||||
max_entries: cfg.max_entries.max(1),
|
max_entries: cfg.max_entries.max(1),
|
||||||
embed_model: cfg.embed_model.clone(),
|
embed_model: cfg.embed_model.clone(),
|
||||||
|
|||||||
+27
-5
@@ -56,6 +56,9 @@ struct Managed {
|
|||||||
pub struct CmfRuntime {
|
pub struct CmfRuntime {
|
||||||
status: Mutex<CmfStatus>,
|
status: Mutex<CmfStatus>,
|
||||||
children: Mutex<Vec<Managed>>,
|
children: Mutex<Vec<Managed>>,
|
||||||
|
/// Server ids whose healthz handshake advertised tool support — lets the
|
||||||
|
/// pipeline route tools traffic to local models with no manual caps.
|
||||||
|
pub detected_tools: Mutex<std::collections::HashSet<String>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl CmfRuntime {
|
impl CmfRuntime {
|
||||||
@@ -63,6 +66,7 @@ impl CmfRuntime {
|
|||||||
Arc::new(Self {
|
Arc::new(Self {
|
||||||
status: Mutex::new(CmfStatus::default()),
|
status: Mutex::new(CmfStatus::default()),
|
||||||
children: Mutex::new(Vec::new()),
|
children: Mutex::new(Vec::new()),
|
||||||
|
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -193,6 +197,7 @@ impl Default for CmfRuntime {
|
|||||||
Self {
|
Self {
|
||||||
status: Mutex::new(CmfStatus::default()),
|
status: Mutex::new(CmfStatus::default()),
|
||||||
children: Mutex::new(Vec::new()),
|
children: Mutex::new(Vec::new()),
|
||||||
|
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -332,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
|||||||
.map_err(|e| e.to_string())
|
.map_err(|e| e.to_string())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
|
/// Poll `/healthz` until the local server answers 200 (or the deadline
|
||||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
|
||||||
|
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
|
||||||
|
/// caller can register them; older servers return `None` there.
|
||||||
|
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
|
||||||
let url = format!("http://{host}:{port}/healthz");
|
let url = format!("http://{host}:{port}/healthz");
|
||||||
let client = reqwest::Client::new();
|
let client = reqwest::Client::new();
|
||||||
for _ in 0..tries {
|
for _ in 0..tries {
|
||||||
@@ -344,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
|||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
if r.status().is_success() {
|
if r.status().is_success() {
|
||||||
return true;
|
let tools = r
|
||||||
|
.json::<serde_json::Value>()
|
||||||
|
.await
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v["capabilities"]["tools"].as_bool());
|
||||||
|
return (true, tools);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||||
}
|
}
|
||||||
false
|
(false, None)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
||||||
@@ -432,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
|
|||||||
if !rt.is_running(&server.id) {
|
if !rt.is_running(&server.id) {
|
||||||
continue; // failed to spawn / file missing
|
continue; // failed to spawn / file missing
|
||||||
}
|
}
|
||||||
if wait_healthy(&cfg.local_host, server.port, 240).await {
|
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
|
||||||
|
if ok {
|
||||||
rt.set_healthy(&server.id, true);
|
rt.set_healthy(&server.id, true);
|
||||||
|
// capability handshake: the server said whether its model's chat
|
||||||
|
// template can drive tool calling — no manual caps needed
|
||||||
|
if tools == Some(true) {
|
||||||
|
rt.detected_tools.lock().unwrap().insert(server.id.clone());
|
||||||
|
tracing::info!(id = %server.id, "local server advertises tool support");
|
||||||
|
} else {
|
||||||
|
rt.detected_tools.lock().unwrap().remove(&server.id);
|
||||||
|
}
|
||||||
rt.log(format!(
|
rt.log(format!(
|
||||||
"local CMF server ready — '{}' at http://{}:{}/v1",
|
"local CMF server ready — '{}' at http://{}:{}/v1",
|
||||||
server.id, cfg.local_host, server.port
|
server.id, cfg.local_host, server.port
|
||||||
|
|||||||
+12
-3
@@ -130,6 +130,10 @@ pub struct CmfServer {
|
|||||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub skip_mtp: bool,
|
pub skip_mtp: bool,
|
||||||
|
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
|
||||||
|
/// gateway auto-detects tool support from the server's healthz handshake.
|
||||||
|
#[serde(default)]
|
||||||
|
pub caps: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||||
@@ -229,6 +233,7 @@ impl CmfCfg {
|
|||||||
o1_window: None,
|
o1_window: None,
|
||||||
o1_sink: None,
|
o1_sink: None,
|
||||||
skip_mtp: false,
|
skip_mtp: false,
|
||||||
|
caps: Vec::new(),
|
||||||
}];
|
}];
|
||||||
}
|
}
|
||||||
Vec::new()
|
Vec::new()
|
||||||
@@ -620,6 +625,9 @@ pub struct ApiKeyCfg {
|
|||||||
pub account: String,
|
pub account: String,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub rate_per_min: u32,
|
pub rate_per_min: u32,
|
||||||
|
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub monthly_budget_usd: Option<f64>,
|
||||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
pub allow_models: Vec<String>,
|
pub allow_models: Vec<String>,
|
||||||
}
|
}
|
||||||
@@ -829,9 +837,10 @@ fn default_shadow_soak() -> usize {
|
|||||||
pub struct CacheCfg {
|
pub struct CacheCfg {
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub enabled: bool,
|
pub enabled: bool,
|
||||||
/// Cosine-similarity threshold for a hit (0..1).
|
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
|
||||||
|
/// the number you typed (an f32 round-trips as 0.9200000166893005).
|
||||||
#[serde(default = "default_semcache_threshold")]
|
#[serde(default = "default_semcache_threshold")]
|
||||||
pub threshold: f32,
|
pub threshold: f64,
|
||||||
/// Entry time-to-live, e.g. `1h`.
|
/// Entry time-to-live, e.g. `1h`.
|
||||||
#[serde(default = "default_semcache_ttl")]
|
#[serde(default = "default_semcache_ttl")]
|
||||||
pub ttl: String,
|
pub ttl: String,
|
||||||
@@ -853,7 +862,7 @@ impl Default for CacheCfg {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
fn default_semcache_threshold() -> f32 {
|
fn default_semcache_threshold() -> f64 {
|
||||||
0.92
|
0.92
|
||||||
}
|
}
|
||||||
fn default_semcache_ttl() -> String {
|
fn default_semcache_ttl() -> String {
|
||||||
|
|||||||
@@ -13,6 +13,8 @@ pub enum GatewayError {
|
|||||||
Unauthorized(String),
|
Unauthorized(String),
|
||||||
#[error("rate limit exceeded")]
|
#[error("rate limit exceeded")]
|
||||||
RateLimited { retry_after_secs: u64 },
|
RateLimited { retry_after_secs: u64 },
|
||||||
|
#[error("monthly budget exhausted (limit ${budget_usd})")]
|
||||||
|
BudgetExhausted { budget_usd: f64 },
|
||||||
#[error("upstream unavailable: {0}")]
|
#[error("upstream unavailable: {0}")]
|
||||||
UpstreamUnavailable(String),
|
UpstreamUnavailable(String),
|
||||||
#[error("internal error: {0}")]
|
#[error("internal error: {0}")]
|
||||||
@@ -37,6 +39,11 @@ impl GatewayError {
|
|||||||
"rate_limit_error",
|
"rate_limit_error",
|
||||||
"rate_limited",
|
"rate_limited",
|
||||||
),
|
),
|
||||||
|
Self::BudgetExhausted { .. } => (
|
||||||
|
StatusCode::TOO_MANY_REQUESTS,
|
||||||
|
"insufficient_quota",
|
||||||
|
"insufficient_quota",
|
||||||
|
),
|
||||||
Self::UpstreamUnavailable(_) => (
|
Self::UpstreamUnavailable(_) => (
|
||||||
StatusCode::BAD_GATEWAY,
|
StatusCode::BAD_GATEWAY,
|
||||||
"upstream_unavailable",
|
"upstream_unavailable",
|
||||||
|
|||||||
+15
-3
@@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
.route("/health", get(|| async { "ok" }))
|
.route("/health", get(|| async { "ok" }))
|
||||||
.route("/readyz", get(|| async { "ready" }))
|
.route("/readyz", get(|| async { "ready" }))
|
||||||
.route("/metrics", get(admin::metrics))
|
.route("/metrics", get(admin::metrics))
|
||||||
.merge(protocols::build_router());
|
.merge(protocols::build_router(state.clone()));
|
||||||
|
|
||||||
if admin_enabled {
|
if admin_enabled {
|
||||||
app = app
|
app = app
|
||||||
@@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
.allow_methods(tower_http::cors::Any)
|
.allow_methods(tower_http::cors::Any)
|
||||||
.allow_headers(tower_http::cors::Any);
|
.allow_headers(tower_http::cors::Any);
|
||||||
|
|
||||||
let app = app.with_state(state).layer(cors);
|
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
|
||||||
|
// small enough that a hostile client can't balloon memory
|
||||||
|
let app =
|
||||||
|
app.with_state(state)
|
||||||
|
.layer(cors)
|
||||||
|
.layer(tower_http::limit::RequestBodyLimitLayer::new(
|
||||||
|
32 * 1024 * 1024,
|
||||||
|
));
|
||||||
|
|
||||||
let listener = {
|
let listener = {
|
||||||
let mut attempt = 0u32;
|
let mut attempt = 0u32;
|
||||||
@@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
tracing::info!("open this URL to finish setup: {url}");
|
tracing::info!("open this URL to finish setup: {url}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
axum::serve(listener, app).await?;
|
axum::serve(listener, app)
|
||||||
|
.with_graceful_shutdown(async {
|
||||||
|
let _ = tokio::signal::ctrl_c().await;
|
||||||
|
tracing::info!("shutdown signal received — draining connections");
|
||||||
|
})
|
||||||
|
.await?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|||||||
+50
-4
@@ -183,6 +183,9 @@ impl Pipeline {
|
|||||||
&self,
|
&self,
|
||||||
req: &ChatRequest,
|
req: &ChatRequest,
|
||||||
live: &crate::state::Live,
|
live: &crate::state::Live,
|
||||||
|
// Tool support detected via the local servers' healthz handshake —
|
||||||
|
// treated as equivalent to declared caps when filtering candidates.
|
||||||
|
detected_tools: &std::collections::HashSet<String>,
|
||||||
) -> Result<(RouteDecision, String, Vec<String>)> {
|
) -> Result<(RouteDecision, String, Vec<String>)> {
|
||||||
// 1. Extract text for classification
|
// 1. Extract text for classification
|
||||||
let text = self.extract_text(
|
let text = self.extract_text(
|
||||||
@@ -274,10 +277,13 @@ impl Pipeline {
|
|||||||
|
|
||||||
// Filter candidates by capabilities (e.g. tools support)
|
// Filter candidates by capabilities (e.g. tools support)
|
||||||
let needs_tools = !req.tools.is_empty();
|
let needs_tools = !req.tools.is_empty();
|
||||||
|
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
|
||||||
|
prov.caps().tools || detected_tools.contains(id)
|
||||||
|
};
|
||||||
let mut final_candidates = Vec::new();
|
let mut final_candidates = Vec::new();
|
||||||
for model_id in candidates {
|
for model_id in candidates {
|
||||||
if let Some(prov) = live.registry.get(&model_id) {
|
if let Some(prov) = live.registry.get(&model_id) {
|
||||||
if needs_tools && !prov.caps().tools {
|
if needs_tools && !has_tools(&model_id, &prov) {
|
||||||
tracing::debug!(
|
tracing::debug!(
|
||||||
"Candidate '{}' skipped: lacks support for requested tools",
|
"Candidate '{}' skipped: lacks support for requested tools",
|
||||||
model_id
|
model_id
|
||||||
@@ -306,13 +312,49 @@ impl Pipeline {
|
|||||||
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
|
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
|
||||||
for id in live.local_candidates() {
|
for id in live.local_candidates() {
|
||||||
if let Some(prov) = live.registry.get(&id) {
|
if let Some(prov) = live.registry.get(&id) {
|
||||||
if needs_tools && !prov.caps().tools {
|
if needs_tools && !has_tools(&id, &prov) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
final_candidates.push(id);
|
final_candidates.push(id);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Capability widening: a tier chain built of tools-less local models
|
||||||
|
// must not kill an agent request. When the filtered chain is empty on
|
||||||
|
// an AUTO route, widen to any pool model that satisfies the required
|
||||||
|
// capabilities — preferring the routing table's own medium→high order
|
||||||
|
// (an agentic request is worth a capable model), then the rest of the
|
||||||
|
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
|
||||||
|
// this every low-tier verdict was a hard failure.
|
||||||
|
if final_candidates.is_empty() {
|
||||||
|
let mut widened: Vec<String> = Vec::new();
|
||||||
|
for tier in ["medium", "high"] {
|
||||||
|
for id in live.routing.candidates(tier) {
|
||||||
|
if !widened.contains(&id) {
|
||||||
|
widened.push(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for id in live.registry.ids() {
|
||||||
|
if !widened.contains(id) {
|
||||||
|
widened.push(id.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for id in widened {
|
||||||
|
if let Some(prov) = live.registry.get(&id) {
|
||||||
|
if needs_tools && !has_tools(&id, &prov) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
final_candidates.push(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !final_candidates.is_empty() {
|
||||||
|
tracing::info!(
|
||||||
|
chosen = %final_candidates[0],
|
||||||
|
"capability widening: tier chain had no model with the required capabilities"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
if final_candidates.is_empty() {
|
if final_candidates.is_empty() {
|
||||||
return Err(GatewayError::UpstreamUnavailable(
|
return Err(GatewayError::UpstreamUnavailable(
|
||||||
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
||||||
@@ -324,7 +366,9 @@ impl Pipeline {
|
|||||||
|
|
||||||
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
||||||
let live = state.live();
|
let live = state.live();
|
||||||
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
|
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||||
|
let (decision, route_source, mut final_candidates) =
|
||||||
|
self.resolve(&req, &live, &detected_tools).await?;
|
||||||
|
|
||||||
// Serving-when-promoted (gated): a promoted task-label is served by the
|
// Serving-when-promoted (gated): a promoted task-label is served by the
|
||||||
// local model FIRST (failover to cloud on error). Per-request VETO uses
|
// local model FIRST (failover to cloud on error). Per-request VETO uses
|
||||||
@@ -471,7 +515,9 @@ impl Pipeline {
|
|||||||
.to_string();
|
.to_string();
|
||||||
|
|
||||||
let live = state.live();
|
let live = state.live();
|
||||||
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
|
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||||
|
let (decision, route_source, final_candidates) =
|
||||||
|
match self.resolve(&req, &live, &detected_tools).await {
|
||||||
Ok(t) => t,
|
Ok(t) => t,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
record_error(state, &account, &protocol, &directive, started, &e);
|
record_error(state, &account, &protocol, &directive, started, &e);
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
|||||||
|
|
||||||
async fn handler(
|
async fn handler(
|
||||||
State(state): State<SharedState>,
|
State(state): State<SharedState>,
|
||||||
|
acct: Option<axum::Extension<super::AccountTag>>,
|
||||||
Json(req): Json<MessagesRequest>,
|
Json(req): Json<MessagesRequest>,
|
||||||
) -> Result<Response> {
|
) -> Result<Response> {
|
||||||
if !state.live().cfg.protocols.anthropic_messages {
|
if !state.live().cfg.protocols.anthropic_messages {
|
||||||
@@ -108,6 +109,7 @@ async fn handler(
|
|||||||
stream: req.stream,
|
stream: req.stream,
|
||||||
meta: RequestMeta {
|
meta: RequestMeta {
|
||||||
protocol: "anthropic_messages".into(),
|
protocol: "anthropic_messages".into(),
|
||||||
|
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||||
..Default::default()
|
..Default::default()
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|||||||
+51
-1
@@ -12,10 +12,59 @@ pub mod openai_models;
|
|||||||
use crate::state::SharedState;
|
use crate::state::SharedState;
|
||||||
use axum::Router;
|
use axum::Router;
|
||||||
|
|
||||||
|
/// Account resolved by the API-key middleware, consumed by protocol handlers
|
||||||
|
/// for attribution (stats, budgets). Present only when keys are configured.
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct AccountTag(pub String);
|
||||||
|
|
||||||
|
/// API-key gate for the model-facing surface. Open mode (no keys configured)
|
||||||
|
/// passes everything through — the admin console already warns about it.
|
||||||
|
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
|
||||||
|
/// enforced with a fixed one-minute window (429 + Retry-After), and a
|
||||||
|
/// monthly budget, when set, turns into 429 insufficient_quota once the
|
||||||
|
/// account's month-to-date spend crosses it.
|
||||||
|
async fn api_auth(
|
||||||
|
axum::extract::State(state): axum::extract::State<SharedState>,
|
||||||
|
mut req: axum::extract::Request,
|
||||||
|
next: axum::middleware::Next,
|
||||||
|
) -> axum::response::Response {
|
||||||
|
use axum::response::IntoResponse;
|
||||||
|
let live = state.live();
|
||||||
|
if live.cfg.api_keys.is_empty() {
|
||||||
|
return next.run(req).await;
|
||||||
|
}
|
||||||
|
let provided = req
|
||||||
|
.headers()
|
||||||
|
.get(axum::http::header::AUTHORIZATION)
|
||||||
|
.and_then(|v| v.to_str().ok())
|
||||||
|
.and_then(|v| v.strip_prefix("Bearer "))
|
||||||
|
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
|
||||||
|
.map(str::trim)
|
||||||
|
.unwrap_or("");
|
||||||
|
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
|
||||||
|
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
|
||||||
|
.into_response();
|
||||||
|
};
|
||||||
|
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
|
||||||
|
return crate::error::GatewayError::RateLimited {
|
||||||
|
retry_after_secs: state.rate.retry_after_secs(),
|
||||||
|
}
|
||||||
|
.into_response();
|
||||||
|
}
|
||||||
|
if let Some(budget) = key.monthly_budget_usd {
|
||||||
|
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
|
||||||
|
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
|
||||||
|
.into_response();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
req.extensions_mut().insert(AccountTag(key.account.clone()));
|
||||||
|
next.run(req).await
|
||||||
|
}
|
||||||
|
|
||||||
/// Build the protocols router. Implemented adapters are **always** mounted;
|
/// Build the protocols router. Implemented adapters are **always** mounted;
|
||||||
/// each handler checks the live `protocols.*` flag from config itself, so
|
/// each handler checks the live `protocols.*` flag from config itself, so
|
||||||
/// toggles in the admin panel take effect without a restart (hot switching).
|
/// toggles in the admin panel take effect without a restart (hot switching).
|
||||||
pub fn build_router() -> Router<SharedState> {
|
pub fn build_router(state: SharedState) -> Router<SharedState> {
|
||||||
Router::new()
|
Router::new()
|
||||||
.merge(openai_chat::routes())
|
.merge(openai_chat::routes())
|
||||||
.merge(openai_completions::routes())
|
.merge(openai_completions::routes())
|
||||||
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
|
|||||||
.merge(anthropic_messages::routes())
|
.merge(anthropic_messages::routes())
|
||||||
.merge(mcp::routes())
|
.merge(mcp::routes())
|
||||||
.merge(native_passthrough::routes())
|
.merge(native_passthrough::routes())
|
||||||
|
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
|||||||
|
|
||||||
async fn handler(
|
async fn handler(
|
||||||
State(state): State<SharedState>,
|
State(state): State<SharedState>,
|
||||||
|
acct: Option<axum::Extension<super::AccountTag>>,
|
||||||
Json(req): Json<OpenAiChatRequest>,
|
Json(req): Json<OpenAiChatRequest>,
|
||||||
) -> Result<Response> {
|
) -> Result<Response> {
|
||||||
// hot protocol toggle: if the adapter is disabled in config — return 404
|
// hot protocol toggle: if the adapter is disabled in config — return 404
|
||||||
@@ -104,6 +105,7 @@ async fn handler(
|
|||||||
stream: req.stream,
|
stream: req.stream,
|
||||||
meta: RequestMeta {
|
meta: RequestMeta {
|
||||||
protocol: "openai_chat".into(),
|
protocol: "openai_chat".into(),
|
||||||
|
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||||
..Default::default()
|
..Default::default()
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
|
|||||||
|
|
||||||
async fn handler(
|
async fn handler(
|
||||||
State(state): State<SharedState>,
|
State(state): State<SharedState>,
|
||||||
|
acct: Option<axum::Extension<super::AccountTag>>,
|
||||||
Json(req): Json<CompletionsRequest>,
|
Json(req): Json<CompletionsRequest>,
|
||||||
) -> Result<Response> {
|
) -> Result<Response> {
|
||||||
if !state.live().cfg.protocols.openai_completions {
|
if !state.live().cfg.protocols.openai_completions {
|
||||||
@@ -89,6 +90,7 @@ async fn handler(
|
|||||||
stream: req.stream,
|
stream: req.stream,
|
||||||
meta: RequestMeta {
|
meta: RequestMeta {
|
||||||
protocol: "openai_completions".into(),
|
protocol: "openai_completions".into(),
|
||||||
|
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||||
..Default::default()
|
..Default::default()
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|||||||
+1
-1
@@ -48,7 +48,7 @@ impl Registry {
|
|||||||
price_out: 0.0,
|
price_out: 0.0,
|
||||||
kind: "chat".into(),
|
kind: "chat".into(),
|
||||||
api_key_env: None,
|
api_key_env: None,
|
||||||
caps: Vec::new(),
|
caps: s.caps.clone(),
|
||||||
temperature: s.temperature,
|
temperature: s.temperature,
|
||||||
top_p: s.top_p,
|
top_p: s.top_p,
|
||||||
max_tokens: s.max_tokens,
|
max_tokens: s.max_tokens,
|
||||||
|
|||||||
@@ -66,6 +66,42 @@ impl Live {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
|
||||||
|
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
|
||||||
|
/// fine for office-grade fairness and costs one HashMap lookup per request.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct RateWindows {
|
||||||
|
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RateWindows {
|
||||||
|
pub fn allow(&self, key: &str, per_min: u32) -> bool {
|
||||||
|
let minute = std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_secs()
|
||||||
|
/ 60;
|
||||||
|
let mut g = self.inner.lock().unwrap();
|
||||||
|
let e = g.entry(key.to_string()).or_insert((minute, 0));
|
||||||
|
if e.0 != minute {
|
||||||
|
*e = (minute, 0);
|
||||||
|
}
|
||||||
|
if e.1 >= per_min {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
e.1 += 1;
|
||||||
|
true
|
||||||
|
}
|
||||||
|
/// Seconds until the current minute window rolls over.
|
||||||
|
pub fn retry_after_secs(&self) -> u64 {
|
||||||
|
60 - (std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.unwrap_or_default()
|
||||||
|
.as_secs()
|
||||||
|
% 60)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct AppState {
|
pub struct AppState {
|
||||||
pub live: ArcSwap<Live>,
|
pub live: ArcSwap<Live>,
|
||||||
pub stats: Arc<Stats>,
|
pub stats: Arc<Stats>,
|
||||||
@@ -85,6 +121,8 @@ pub struct AppState {
|
|||||||
/// Managed local CMF model server (install/update/spawn lifecycle + status).
|
/// Managed local CMF model server (install/update/spawn lifecycle + status).
|
||||||
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
|
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
|
||||||
pub cache: Arc<SemanticCache>,
|
pub cache: Arc<SemanticCache>,
|
||||||
|
/// Per-API-key fixed-window rate limiter.
|
||||||
|
pub rate: RateWindows,
|
||||||
pub secrets: SecretStore,
|
pub secrets: SecretStore,
|
||||||
pub config_path: String,
|
pub config_path: String,
|
||||||
pub pipeline: crate::pipeline::Pipeline,
|
pub pipeline: crate::pipeline::Pipeline,
|
||||||
@@ -164,6 +202,7 @@ impl SharedState {
|
|||||||
admin_open_public: std::sync::atomic::AtomicBool::new(false),
|
admin_open_public: std::sync::atomic::AtomicBool::new(false),
|
||||||
cmf,
|
cmf,
|
||||||
cache,
|
cache,
|
||||||
|
rate: RateWindows::default(),
|
||||||
secrets,
|
secrets,
|
||||||
config_path,
|
config_path,
|
||||||
pipeline: crate::pipeline::Pipeline::new(),
|
pipeline: crate::pipeline::Pipeline::new(),
|
||||||
|
|||||||
+112
-9
@@ -137,6 +137,11 @@ struct Inner {
|
|||||||
/// = folded + by_account_alltime, with no double counting: a record is
|
/// = folded + by_account_alltime, with no double counting: a record is
|
||||||
/// either in the file or folded, never both.
|
/// either in the file or folded, never both.
|
||||||
folded: HashMap<String, Totals>,
|
folded: HashMap<String, Totals>,
|
||||||
|
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
|
||||||
|
month_spend: HashMap<String, (String, f64)>,
|
||||||
|
/// Records with ts ≤ this are already reflected in the persisted
|
||||||
|
/// month_spend (watermark against double counting on replay).
|
||||||
|
month_upto_ts: u64,
|
||||||
/// Appends since the last compaction size check.
|
/// Appends since the last compaction size check.
|
||||||
writes_since_check: u32,
|
writes_since_check: u32,
|
||||||
}
|
}
|
||||||
@@ -193,6 +198,7 @@ impl Stats {
|
|||||||
.entry(rec.account.clone())
|
.entry(rec.account.clone())
|
||||||
.or_default()
|
.or_default()
|
||||||
.apply(&rec);
|
.apply(&rec);
|
||||||
|
Self::apply_month(&mut inner, &rec);
|
||||||
if rec.ts >= cutoff {
|
if rec.ts >= cutoff {
|
||||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||||
}
|
}
|
||||||
@@ -200,6 +206,33 @@ impl Stats {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
|
||||||
|
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let m = month_of(rec.ts);
|
||||||
|
let e = inner
|
||||||
|
.month_spend
|
||||||
|
.entry(rec.account.clone())
|
||||||
|
.or_insert((m.clone(), 0.0));
|
||||||
|
if e.0 != m {
|
||||||
|
*e = (m, 0.0);
|
||||||
|
}
|
||||||
|
e.1 += rec.cost_usd;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Month-to-date spend for an account (budget checks).
|
||||||
|
pub fn month_spend(&self, account: &str) -> f64 {
|
||||||
|
let inner = self.inner.lock().unwrap();
|
||||||
|
let m = month_of(now_secs());
|
||||||
|
inner
|
||||||
|
.month_spend
|
||||||
|
.get(account)
|
||||||
|
.filter(|(mm, _)| *mm == m)
|
||||||
|
.map(|(_, v)| *v)
|
||||||
|
.unwrap_or(0.0)
|
||||||
|
}
|
||||||
|
|
||||||
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
|
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
|
||||||
inner.recent.push_back(rec.clone());
|
inner.recent.push_back(rec.clone());
|
||||||
while inner.recent.len() > ring_size {
|
while inner.recent.len() > ring_size {
|
||||||
@@ -256,6 +289,7 @@ impl Stats {
|
|||||||
.entry(rec.account.clone())
|
.entry(rec.account.clone())
|
||||||
.or_default()
|
.or_default()
|
||||||
.apply(&rec);
|
.apply(&rec);
|
||||||
|
Self::apply_month(&mut inner, &rec);
|
||||||
inner.writes_since_check += 1;
|
inner.writes_since_check += 1;
|
||||||
}
|
}
|
||||||
if let Some(path) = &self.file {
|
if let Some(path) = &self.file {
|
||||||
@@ -283,11 +317,42 @@ impl Stats {
|
|||||||
let Ok(content) = std::fs::read_to_string(&path) else {
|
let Ok(content) = std::fs::read_to_string(&path) else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
#[derive(serde::Deserialize)]
|
||||||
|
struct Sidecar {
|
||||||
|
#[serde(default)]
|
||||||
|
by_account: HashMap<String, Totals>,
|
||||||
|
#[serde(default)]
|
||||||
|
month_spend: HashMap<String, (String, f64)>,
|
||||||
|
#[serde(default)]
|
||||||
|
month_upto_ts: u64,
|
||||||
|
}
|
||||||
|
// new format {by_account, month_spend, month_upto_ts}; the original
|
||||||
|
// file held the bare by_account map — accept both
|
||||||
|
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
|
||||||
|
let mut inner = self.inner.lock().unwrap();
|
||||||
|
inner.folded = sc.by_account;
|
||||||
|
inner.month_spend = sc.month_spend;
|
||||||
|
inner.month_upto_ts = sc.month_upto_ts;
|
||||||
|
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||||
self.inner.lock().unwrap().folded = m;
|
self.inner.lock().unwrap().folded = m;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn write_sidecar(&self, inner: &Inner) {
|
||||||
|
let Some(tpath) = self.totals_path() else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let payload = serde_json::json!({
|
||||||
|
"by_account": inner.folded,
|
||||||
|
"month_spend": inner.month_spend,
|
||||||
|
"month_upto_ts": now_secs(),
|
||||||
|
});
|
||||||
|
let tmp = format!("{tpath}.tmp");
|
||||||
|
if std::fs::write(&tmp, payload.to_string()).is_ok() {
|
||||||
|
let _ = std::fs::rename(&tmp, &tpath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
|
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
|
||||||
/// older than the retention window fold into the cumulative per-account
|
/// older than the retention window fold into the cumulative per-account
|
||||||
/// sidecar and are dropped from the file (tmp + rename on both files, so
|
/// sidecar and are dropped from the file (tmp + rename on both files, so
|
||||||
@@ -305,6 +370,10 @@ impl Stats {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
inner.writes_since_check = 0;
|
inner.writes_since_check = 0;
|
||||||
|
// piggyback: persist month-to-date spend so budgets survive
|
||||||
|
// restarts even between compactions
|
||||||
|
inner.month_upto_ts = now_secs();
|
||||||
|
self.write_sidecar(&inner);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
|
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
|
||||||
@@ -359,14 +428,8 @@ impl Stats {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
// persist the folded totals FIRST, then swap the trimmed file in
|
// persist the folded totals FIRST, then swap the trimmed file in
|
||||||
if let Some(tpath) = self.totals_path() {
|
inner.month_upto_ts = now_secs();
|
||||||
if let Ok(json) = serde_json::to_string(&inner.folded) {
|
self.write_sidecar(&inner);
|
||||||
let tmp = format!("{tpath}.tmp");
|
|
||||||
if std::fs::write(&tmp, json).is_ok() {
|
|
||||||
let _ = std::fs::rename(&tmp, &tpath);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
let tmp = format!("{path}.tmp");
|
let tmp = format!("{path}.tmp");
|
||||||
if std::fs::write(&tmp, kept).is_ok() {
|
if std::fs::write(&tmp, kept).is_ok() {
|
||||||
let _ = std::fs::rename(&tmp, path);
|
let _ = std::fs::rename(&tmp, path);
|
||||||
@@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
|
|||||||
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
|
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn month_of(ts: u64) -> String {
|
||||||
|
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
|
||||||
|
let days = ts / 86_400;
|
||||||
|
let mut year = 1970u64;
|
||||||
|
let mut d = days;
|
||||||
|
loop {
|
||||||
|
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||||
|
let len = if leap { 366 } else { 365 };
|
||||||
|
if d < len {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
d -= len;
|
||||||
|
year += 1;
|
||||||
|
}
|
||||||
|
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||||
|
let ml = [
|
||||||
|
31,
|
||||||
|
if leap { 29 } else { 28 },
|
||||||
|
31,
|
||||||
|
30,
|
||||||
|
31,
|
||||||
|
30,
|
||||||
|
31,
|
||||||
|
31,
|
||||||
|
30,
|
||||||
|
31,
|
||||||
|
30,
|
||||||
|
31,
|
||||||
|
];
|
||||||
|
let mut month = 1;
|
||||||
|
for len in ml {
|
||||||
|
if d < len {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
d -= len;
|
||||||
|
month += 1;
|
||||||
|
}
|
||||||
|
format!("{year:04}-{month:02}")
|
||||||
|
}
|
||||||
|
|
||||||
fn avg(sum: u64, n: u64) -> u64 {
|
fn avg(sum: u64, n: u64) -> u64 {
|
||||||
sum.checked_div(n).unwrap_or(0)
|
sum.checked_div(n).unwrap_or(0)
|
||||||
}
|
}
|
||||||
|
|||||||
Vendored
+6
-3
@@ -599,17 +599,20 @@ body.modal-open { overflow: hidden; }
|
|||||||
|
|
||||||
/* Complexity bands slider */
|
/* Complexity bands slider */
|
||||||
.bands-wrap { margin: 4px 0 16px; }
|
.bands-wrap { margin: 4px 0 16px; }
|
||||||
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--panel-2, #23252c); margin: 26px 8px 10px; }
|
/* one-line command chip (install commands etc.) — theme-aware */
|
||||||
|
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
|
||||||
|
|
||||||
|
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
|
||||||
.bands-track.disabled { opacity: .45; pointer-events: none; }
|
.bands-track.disabled { opacity: .45; pointer-events: none; }
|
||||||
.bands-zone { position: absolute; top: 0; bottom: 0; }
|
.bands-zone { position: absolute; top: 0; bottom: 0; }
|
||||||
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
|
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
|
||||||
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
|
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
|
||||||
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
|
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
|
||||||
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
|
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
|
||||||
background: var(--fg, #fff); border: 3px solid var(--accent, #6c6cf5); cursor: grab; touch-action: none; z-index: 2; }
|
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
|
||||||
.bands-handle:active { cursor: grabbing; }
|
.bands-handle:active { cursor: grabbing; }
|
||||||
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
|
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
|
||||||
font-family: var(--mono, monospace); color: var(--fg); background: var(--panel, #1b1d23); padding: 1px 6px;
|
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
|
||||||
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
|
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
|
||||||
.bands-val.raised { top: -46px; }
|
.bands-val.raised { top: -46px; }
|
||||||
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
|
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
|
||||||
|
|||||||
Vendored
+1
@@ -74,6 +74,7 @@ export const api = {
|
|||||||
// protocols
|
// protocols
|
||||||
getProtocols: () => j("GET", "/protocols"),
|
getProtocols: () => j("GET", "/protocols"),
|
||||||
harnesses: () => j("GET", "/harnesses"),
|
harnesses: () => j("GET", "/harnesses"),
|
||||||
|
balance: () => j("GET", "/balance"),
|
||||||
putProtocols: (b) => j("PUT", "/protocols", b),
|
putProtocols: (b) => j("PUT", "/protocols", b),
|
||||||
|
|
||||||
// settings
|
// settings
|
||||||
|
|||||||
Vendored
+12
@@ -634,4 +634,16 @@ export default {
|
|||||||
"tier.low": "leicht",
|
"tier.low": "leicht",
|
||||||
"tier.medium": "mittel",
|
"tier.medium": "mittel",
|
||||||
"tier.high": "schwer",
|
"tier.high": "schwer",
|
||||||
|
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
|
||||||
|
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
|
||||||
|
"settings.cache.entries": "Einträge",
|
||||||
|
"settings.cache.hits": "Treffer",
|
||||||
|
"settings.cache.saved": "gespart",
|
||||||
|
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
|
||||||
|
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
|
||||||
|
"keys.budgetOf": "Budget {b}/Mon.",
|
||||||
|
"dash.kpi.orBalance": "OpenRouter-Guthaben",
|
||||||
|
"keys.h.install": "Installieren:",
|
||||||
|
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
|
||||||
|
"keys.snip.apply": "Schlüssel einsetzen",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+12
@@ -627,4 +627,16 @@ export default {
|
|||||||
"tier.low": "low",
|
"tier.low": "low",
|
||||||
"tier.medium": "medium",
|
"tier.medium": "medium",
|
||||||
"tier.high": "high",
|
"tier.high": "high",
|
||||||
|
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
|
||||||
|
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
|
||||||
|
"settings.cache.entries": "entries",
|
||||||
|
"settings.cache.hits": "hits",
|
||||||
|
"settings.cache.saved": "saved",
|
||||||
|
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
|
||||||
|
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
|
||||||
|
"keys.budgetOf": "budget {b}/mo",
|
||||||
|
"dash.kpi.orBalance": "OpenRouter balance",
|
||||||
|
"keys.h.install": "Install:",
|
||||||
|
"keys.snip.keyInputPh": "paste your API key",
|
||||||
|
"keys.snip.apply": "Insert key",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+12
@@ -634,4 +634,16 @@ export default {
|
|||||||
"tier.low": "fácil",
|
"tier.low": "fácil",
|
||||||
"tier.medium": "medio",
|
"tier.medium": "medio",
|
||||||
"tier.high": "difícil",
|
"tier.high": "difícil",
|
||||||
|
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
|
||||||
|
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
|
||||||
|
"settings.cache.entries": "entradas",
|
||||||
|
"settings.cache.hits": "aciertos",
|
||||||
|
"settings.cache.saved": "ahorrado",
|
||||||
|
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
|
||||||
|
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
|
||||||
|
"keys.budgetOf": "presupuesto {b}/mes",
|
||||||
|
"dash.kpi.orBalance": "Saldo OpenRouter",
|
||||||
|
"keys.h.install": "Instalar:",
|
||||||
|
"keys.snip.keyInputPh": "pega tu clave API",
|
||||||
|
"keys.snip.apply": "Insertar clave",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+12
@@ -634,4 +634,16 @@ export default {
|
|||||||
"tier.low": "facile",
|
"tier.low": "facile",
|
||||||
"tier.medium": "moyen",
|
"tier.medium": "moyen",
|
||||||
"tier.high": "difficile",
|
"tier.high": "difficile",
|
||||||
|
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
|
||||||
|
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
|
||||||
|
"settings.cache.entries": "entrées",
|
||||||
|
"settings.cache.hits": "hits",
|
||||||
|
"settings.cache.saved": "économisé",
|
||||||
|
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
|
||||||
|
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
|
||||||
|
"keys.budgetOf": "budget {b}/mois",
|
||||||
|
"dash.kpi.orBalance": "Solde OpenRouter",
|
||||||
|
"keys.h.install": "Installer :",
|
||||||
|
"keys.snip.keyInputPh": "collez votre clé API",
|
||||||
|
"keys.snip.apply": "Insérer la clé",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+14
-2
@@ -25,7 +25,7 @@ export default {
|
|||||||
"common.all": "все",
|
"common.all": "все",
|
||||||
|
|
||||||
// nav
|
// nav
|
||||||
"nav.dashboard": "Дашборд",
|
"nav.dashboard": "Рабочий стол",
|
||||||
"nav.models": "Модели",
|
"nav.models": "Модели",
|
||||||
"nav.import": "Импорт модели",
|
"nav.import": "Импорт модели",
|
||||||
"nav.routing": "Роутинг",
|
"nav.routing": "Роутинг",
|
||||||
@@ -56,7 +56,7 @@ export default {
|
|||||||
"login.failed": "Неверный токен администратора.",
|
"login.failed": "Неверный токен администратора.",
|
||||||
|
|
||||||
// dashboard
|
// dashboard
|
||||||
"dash.title": "Дашборд",
|
"dash.title": "Рабочий стол",
|
||||||
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
|
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
|
||||||
"dash.clear": "Очистить логи",
|
"dash.clear": "Очистить логи",
|
||||||
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
|
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
|
||||||
@@ -626,4 +626,16 @@ export default {
|
|||||||
"tier.low": "лёгкий",
|
"tier.low": "лёгкий",
|
||||||
"tier.medium": "средний",
|
"tier.medium": "средний",
|
||||||
"tier.high": "сложный",
|
"tier.high": "сложный",
|
||||||
|
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
|
||||||
|
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
|
||||||
|
"settings.cache.entries": "записей",
|
||||||
|
"settings.cache.hits": "попаданий",
|
||||||
|
"settings.cache.saved": "сэкономлено",
|
||||||
|
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
|
||||||
|
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
|
||||||
|
"keys.budgetOf": "бюджет {b}/мес",
|
||||||
|
"dash.kpi.orBalance": "Баланс OpenRouter",
|
||||||
|
"keys.h.install": "Установить:",
|
||||||
|
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
|
||||||
|
"keys.snip.apply": "Вставить ключ",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+12
@@ -634,4 +634,16 @@ export default {
|
|||||||
"tier.low": "kolay",
|
"tier.low": "kolay",
|
||||||
"tier.medium": "orta",
|
"tier.medium": "orta",
|
||||||
"tier.high": "zor",
|
"tier.high": "zor",
|
||||||
|
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
|
||||||
|
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
|
||||||
|
"settings.cache.entries": "kayıt",
|
||||||
|
"settings.cache.hits": "isabet",
|
||||||
|
"settings.cache.saved": "tasarruf",
|
||||||
|
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
|
||||||
|
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
|
||||||
|
"keys.budgetOf": "bütçe {b}/ay",
|
||||||
|
"dash.kpi.orBalance": "OpenRouter bakiyesi",
|
||||||
|
"keys.h.install": "Kurulum:",
|
||||||
|
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
|
||||||
|
"keys.snip.apply": "Anahtarı ekle",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+12
@@ -634,4 +634,16 @@ export default {
|
|||||||
"tier.low": "简单",
|
"tier.low": "简单",
|
||||||
"tier.medium": "中等",
|
"tier.medium": "中等",
|
||||||
"tier.high": "困难",
|
"tier.high": "困难",
|
||||||
|
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
|
||||||
|
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
|
||||||
|
"settings.cache.entries": "条目",
|
||||||
|
"settings.cache.hits": "命中",
|
||||||
|
"settings.cache.saved": "已节省",
|
||||||
|
"keys.form.budget": "每月预算,$(0 = 不限)",
|
||||||
|
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
|
||||||
|
"keys.budgetOf": "预算 {b}/月",
|
||||||
|
"dash.kpi.orBalance": "OpenRouter 余额",
|
||||||
|
"keys.h.install": "安装:",
|
||||||
|
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
|
||||||
|
"keys.snip.apply": "插入密钥",
|
||||||
};
|
};
|
||||||
|
|||||||
Vendored
+6
-1
@@ -238,6 +238,7 @@ export async function renderBenchmark() {
|
|||||||
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
|
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
|
||||||
if (again.tier === "high") {
|
if (again.tier === "high") {
|
||||||
info.tier = again.tier;
|
info.tier = again.tier;
|
||||||
|
info.score = again.score;
|
||||||
info.selected_model = again.selected_model;
|
info.selected_model = again.selected_model;
|
||||||
info.task_label = again.task_label;
|
info.task_label = again.task_label;
|
||||||
info.route_source = again.route_source;
|
info.route_source = again.route_source;
|
||||||
@@ -269,6 +270,8 @@ export async function renderBenchmark() {
|
|||||||
|| String(info.route_source || "").includes("degraded");
|
|| String(info.route_source || "").includes("degraded");
|
||||||
row.task = info.task_label || "—";
|
row.task = info.task_label || "—";
|
||||||
row.tier = info.tier || "—";
|
row.tier = info.tier || "—";
|
||||||
|
row.cscore = info.score || 0; // непрерывный коэффициент сложности
|
||||||
|
// (row.score занят долей правильности ответа из evaluate())
|
||||||
row.chosen = info.selected_model || "—";
|
row.chosen = info.selected_model || "—";
|
||||||
row.source = info.route_source || "—";
|
row.source = info.route_source || "—";
|
||||||
row.expected = test.label; // null = classification not judged
|
row.expected = test.label; // null = classification not judged
|
||||||
@@ -402,7 +405,9 @@ export async function renderBenchmark() {
|
|||||||
: h("span", { class: r.taskOk ? "" : "err" },
|
: h("span", { class: r.taskOk ? "" : "err" },
|
||||||
(r.taskOk ? "✓ " : "✗ ") + r.task,
|
(r.taskOk ? "✓ " : "✗ ") + r.task,
|
||||||
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
|
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
|
||||||
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—"),
|
h("td", {},
|
||||||
|
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||||
|
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
|
||||||
h("td", {},
|
h("td", {},
|
||||||
h("span", { class: "mono small" }, r.chosen || "—"),
|
h("span", { class: "mono small" }, r.chosen || "—"),
|
||||||
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
|
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
|
||||||
|
|||||||
Vendored
+12
-4
@@ -1,6 +1,6 @@
|
|||||||
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
|
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
|
||||||
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
|
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
|
||||||
import { t } from "../i18n.js";
|
import { t, trTier } from "../i18n.js";
|
||||||
import { api } from "../api.js";
|
import { api } from "../api.js";
|
||||||
import { lineChart, barList } from "../charts.js";
|
import { lineChart, barList } from "../charts.js";
|
||||||
import { SITE_URL } from "../app.js";
|
import { SITE_URL } from "../app.js";
|
||||||
@@ -110,12 +110,13 @@ export async function renderDashboard() {
|
|||||||
|
|
||||||
async function load() {
|
async function load() {
|
||||||
const rangeSecs = range;
|
const rangeSecs = range;
|
||||||
let stats, recent, health;
|
let stats, recent, health, balance;
|
||||||
try {
|
try {
|
||||||
[stats, recent, health] = await Promise.all([
|
[stats, recent, health, balance] = await Promise.all([
|
||||||
api.stats(rangeSecs, groupby),
|
api.stats(rangeSecs, groupby),
|
||||||
api.requests(40, 0),
|
api.requests(40, 0),
|
||||||
api.health().catch(() => null),
|
api.health().catch(() => null),
|
||||||
|
api.balance().catch(() => null),
|
||||||
]);
|
]);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
mount(root, h("div", { class: "callout" }, String(e.message || e)));
|
mount(root, h("div", { class: "callout" }, String(e.message || e)));
|
||||||
@@ -162,6 +163,11 @@ export async function renderDashboard() {
|
|||||||
kpi(num(tot.requests), t("dash.kpi.requests")),
|
kpi(num(tot.requests), t("dash.kpi.requests")),
|
||||||
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
|
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
|
||||||
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
|
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
|
||||||
|
balance && balance.openrouter_usd != null
|
||||||
|
? kpi(
|
||||||
|
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
|
||||||
|
t("dash.kpi.orBalance"))
|
||||||
|
: null,
|
||||||
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
|
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
|
||||||
kpi(pct(tot.success_rate), t("dash.kpi.success")),
|
kpi(pct(tot.success_rate), t("dash.kpi.success")),
|
||||||
kpi(num(tot.failovers), t("dash.kpi.failovers"))
|
kpi(num(tot.failovers), t("dash.kpi.failovers"))
|
||||||
@@ -271,7 +277,9 @@ function recentTable(rows) {
|
|||||||
{},
|
{},
|
||||||
h("td", { class: "mono" }, timeAgo(r.ts)),
|
h("td", { class: "mono" }, timeAgo(r.ts)),
|
||||||
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
|
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
|
||||||
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
|
h("td", {},
|
||||||
|
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||||
|
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
|
||||||
h("td", {}, r.task_label || "—"),
|
h("td", {}, r.task_label || "—"),
|
||||||
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
|
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
|
||||||
h("td", { class: "mono" }, money(r.cost_usd)),
|
h("td", { class: "mono" }, money(r.cost_usd)),
|
||||||
|
|||||||
Vendored
+147
-14
@@ -10,12 +10,14 @@ export async function renderKeys() {
|
|||||||
function openCreate(reload) {
|
function openCreate(reload) {
|
||||||
const accountIn = h("input", { placeholder: "team-a" });
|
const accountIn = h("input", { placeholder: "team-a" });
|
||||||
const rateIn = h("input", { type: "number", value: 600 });
|
const rateIn = h("input", { type: "number", value: 600 });
|
||||||
|
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
|
||||||
const customIn = h("input", { placeholder: "sk-gw-…" });
|
const customIn = h("input", { placeholder: "sk-gw-…" });
|
||||||
const body = h(
|
const body = h(
|
||||||
"div",
|
"div",
|
||||||
{},
|
{},
|
||||||
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
|
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
|
||||||
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
|
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
|
||||||
|
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
|
||||||
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
|
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
|
||||||
);
|
);
|
||||||
modal(t("keys.form.title"), body, async () => {
|
modal(t("keys.form.title"), body, async () => {
|
||||||
@@ -24,6 +26,7 @@ export async function renderKeys() {
|
|||||||
key: customIn.value.trim(),
|
key: customIn.value.trim(),
|
||||||
account: accountIn.value.trim() || "default",
|
account: accountIn.value.trim() || "default",
|
||||||
rate_per_min: parseInt(rateIn.value) || 0,
|
rate_per_min: parseInt(rateIn.value) || 0,
|
||||||
|
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
|
||||||
allow_models: [],
|
allow_models: [],
|
||||||
});
|
});
|
||||||
reload();
|
reload();
|
||||||
@@ -51,18 +54,76 @@ export async function renderKeys() {
|
|||||||
function snippetsCard(openMode) {
|
function snippetsCard(openMode) {
|
||||||
const origin = location.origin;
|
const origin = location.origin;
|
||||||
const base = origin + "/v1";
|
const base = origin + "/v1";
|
||||||
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||||
|
// Every scripty snippet exists in two OS flavors: bash heredocs for
|
||||||
|
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
|
||||||
|
// (python/js) and GUI instructions are OS-independent.
|
||||||
|
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
|
||||||
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
|
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
|
||||||
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
|
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
|
||||||
const snips = {
|
const buildSnips = (osKind) => osKind === "ps" ? {
|
||||||
"Claude Code": `# ${t("keys.h.claude")}
|
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
|
||||||
export ANTHROPIC_BASE_URL="${origin}"
|
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
|
||||||
|
$env:ANTHROPIC_MODEL = "cortiq-auto"
|
||||||
|
|
||||||
|
claude`,
|
||||||
|
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
|
||||||
|
Add-Content "$HOME\\.codex\\config.toml" @'
|
||||||
|
model = "cortiq-auto"
|
||||||
|
model_provider = "cortiq"
|
||||||
|
|
||||||
|
[model_providers.cortiq]
|
||||||
|
name = "Cortiq Gateway"
|
||||||
|
base_url = "${base}"
|
||||||
|
env_key = "CORTIQ_API_KEY"
|
||||||
|
wire_api = "chat"
|
||||||
|
'@
|
||||||
|
$env:CORTIQ_API_KEY = "${key}"
|
||||||
|
|
||||||
|
codex`,
|
||||||
|
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
|
||||||
|
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
|
||||||
|
{
|
||||||
|
"$schema": "https://opencode.ai/config.json",
|
||||||
|
"provider": {
|
||||||
|
"cortiq": {
|
||||||
|
"npm": "@ai-sdk/openai-compatible",
|
||||||
|
"name": "Cortiq Gateway",
|
||||||
|
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||||
|
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
'@
|
||||||
|
|
||||||
|
opencode`,
|
||||||
|
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
|
||||||
|
Add-Content "$HOME\\.continue\\config.yaml" @'
|
||||||
|
models:
|
||||||
|
- name: Cortiq Auto
|
||||||
|
provider: openai
|
||||||
|
model: cortiq-auto
|
||||||
|
apiBase: ${base}
|
||||||
|
apiKey: ${key}
|
||||||
|
'@`,
|
||||||
|
Cline: `${t("keys.h.cline1")}
|
||||||
|
API Provider: OpenAI Compatible
|
||||||
|
Base URL: ${base}
|
||||||
|
API Key: ${key}
|
||||||
|
Model ID: cortiq-auto
|
||||||
|
|
||||||
|
${t("keys.h.cline2")}`,
|
||||||
|
curl: `curl.exe ${base}/chat/completions \`
|
||||||
|
-H "Authorization: Bearer ${key}" \`
|
||||||
|
-H "Content-Type: application/json" \`
|
||||||
|
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||||
|
} : {
|
||||||
|
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
|
||||||
export ANTHROPIC_AUTH_TOKEN="${key}"
|
export ANTHROPIC_AUTH_TOKEN="${key}"
|
||||||
export ANTHROPIC_MODEL="cortiq-auto"
|
export ANTHROPIC_MODEL="cortiq-auto"
|
||||||
|
|
||||||
claude`,
|
claude`,
|
||||||
Codex: `# ${t("keys.h.codex")}
|
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||||
mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
|
||||||
model = "cortiq-auto"
|
model = "cortiq-auto"
|
||||||
model_provider = "cortiq"
|
model_provider = "cortiq"
|
||||||
|
|
||||||
@@ -75,8 +136,7 @@ EOF
|
|||||||
export CORTIQ_API_KEY="${key}"
|
export CORTIQ_API_KEY="${key}"
|
||||||
|
|
||||||
codex`,
|
codex`,
|
||||||
OpenCode: `# ${t("keys.h.opencode")}
|
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||||
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
|
||||||
{
|
{
|
||||||
"$schema": "https://opencode.ai/config.json",
|
"$schema": "https://opencode.ai/config.json",
|
||||||
"provider": {
|
"provider": {
|
||||||
@@ -91,8 +151,7 @@ mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
|||||||
EOF
|
EOF
|
||||||
|
|
||||||
opencode`,
|
opencode`,
|
||||||
Continue: `# ${t("keys.h.continue")}
|
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||||
mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
|
||||||
models:
|
models:
|
||||||
- name: Cortiq Auto
|
- name: Cortiq Auto
|
||||||
provider: openai
|
provider: openai
|
||||||
@@ -128,8 +187,29 @@ const resp = await client.chat.completions.create({
|
|||||||
});
|
});
|
||||||
console.log(resp.choices[0].message.content);`,
|
console.log(resp.choices[0].message.content);`,
|
||||||
};
|
};
|
||||||
|
let shSnips, psSnips, snips;
|
||||||
|
function rebuildSnips() {
|
||||||
|
shSnips = buildSnips("sh");
|
||||||
|
psSnips = buildSnips("ps");
|
||||||
|
// python/js/Cline одинаковы — дополняем ps-набор из sh
|
||||||
|
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
|
||||||
|
snips = os === "ps" ? psSnips : shSnips;
|
||||||
|
}
|
||||||
|
rebuildSnips();
|
||||||
let cur = "Claude Code";
|
let cur = "Claude Code";
|
||||||
|
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
|
||||||
|
// in a pasted snippet is not a comment in default interactive zsh, and a
|
||||||
|
// "!" inside it triggers history expansion ("event not found")
|
||||||
|
const HINTS = {
|
||||||
|
"Claude Code": () => t("keys.h.claude"),
|
||||||
|
Codex: () => t("keys.h.codex"),
|
||||||
|
OpenCode: () => t("keys.h.opencode"),
|
||||||
|
Continue: () => t("keys.h.continue"),
|
||||||
|
};
|
||||||
|
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
|
||||||
|
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
|
||||||
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
|
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
|
||||||
|
paintHint();
|
||||||
const seg = h(
|
const seg = h(
|
||||||
"div",
|
"div",
|
||||||
{ class: "seg" },
|
{ class: "seg" },
|
||||||
@@ -137,6 +217,7 @@ console.log(resp.choices[0].message.content);`,
|
|||||||
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
|
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
|
||||||
cur = k;
|
cur = k;
|
||||||
pre.textContent = snips[k];
|
pre.textContent = snips[k];
|
||||||
|
paintHint();
|
||||||
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
|
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
|
||||||
paintStatus();
|
paintStatus();
|
||||||
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
|
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
|
||||||
@@ -164,7 +245,15 @@ console.log(resp.choices[0].message.content);`,
|
|||||||
}).catch(() => {});
|
}).catch(() => {});
|
||||||
|
|
||||||
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
|
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
|
||||||
// status line under the snippet ("installed" / "already configured").
|
// status line under the snippet ("installed" / "already configured") —
|
||||||
|
// and a one-line INSTALL command (per OS) when it's missing.
|
||||||
|
const INSTALL = {
|
||||||
|
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
|
||||||
|
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
|
||||||
|
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
|
||||||
|
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
|
||||||
|
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
|
||||||
|
};
|
||||||
let detected = {};
|
let detected = {};
|
||||||
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
|
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
|
||||||
function paintStatus() {
|
function paintStatus() {
|
||||||
@@ -176,7 +265,19 @@ console.log(resp.choices[0].message.content);`,
|
|||||||
statusHost.innerHTML = "";
|
statusHost.innerHTML = "";
|
||||||
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
|
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
|
||||||
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
|
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
|
||||||
else mount(statusHost, h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")));
|
else {
|
||||||
|
const cmd = (INSTALL[cur] || {})[os];
|
||||||
|
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
|
||||||
|
await navigator.clipboard.writeText(cmd);
|
||||||
|
toast(t("common.copied"), "good");
|
||||||
|
} }, "⧉");
|
||||||
|
mount(statusHost,
|
||||||
|
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
|
||||||
|
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
|
||||||
|
h("span", { class: "muted" }, t("keys.h.install")),
|
||||||
|
h("code", { class: "mono small cmd-chip" }, cmd),
|
||||||
|
copyInstall) : null);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
api.harnesses().then((d) => {
|
api.harnesses().then((d) => {
|
||||||
detected = d || {};
|
detected = d || {};
|
||||||
@@ -187,6 +288,32 @@ console.log(resp.choices[0].message.content);`,
|
|||||||
paintStatus();
|
paintStatus();
|
||||||
}).catch(() => {});
|
}).catch(() => {});
|
||||||
|
|
||||||
|
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
|
||||||
|
const osSeg = h(
|
||||||
|
"div",
|
||||||
|
{ class: "seg" },
|
||||||
|
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
|
||||||
|
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
|
||||||
|
);
|
||||||
|
osSeg.querySelectorAll("button").forEach((b, i) =>
|
||||||
|
b.addEventListener("click", () => {
|
||||||
|
os = i === 0 ? "sh" : "ps";
|
||||||
|
rebuildSnips();
|
||||||
|
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
|
||||||
|
pre.textContent = snips[cur];
|
||||||
|
paintStatus();
|
||||||
|
}));
|
||||||
|
|
||||||
|
// paste-your-key: substitutes the key into every snippet immediately
|
||||||
|
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
|
||||||
|
const applyKey = () => {
|
||||||
|
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||||
|
rebuildSnips();
|
||||||
|
pre.textContent = snips[cur];
|
||||||
|
};
|
||||||
|
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
|
||||||
|
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
|
||||||
|
|
||||||
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||||
copy.addEventListener("click", async () => {
|
copy.addEventListener("click", async () => {
|
||||||
await navigator.clipboard.writeText(pre.textContent);
|
await navigator.clipboard.writeText(pre.textContent);
|
||||||
@@ -198,7 +325,8 @@ console.log(resp.choices[0].message.content);`,
|
|||||||
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
|
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
|
||||||
h("div", { class: "muted small", style: "margin-bottom:8px" },
|
h("div", { class: "muted small", style: "margin-bottom:8px" },
|
||||||
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
|
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
|
||||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), copy),
|
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
|
||||||
|
hintHost,
|
||||||
pre,
|
pre,
|
||||||
statusHost,
|
statusHost,
|
||||||
protoWarn,
|
protoWarn,
|
||||||
@@ -273,7 +401,12 @@ console.log(resp.choices[0].message.content);`,
|
|||||||
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
|
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
|
||||||
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
|
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
|
||||||
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
|
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
|
||||||
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
|
h("td", { class: "right-align" },
|
||||||
|
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
|
||||||
|
k.monthly_budget_usd
|
||||||
|
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
|
||||||
|
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
|
||||||
|
: null),
|
||||||
h(
|
h(
|
||||||
"td",
|
"td",
|
||||||
{ class: "right-align" },
|
{ class: "right-align" },
|
||||||
|
|||||||
Vendored
+24
-1
@@ -194,8 +194,30 @@ export async function renderSettings() {
|
|||||||
const modelsData = await api.listModels();
|
const modelsData = await api.listModels();
|
||||||
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
|
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
|
||||||
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
|
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
|
||||||
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
|
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
|
||||||
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
|
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
|
||||||
|
// Live proof-of-life for the cache card: entries/hits/saved — and an
|
||||||
|
// honest warning when the cache is on but has no embedding model to work
|
||||||
|
// with (it silently does nothing in that state).
|
||||||
|
let cacheStats = null;
|
||||||
|
try {
|
||||||
|
cacheStats = (await api.stats("24h", "model")).cache || null;
|
||||||
|
} catch { /* stats off */ }
|
||||||
|
function cacheStatus() {
|
||||||
|
const wrap = h("div", { style: "margin-bottom:10px" });
|
||||||
|
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
|
||||||
|
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
|
||||||
|
return wrap;
|
||||||
|
}
|
||||||
|
if (cacheStats && s.cache?.enabled) {
|
||||||
|
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
|
||||||
|
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
|
||||||
|
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
|
||||||
|
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
|
||||||
|
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
|
||||||
|
}
|
||||||
|
return wrap;
|
||||||
|
}
|
||||||
const cacheEmbedSel = h(
|
const cacheEmbedSel = h(
|
||||||
"select",
|
"select",
|
||||||
{},
|
{},
|
||||||
@@ -610,6 +632,7 @@ export async function renderSettings() {
|
|||||||
"div",
|
"div",
|
||||||
{ class: "card" },
|
{ class: "card" },
|
||||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
|
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
|
||||||
|
cacheStatus(),
|
||||||
cacheEnabled.node,
|
cacheEnabled.node,
|
||||||
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
|
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
|
||||||
field(t("settings.cache.embed"), cacheEmbedSel),
|
field(t("settings.cache.embed"), cacheEmbedSel),
|
||||||
|
|||||||
Reference in New Issue
Block a user