Compare commits

..
11 Commits
Author SHA1 Message Date
infosave2007 1c59906b49 chore: bump v0.2.43
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 16:12:17 +03:00
infosave2007 19f666e7b1 feat(cmf): auto-detect tool support of managed local servers
cortiq serve >=0.5.62 advertises the loaded model's capabilities in
/healthz ({"capabilities":{"tools":true}} when the chat template has a
tools branch). The gateway now reads that handshake when the managed
server becomes healthy and treats detected tool support as equivalent
to declared caps in the candidate filter — so a low-tier chain made of
a local model no longer widens agent (tools) traffic to cloud models
once the local server can actually drive tools.

Also adds an optional manual "caps" override to [[cmf.servers]] for
older serve builds that do not advertise capabilities.
2026-08-08 15:52:20 +03:00
infosave2007 dbcd9827f6 fix(pipeline): capability filtering widens the chain instead of killing the request
OpenCode (like every coding agent) sends tool definitions with each request.
The router rated the prompt low → the low-tier chain contained only the
local model → the tools filter emptied the chain → hard failure: "No
healthy model available that satisfies required capabilities", with the
agent stuck in a retry loop.

On an AUTO route whose filtered chain comes up empty, the pipeline now
widens to any pool model satisfying the required capabilities — preferring
the routing table's own medium→high order (an agentic request deserves a
capable model), then the rest of the pool — and logs the widening. Pinned
requests still fail loudly as before. Verified with OpenCode's exact
request shape: cortiq-auto + tools at tier low now answers from
deepseek in 1.2s instead of erroring.
2026-08-08 14:25:00 +03:00
infosave2007 b689180717 fix(keys): snippets survive being pasted into real zsh
The copyable text opened with a "# hint" line — but default interactive
zsh has interactivecomments off, so the line was parsed as a command, and
the "!" inside «(перезапишет!)» fired history expansion: pasting the
OpenCode snippet died with "zsh: event not found: )" before doing
anything. Hints now live in the UI above the snippet; the clipboard gets
commands only. Verified by piping the actual rendered snippet through an
interactive zsh: the config file lands intact.
2026-08-08 14:16:22 +03:00
infosave2007 f385c2254b fix(ui): install-command chip and bands slider readable in the light theme
The chip used var(--panel-2, #23252c) — a variable that doesn't exist with
a hard dark fallback, so in the light theme the install command was a dark
box with dark text. Same class of bug in the bands slider track and handle
labels (invented --panel-2/--fg vars). All moved to the real theme
variables (--panel/--border/--text) via a .cmd-chip class; verified by
computed styles in the light theme (text rgb(15,23,42) on white) and
screenshots of both views.
2026-08-08 14:12:39 +03:00
infosave2007 9607a38792 feat(keys): paste-your-key field — snippets carry the real key instantly
Next to the copy button: a key input and an «Insert key» button (Enter works
too). The pasted key substitutes into every harness/SDK snippet immediately
and survives tab and OS switches; a freshly created key pre-fills the field.
No more copying a snippet and then hand-editing the placeholder. 2 strings
×7 languages.
2026-08-08 14:07:11 +03:00
infosave2007 239ddc767c feat(keys): harness snippets for every OS + install command when missing
The harness auto-setup scripts were bash-only (heredocs, mkdir -p) — useless
on Windows. Every scripty snippet now exists in two flavors behind a
🐧🍎/🪟 switch (auto-detected from the browser): bash heredocs for
Linux/macOS and PowerShell here-strings ($env:, New-Item, @'..'@,
curl.exe with backtick continuations) for Windows. SDK snippets and GUI
instructions stay OS-independent.

And when detection says a harness is NOT installed on the gateway machine,
the status line now offers its one-line install command for the selected OS
(claude.ai install.sh / install.ps1, npm -g for Codex and OpenCode-on-
Windows, code --install-extension for Continue/Cline) with a copy button.
1 string ×7 languages.
2026-08-08 14:05:47 +03:00
infosave2007 057abaa95a feat!: API keys actually gate the gateway — auth, attribution, rate limits, budgets
The office-readiness audit found the foundation missing: configured API keys
were decorative. Any bearer — or none — passed every model endpoint, per-key
rate limits existed only in the form, and all real traffic was recorded as
"anonymous", leaving the per-key spend table permanently blank.

A middleware on the model surface (/v1/*, /route, /mcp) now enforces keys:
- no keys configured → open mode as before (the console already warns);
- keys exist → Bearer or x-api-key must match (401 invalid_api_key);
- per-key rate_per_min → fixed one-minute window, 429 with Retry-After;
- NEW per-key monthly_budget_usd → 429 insufficient_quota once the account's
  month-to-date spend crosses the ceiling;
- the matched account rides the request into stats — the keys table's spend
  columns finally see real traffic.

Month-to-date spend is tracked per account in stats and persisted in the
totals sidecar with a watermark (no double counting across restarts). The
key dialog gains a budget field; the table shows "budget $N/mo" in red once
exhausted. All verified live: 401 wrong/missing key, 200 + attribution with
the right key, 429 + Retry-After over the limit, 429 insufficient_quota on
the second request past a $0.000001 budget.

Also: the dashboard shows the OpenRouter credit balance (GET
/api/v1/credits via the stored key, cached 5 min, hidden without a key, red
under $5 — live: $16.18), the server drains connections on Ctrl-C instead
of dropping them, request bodies are capped at 32 MB, and docs/DEPLOY(.ru)
cover office deployment: systemd/launchd/Windows service, nginx/caddy TLS,
security checklist, backups. 10 strings ×7 languages.
2026-08-08 14:01:16 +03:00
infosave2007 dd864eed22 fix(cache): honest settings card — clean threshold, live stats, no-embedder warning
Full audit of the semantic cache against a live instance (mock embedder,
identical requests): the mechanism works — miss at 3.1s, then hits at 2ms
with entries/hits/saved-$ counting. Two honest gaps fixed:

- the threshold round-tripped through f32 and rendered as
  0.9200000166893005 in both the TOML and the settings input — the config
  field is f64 now (writes exactly what you typed) and the input rounds
  legacy values to 2 decimals;
- the card now shows proof of life (entries / hits / hit-rate / saved $)
  and, when the cache is enabled with NO embedding model available, a
  warning that it is silently doing nothing and how to fix it — previously
  that state looked identical to a working cache.

Also audited in the same pass, no changes needed: Prometheus /metrics
(toggle on/off applies live, 404 when off), and all 7 protocol switches
(each gates its endpoint instantly both ways). 4 strings ×7 languages.
2026-08-08 13:21:43 +03:00
infosave2007 c1ff675d80 chore: release 0.2.42
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 12:18:04 +03:00
infosave2007 9ef1af3ae1 feat(ui): the continuous complexity score shows next to the tier
Benchmark (routing mode) and the dashboard's recent-requests list now print
the router's raw 0..1 score under/next to the tier badge — people can see
what different prompts actually weigh and tune the boundary sliders from
real data. Tooltip explains the link to the Routing sliders (7 languages).

Caught before commit: naming the field row.score collided with the answer-
correctness fraction from evaluate() and dragged the benchmark's quality KPI
down to the average complexity — renamed to cscore, KPI verified back at
100%. Dashboard tier badges also render translated names now, and the
Russian locale calls the dashboard «Рабочий стол».
2026-08-08 12:11:07 +03:00
34 changed files with 955 additions and 54 deletions
+50
View File
@@ -3,6 +3,56 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.43] - 2026-08-08
### Added
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
configured = open mode, as before.
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
when the server becomes healthy, so a local model whose chat template can drive tool
calling serves agent (tools) traffic with zero manual configuration. Manual override:
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
remaining credit is a KPI card, red under $5.
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
per harness, an install command chip when the harness is missing, and a paste-your-key
field that substitutes the real key into every snippet.
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
Caddy TLS fronting, security checklist, backup list.
### Fixed
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
route's chain has no model with the required capabilities (tools), the gateway escalates
through medium→high chains and then the rest of the pool instead of returning
"No healthy model available". Explicit pins still fail loudly.
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
no history-expansion `!`), and the install chips are readable in the light theme.
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
into the sidecar totals file — per-account all-time and month spend survive compaction
and restarts.
### Breaking
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
will now be rejected with 401 — either remove the keys from the config or start sending
`Authorization: Bearer <key>`.
## [0.2.42] - 2026-08-08
### Added
- **The continuous complexity score is visible**: the benchmark's routing table and the
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
see what different prompts actually weigh and tune the boundary sliders from real traffic.
### Changed
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
«Рабочий стол».
## [0.2.41] - 2026-08-08
### Added
Generated
+1 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "cortiq-gateway"
version = "0.2.41"
version = "0.2.43"
dependencies = [
"anyhow",
"arc-swap",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "cortiq-gateway"
version = "0.2.41"
version = "0.2.43"
edition = "2021"
rust-version = "1.75"
license = "Apache-2.0"
+2
View File
@@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
---
## 🖥️ Admin console
+2
View File
@@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
---
## 🖥️ Панель управления
+122
View File
@@ -0,0 +1,122 @@
# Deploying cortiq-gateway for a team / office
One gateway on a small server, API keys per department, local models for the
cheap tiers, cloud models for the hard ones. This page covers the ops side.
## 1. Run as a service
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Task Scheduler → Create Task → "Run whether user is logged on or not",
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS and the outside world
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
and bind the gateway to localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE streaming
proxy_read_timeout 3600s; # long generations
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
same with automatic certificates.)
## 3. Security checklist
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
the console warns that it is open to the network.
- **API keys**: create one per department/developer on the keys page. With at
least one key configured the model APIs require Bearer / x-api-key —
requests without a valid key get 401.
- **Rate limits**: per-key requests/min → 429 with Retry-After.
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
account's month-to-date spend crosses it.
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
## 4. Backups
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
- `secrets.toml` — provider API keys (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
cumulative per-key totals and month-to-date budget spend.
Copy those three and you can rebuild the instance anywhere.
## 5. Observability
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
OpenRouter balance (red under $5).
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
+123
View File
@@ -0,0 +1,123 @@
# Развёртывание cortiq-gateway для команды / офиса
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
## 1. Запуск как сервис
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
--no-port-search`. Либо [NSSM](https://nssm.cc):
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS и внешний мир
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
привяжите к localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE-стриминг
proxy_read_timeout 3600s; # долгие генерации
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
с автоматическими сертификатами.)
## 3. Чек-лист безопасности
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
него консоль предупреждает, что открыта сети.
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
запросы без валидного ключа получают 401.
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
траты аккаунта за месяц превысят лимит.
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
## 4. Бэкапы
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
- `secrets.toml` — ключи провайдеров (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
тоталы по ключам и месячные траты для бюджетов.
Скопируйте эти три файла — и инстанс поднимается где угодно.
## 5. Наблюдаемость
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
сложности, баланс OpenRouter (красный ниже $5).
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
+51
View File
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
get(get_protocols).put(put_protocols),
)
.route("/admin/api/harnesses", get(get_harnesses))
.route("/admin/api/balance", get(get_balance))
.route("/admin/api/settings", get(get_settings).put(put_settings))
.route("/admin/api/keys", get(list_keys).post(create_key))
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
@@ -1008,6 +1009,51 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
ok(json!({ "ok": true }))
}
// ─────────────────────────────── balance ───────────────────────────────────
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
// (null) when no OpenRouter key is configured — the card hides itself.
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
std::sync::OnceLock::new();
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
if let Some((at, v)) = *cache.lock().unwrap() {
if at.elapsed() < std::time::Duration::from_secs(300) {
return ok(json!({ "openrouter_usd": v }));
}
}
let key = state
.secrets
.resolve("OPENROUTER_API_KEY")
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
let Some(key) = key.filter(|k| !k.is_empty()) else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(8))
.build()
.unwrap_or_default();
let resp = client
.get("https://openrouter.ai/api/v1/credits")
.bearer_auth(&key)
.send()
.await;
let balance = match resp {
Ok(r) if r.status().is_success() => {
let v: serde_json::Value = r.json().await.unwrap_or_default();
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
Some(total - used)
}
_ => None,
};
let Some(balance) = balance else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
ok(json!({ "openrouter_usd": balance }))
}
// ──────────────────────────── harness detection ────────────────────────────
// Which coding harnesses are present on the machine the gateway runs on, so the
// API-keys view can mark tabs "installed" / "already configured for cortiq".
@@ -1357,6 +1403,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
"key_masked": mask_key(&k.key),
"account": k.account,
"rate_per_min": k.rate_per_min,
"monthly_budget_usd": k.monthly_budget_usd,
"allow_models": k.allow_models,
})
})
@@ -1372,6 +1419,8 @@ struct NewKey {
#[serde(default)]
rate_per_min: u32,
#[serde(default)]
monthly_budget_usd: Option<f64>,
#[serde(default)]
allow_models: Vec<String>,
}
@@ -1390,6 +1439,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
account: b.account,
rate_per_min: b.rate_per_min,
allow_models: b.allow_models,
monthly_budget_usd: b.monthly_budget_usd,
});
state.reload(cfg)?;
// show the full key only once
@@ -2147,6 +2197,7 @@ fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<
o1_window: None,
o1_sink: None,
skip_mtp: false,
caps: Vec::new(),
});
Some(id)
}
+1 -1
View File
@@ -55,7 +55,7 @@ impl SemanticCache {
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
Arc::new(Self {
enabled: cfg.enabled,
threshold: cfg.threshold,
threshold: cfg.threshold as f32,
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
max_entries: cfg.max_entries.max(1),
embed_model: cfg.embed_model.clone(),
+27 -5
View File
@@ -56,6 +56,9 @@ struct Managed {
pub struct CmfRuntime {
status: Mutex<CmfStatus>,
children: Mutex<Vec<Managed>>,
/// Server ids whose healthz handshake advertised tool support — lets the
/// pipeline route tools traffic to local models with no manual caps.
pub detected_tools: Mutex<std::collections::HashSet<String>>,
}
impl CmfRuntime {
@@ -63,6 +66,7 @@ impl CmfRuntime {
Arc::new(Self {
status: Mutex::new(CmfStatus::default()),
children: Mutex::new(Vec::new()),
detected_tools: Mutex::new(std::collections::HashSet::new()),
})
}
@@ -193,6 +197,7 @@ impl Default for CmfRuntime {
Self {
status: Mutex::new(CmfStatus::default()),
children: Mutex::new(Vec::new()),
detected_tools: Mutex::new(std::collections::HashSet::new()),
}
}
}
@@ -332,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
.map_err(|e| e.to_string())
}
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
/// Poll `/healthz` until the local server answers 200 (or the deadline
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
/// caller can register them; older servers return `None` there.
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
let url = format!("http://{host}:{port}/healthz");
let client = reqwest::Client::new();
for _ in 0..tries {
@@ -344,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
.await
{
if r.status().is_success() {
return true;
let tools = r
.json::<serde_json::Value>()
.await
.ok()
.and_then(|v| v["capabilities"]["tools"].as_bool());
return (true, tools);
}
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
false
(false, None)
}
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
@@ -432,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
if !rt.is_running(&server.id) {
continue; // failed to spawn / file missing
}
if wait_healthy(&cfg.local_host, server.port, 240).await {
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
if ok {
rt.set_healthy(&server.id, true);
// capability handshake: the server said whether its model's chat
// template can drive tool calling — no manual caps needed
if tools == Some(true) {
rt.detected_tools.lock().unwrap().insert(server.id.clone());
tracing::info!(id = %server.id, "local server advertises tool support");
} else {
rt.detected_tools.lock().unwrap().remove(&server.id);
}
rt.log(format!(
"local CMF server ready — '{}' at http://{}:{}/v1",
server.id, cfg.local_host, server.port
+12 -3
View File
@@ -130,6 +130,10 @@ pub struct CmfServer {
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
#[serde(default)]
pub skip_mtp: bool,
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
/// gateway auto-detects tool support from the server's healthz handshake.
#[serde(default)]
pub caps: Vec<String>,
}
/// CMF-format model factory: import a HuggingFace model, convert it to a
@@ -229,6 +233,7 @@ impl CmfCfg {
o1_window: None,
o1_sink: None,
skip_mtp: false,
caps: Vec::new(),
}];
}
Vec::new()
@@ -620,6 +625,9 @@ pub struct ApiKeyCfg {
pub account: String,
#[serde(default)]
pub rate_per_min: u32,
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub monthly_budget_usd: Option<f64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub allow_models: Vec<String>,
}
@@ -829,9 +837,10 @@ fn default_shadow_soak() -> usize {
pub struct CacheCfg {
#[serde(default)]
pub enabled: bool,
/// Cosine-similarity threshold for a hit (0..1).
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
/// the number you typed (an f32 round-trips as 0.9200000166893005).
#[serde(default = "default_semcache_threshold")]
pub threshold: f32,
pub threshold: f64,
/// Entry time-to-live, e.g. `1h`.
#[serde(default = "default_semcache_ttl")]
pub ttl: String,
@@ -853,7 +862,7 @@ impl Default for CacheCfg {
}
}
}
fn default_semcache_threshold() -> f32 {
fn default_semcache_threshold() -> f64 {
0.92
}
fn default_semcache_ttl() -> String {
+7
View File
@@ -13,6 +13,8 @@ pub enum GatewayError {
Unauthorized(String),
#[error("rate limit exceeded")]
RateLimited { retry_after_secs: u64 },
#[error("monthly budget exhausted (limit ${budget_usd})")]
BudgetExhausted { budget_usd: f64 },
#[error("upstream unavailable: {0}")]
UpstreamUnavailable(String),
#[error("internal error: {0}")]
@@ -37,6 +39,11 @@ impl GatewayError {
"rate_limit_error",
"rate_limited",
),
Self::BudgetExhausted { .. } => (
StatusCode::TOO_MANY_REQUESTS,
"insufficient_quota",
"insufficient_quota",
),
Self::UpstreamUnavailable(_) => (
StatusCode::BAD_GATEWAY,
"upstream_unavailable",
+15 -3
View File
@@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
.route("/health", get(|| async { "ok" }))
.route("/readyz", get(|| async { "ready" }))
.route("/metrics", get(admin::metrics))
.merge(protocols::build_router());
.merge(protocols::build_router(state.clone()));
if admin_enabled {
app = app
@@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> {
.allow_methods(tower_http::cors::Any)
.allow_headers(tower_http::cors::Any);
let app = app.with_state(state).layer(cors);
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
// small enough that a hostile client can't balloon memory
let app =
app.with_state(state)
.layer(cors)
.layer(tower_http::limit::RequestBodyLimitLayer::new(
32 * 1024 * 1024,
));
let listener = {
let mut attempt = 0u32;
@@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("open this URL to finish setup: {url}");
}
}
axum::serve(listener, app).await?;
axum::serve(listener, app)
.with_graceful_shutdown(async {
let _ = tokio::signal::ctrl_c().await;
tracing::info!("shutdown signal received — draining connections");
})
.await?;
Ok(())
}
+50 -4
View File
@@ -183,6 +183,9 @@ impl Pipeline {
&self,
req: &ChatRequest,
live: &crate::state::Live,
// Tool support detected via the local servers' healthz handshake —
// treated as equivalent to declared caps when filtering candidates.
detected_tools: &std::collections::HashSet<String>,
) -> Result<(RouteDecision, String, Vec<String>)> {
// 1. Extract text for classification
let text = self.extract_text(
@@ -274,10 +277,13 @@ impl Pipeline {
// Filter candidates by capabilities (e.g. tools support)
let needs_tools = !req.tools.is_empty();
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
prov.caps().tools || detected_tools.contains(id)
};
let mut final_candidates = Vec::new();
for model_id in candidates {
if let Some(prov) = live.registry.get(&model_id) {
if needs_tools && !prov.caps().tools {
if needs_tools && !has_tools(&model_id, &prov) {
tracing::debug!(
"Candidate '{}' skipped: lacks support for requested tools",
model_id
@@ -306,13 +312,49 @@ impl Pipeline {
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
for id in live.local_candidates() {
if let Some(prov) = live.registry.get(&id) {
if needs_tools && !prov.caps().tools {
if needs_tools && !has_tools(&id, &prov) {
continue;
}
final_candidates.push(id);
}
}
}
// Capability widening: a tier chain built of tools-less local models
// must not kill an agent request. When the filtered chain is empty on
// an AUTO route, widen to any pool model that satisfies the required
// capabilities — preferring the routing table's own medium→high order
// (an agentic request is worth a capable model), then the rest of the
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
// this every low-tier verdict was a hard failure.
if final_candidates.is_empty() {
let mut widened: Vec<String> = Vec::new();
for tier in ["medium", "high"] {
for id in live.routing.candidates(tier) {
if !widened.contains(&id) {
widened.push(id);
}
}
}
for id in live.registry.ids() {
if !widened.contains(id) {
widened.push(id.clone());
}
}
for id in widened {
if let Some(prov) = live.registry.get(&id) {
if needs_tools && !has_tools(&id, &prov) {
continue;
}
final_candidates.push(id);
}
}
if !final_candidates.is_empty() {
tracing::info!(
chosen = %final_candidates[0],
"capability widening: tier chain had no model with the required capabilities"
);
}
}
if final_candidates.is_empty() {
return Err(GatewayError::UpstreamUnavailable(
"No healthy model available that satisfies required capabilities (e.g. tools)"
@@ -324,7 +366,9 @@ impl Pipeline {
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
let live = state.live();
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
let (decision, route_source, mut final_candidates) =
self.resolve(&req, &live, &detected_tools).await?;
// Serving-when-promoted (gated): a promoted task-label is served by the
// local model FIRST (failover to cloud on error). Per-request VETO uses
@@ -471,7 +515,9 @@ impl Pipeline {
.to_string();
let live = state.live();
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
let (decision, route_source, final_candidates) =
match self.resolve(&req, &live, &detected_tools).await {
Ok(t) => t,
Err(e) => {
record_error(state, &account, &protocol, &directive, started, &e);
+2
View File
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<MessagesRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.anthropic_messages {
@@ -108,6 +109,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "anthropic_messages".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
+51 -1
View File
@@ -12,10 +12,59 @@ pub mod openai_models;
use crate::state::SharedState;
use axum::Router;
/// Account resolved by the API-key middleware, consumed by protocol handlers
/// for attribution (stats, budgets). Present only when keys are configured.
#[derive(Clone)]
pub struct AccountTag(pub String);
/// API-key gate for the model-facing surface. Open mode (no keys configured)
/// passes everything through — the admin console already warns about it.
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
/// enforced with a fixed one-minute window (429 + Retry-After), and a
/// monthly budget, when set, turns into 429 insufficient_quota once the
/// account's month-to-date spend crosses it.
async fn api_auth(
axum::extract::State(state): axum::extract::State<SharedState>,
mut req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
use axum::response::IntoResponse;
let live = state.live();
if live.cfg.api_keys.is_empty() {
return next.run(req).await;
}
let provided = req
.headers()
.get(axum::http::header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|v| v.strip_prefix("Bearer "))
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
.map(str::trim)
.unwrap_or("");
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
.into_response();
};
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
return crate::error::GatewayError::RateLimited {
retry_after_secs: state.rate.retry_after_secs(),
}
.into_response();
}
if let Some(budget) = key.monthly_budget_usd {
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
.into_response();
}
}
req.extensions_mut().insert(AccountTag(key.account.clone()));
next.run(req).await
}
/// Build the protocols router. Implemented adapters are **always** mounted;
/// each handler checks the live `protocols.*` flag from config itself, so
/// toggles in the admin panel take effect without a restart (hot switching).
pub fn build_router() -> Router<SharedState> {
pub fn build_router(state: SharedState) -> Router<SharedState> {
Router::new()
.merge(openai_chat::routes())
.merge(openai_completions::routes())
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
.merge(anthropic_messages::routes())
.merge(mcp::routes())
.merge(native_passthrough::routes())
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
}
+2
View File
@@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<OpenAiChatRequest>,
) -> Result<Response> {
// hot protocol toggle: if the adapter is disabled in config — return 404
@@ -104,6 +105,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_chat".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
+2
View File
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<CompletionsRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.openai_completions {
@@ -89,6 +90,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_completions".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
+1 -1
View File
@@ -48,7 +48,7 @@ impl Registry {
price_out: 0.0,
kind: "chat".into(),
api_key_env: None,
caps: Vec::new(),
caps: s.caps.clone(),
temperature: s.temperature,
top_p: s.top_p,
max_tokens: s.max_tokens,
+39
View File
@@ -66,6 +66,42 @@ impl Live {
}
}
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
/// fine for office-grade fairness and costs one HashMap lookup per request.
#[derive(Default)]
pub struct RateWindows {
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
}
impl RateWindows {
pub fn allow(&self, key: &str, per_min: u32) -> bool {
let minute = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
/ 60;
let mut g = self.inner.lock().unwrap();
let e = g.entry(key.to_string()).or_insert((minute, 0));
if e.0 != minute {
*e = (minute, 0);
}
if e.1 >= per_min {
return false;
}
e.1 += 1;
true
}
/// Seconds until the current minute window rolls over.
pub fn retry_after_secs(&self) -> u64 {
60 - (std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
% 60)
}
}
pub struct AppState {
pub live: ArcSwap<Live>,
pub stats: Arc<Stats>,
@@ -85,6 +121,8 @@ pub struct AppState {
/// Managed local CMF model server (install/update/spawn lifecycle + status).
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
pub cache: Arc<SemanticCache>,
/// Per-API-key fixed-window rate limiter.
pub rate: RateWindows,
pub secrets: SecretStore,
pub config_path: String,
pub pipeline: crate::pipeline::Pipeline,
@@ -164,6 +202,7 @@ impl SharedState {
admin_open_public: std::sync::atomic::AtomicBool::new(false),
cmf,
cache,
rate: RateWindows::default(),
secrets,
config_path,
pipeline: crate::pipeline::Pipeline::new(),
+112 -9
View File
@@ -137,6 +137,11 @@ struct Inner {
/// = folded + by_account_alltime, with no double counting: a record is
/// either in the file or folded, never both.
folded: HashMap<String, Totals>,
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
month_spend: HashMap<String, (String, f64)>,
/// Records with ts ≤ this are already reflected in the persisted
/// month_spend (watermark against double counting on replay).
month_upto_ts: u64,
/// Appends since the last compaction size check.
writes_since_check: u32,
}
@@ -193,6 +198,7 @@ impl Stats {
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
if rec.ts >= cutoff {
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
}
@@ -200,6 +206,33 @@ impl Stats {
}
}
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
return;
}
let m = month_of(rec.ts);
let e = inner
.month_spend
.entry(rec.account.clone())
.or_insert((m.clone(), 0.0));
if e.0 != m {
*e = (m, 0.0);
}
e.1 += rec.cost_usd;
}
/// Month-to-date spend for an account (budget checks).
pub fn month_spend(&self, account: &str) -> f64 {
let inner = self.inner.lock().unwrap();
let m = month_of(now_secs());
inner
.month_spend
.get(account)
.filter(|(mm, _)| *mm == m)
.map(|(_, v)| *v)
.unwrap_or(0.0)
}
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
inner.recent.push_back(rec.clone());
while inner.recent.len() > ring_size {
@@ -256,6 +289,7 @@ impl Stats {
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
inner.writes_since_check += 1;
}
if let Some(path) = &self.file {
@@ -283,11 +317,42 @@ impl Stats {
let Ok(content) = std::fs::read_to_string(&path) else {
return;
};
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
#[derive(serde::Deserialize)]
struct Sidecar {
#[serde(default)]
by_account: HashMap<String, Totals>,
#[serde(default)]
month_spend: HashMap<String, (String, f64)>,
#[serde(default)]
month_upto_ts: u64,
}
// new format {by_account, month_spend, month_upto_ts}; the original
// file held the bare by_account map — accept both
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
let mut inner = self.inner.lock().unwrap();
inner.folded = sc.by_account;
inner.month_spend = sc.month_spend;
inner.month_upto_ts = sc.month_upto_ts;
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
self.inner.lock().unwrap().folded = m;
}
}
fn write_sidecar(&self, inner: &Inner) {
let Some(tpath) = self.totals_path() else {
return;
};
let payload = serde_json::json!({
"by_account": inner.folded,
"month_spend": inner.month_spend,
"month_upto_ts": now_secs(),
});
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, payload.to_string()).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
/// older than the retention window fold into the cumulative per-account
/// sidecar and are dropped from the file (tmp + rename on both files, so
@@ -305,6 +370,10 @@ impl Stats {
return;
}
inner.writes_since_check = 0;
// piggyback: persist month-to-date spend so budgets survive
// restarts even between compactions
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
}
}
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
@@ -359,14 +428,8 @@ impl Stats {
}
}
// persist the folded totals FIRST, then swap the trimmed file in
if let Some(tpath) = self.totals_path() {
if let Ok(json) = serde_json::to_string(&inner.folded) {
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
}
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
let tmp = format!("{path}.tmp");
if std::fs::write(&tmp, kept).is_ok() {
let _ = std::fs::rename(&tmp, path);
@@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
}
fn month_of(ts: u64) -> String {
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
let days = ts / 86_400;
let mut year = 1970u64;
let mut d = days;
loop {
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let len = if leap { 366 } else { 365 };
if d < len {
break;
}
d -= len;
year += 1;
}
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let ml = [
31,
if leap { 29 } else { 28 },
31,
30,
31,
30,
31,
31,
30,
31,
30,
31,
];
let mut month = 1;
for len in ml {
if d < len {
break;
}
d -= len;
month += 1;
}
format!("{year:04}-{month:02}")
}
fn avg(sum: u64, n: u64) -> u64 {
sum.checked_div(n).unwrap_or(0)
}
+6 -3
View File
@@ -599,17 +599,20 @@ body.modal-open { overflow: hidden; }
/* Complexity bands slider */
.bands-wrap { margin: 4px 0 16px; }
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--panel-2, #23252c); margin: 26px 8px 10px; }
/* one-line command chip (install commands etc.) — theme-aware */
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
.bands-track.disabled { opacity: .45; pointer-events: none; }
.bands-zone { position: absolute; top: 0; bottom: 0; }
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
background: var(--fg, #fff); border: 3px solid var(--accent, #6c6cf5); cursor: grab; touch-action: none; z-index: 2; }
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
.bands-handle:active { cursor: grabbing; }
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
font-family: var(--mono, monospace); color: var(--fg); background: var(--panel, #1b1d23); padding: 1px 6px;
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
.bands-val.raised { top: -46px; }
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
+1
View File
@@ -74,6 +74,7 @@ export const api = {
// protocols
getProtocols: () => j("GET", "/protocols"),
harnesses: () => j("GET", "/harnesses"),
balance: () => j("GET", "/balance"),
putProtocols: (b) => j("PUT", "/protocols", b),
// settings
+12
View File
@@ -634,4 +634,16 @@ export default {
"tier.low": "leicht",
"tier.medium": "mittel",
"tier.high": "schwer",
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
"settings.cache.entries": "Einträge",
"settings.cache.hits": "Treffer",
"settings.cache.saved": "gespart",
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
"keys.budgetOf": "Budget {b}/Mon.",
"dash.kpi.orBalance": "OpenRouter-Guthaben",
"keys.h.install": "Installieren:",
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
"keys.snip.apply": "Schlüssel einsetzen",
};
+12
View File
@@ -627,4 +627,16 @@ export default {
"tier.low": "low",
"tier.medium": "medium",
"tier.high": "high",
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
"settings.cache.entries": "entries",
"settings.cache.hits": "hits",
"settings.cache.saved": "saved",
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
"keys.budgetOf": "budget {b}/mo",
"dash.kpi.orBalance": "OpenRouter balance",
"keys.h.install": "Install:",
"keys.snip.keyInputPh": "paste your API key",
"keys.snip.apply": "Insert key",
};
+12
View File
@@ -634,4 +634,16 @@ export default {
"tier.low": "fácil",
"tier.medium": "medio",
"tier.high": "difícil",
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
"settings.cache.entries": "entradas",
"settings.cache.hits": "aciertos",
"settings.cache.saved": "ahorrado",
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
"keys.budgetOf": "presupuesto {b}/mes",
"dash.kpi.orBalance": "Saldo OpenRouter",
"keys.h.install": "Instalar:",
"keys.snip.keyInputPh": "pega tu clave API",
"keys.snip.apply": "Insertar clave",
};
+12
View File
@@ -634,4 +634,16 @@ export default {
"tier.low": "facile",
"tier.medium": "moyen",
"tier.high": "difficile",
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
"settings.cache.entries": "entrées",
"settings.cache.hits": "hits",
"settings.cache.saved": "économisé",
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
"keys.budgetOf": "budget {b}/mois",
"dash.kpi.orBalance": "Solde OpenRouter",
"keys.h.install": "Installer :",
"keys.snip.keyInputPh": "collez votre clé API",
"keys.snip.apply": "Insérer la clé",
};
+14 -2
View File
@@ -25,7 +25,7 @@ export default {
"common.all": "все",
// nav
"nav.dashboard": "Дашборд",
"nav.dashboard": "Рабочий стол",
"nav.models": "Модели",
"nav.import": "Импорт модели",
"nav.routing": "Роутинг",
@@ -56,7 +56,7 @@ export default {
"login.failed": "Неверный токен администратора.",
// dashboard
"dash.title": "Дашборд",
"dash.title": "Рабочий стол",
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
"dash.clear": "Очистить логи",
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
@@ -626,4 +626,16 @@ export default {
"tier.low": "лёгкий",
"tier.medium": "средний",
"tier.high": "сложный",
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
"settings.cache.entries": "записей",
"settings.cache.hits": "попаданий",
"settings.cache.saved": "сэкономлено",
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
"keys.budgetOf": "бюджет {b}/мес",
"dash.kpi.orBalance": "Баланс OpenRouter",
"keys.h.install": "Установить:",
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
"keys.snip.apply": "Вставить ключ",
};
+12
View File
@@ -634,4 +634,16 @@ export default {
"tier.low": "kolay",
"tier.medium": "orta",
"tier.high": "zor",
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
"settings.cache.entries": "kayıt",
"settings.cache.hits": "isabet",
"settings.cache.saved": "tasarruf",
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
"keys.budgetOf": "bütçe {b}/ay",
"dash.kpi.orBalance": "OpenRouter bakiyesi",
"keys.h.install": "Kurulum:",
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
"keys.snip.apply": "Anahtarı ekle",
};
+12
View File
@@ -634,4 +634,16 @@ export default {
"tier.low": "简单",
"tier.medium": "中等",
"tier.high": "困难",
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
"settings.cache.entries": "条目",
"settings.cache.hits": "命中",
"settings.cache.saved": "已节省",
"keys.form.budget": "每月预算,$(0 = 不限)",
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
"keys.budgetOf": "预算 {b}/月",
"dash.kpi.orBalance": "OpenRouter 余额",
"keys.h.install": "安装:",
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
"keys.snip.apply": "插入密钥",
};
+6 -1
View File
@@ -238,6 +238,7 @@ export async function renderBenchmark() {
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
if (again.tier === "high") {
info.tier = again.tier;
info.score = again.score;
info.selected_model = again.selected_model;
info.task_label = again.task_label;
info.route_source = again.route_source;
@@ -269,6 +270,8 @@ export async function renderBenchmark() {
|| String(info.route_source || "").includes("degraded");
row.task = info.task_label || "—";
row.tier = info.tier || "—";
row.cscore = info.score || 0; // непрерывный коэффициент сложности
// (row.score занят долей правильности ответа из evaluate())
row.chosen = info.selected_model || "—";
row.source = info.route_source || "—";
row.expected = test.label; // null = classification not judged
@@ -402,7 +405,9 @@ export async function renderBenchmark() {
: h("span", { class: r.taskOk ? "" : "err" },
(r.taskOk ? "✓ " : "✗ ") + r.task,
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—"),
h("td", {},
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
h("td", {},
h("span", { class: "mono small" }, r.chosen || "—"),
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
+12 -4
View File
@@ -1,6 +1,6 @@
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
import { SITE_URL } from "../app.js";
@@ -110,12 +110,13 @@ export async function renderDashboard() {
async function load() {
const rangeSecs = range;
let stats, recent, health;
let stats, recent, health, balance;
try {
[stats, recent, health] = await Promise.all([
[stats, recent, health, balance] = await Promise.all([
api.stats(rangeSecs, groupby),
api.requests(40, 0),
api.health().catch(() => null),
api.balance().catch(() => null),
]);
} catch (e) {
mount(root, h("div", { class: "callout" }, String(e.message || e)));
@@ -162,6 +163,11 @@ export async function renderDashboard() {
kpi(num(tot.requests), t("dash.kpi.requests")),
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
balance && balance.openrouter_usd != null
? kpi(
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
t("dash.kpi.orBalance"))
: null,
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
kpi(pct(tot.success_rate), t("dash.kpi.success")),
kpi(num(tot.failovers), t("dash.kpi.failovers"))
@@ -271,7 +277,9 @@ function recentTable(rows) {
{},
h("td", { class: "mono" }, timeAgo(r.ts)),
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
h("td", {},
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
h("td", {}, r.task_label || "—"),
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
h("td", { class: "mono" }, money(r.cost_usd)),
+147 -14
View File
@@ -10,12 +10,14 @@ export async function renderKeys() {
function openCreate(reload) {
const accountIn = h("input", { placeholder: "team-a" });
const rateIn = h("input", { type: "number", value: 600 });
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
const customIn = h("input", { placeholder: "sk-gw-…" });
const body = h(
"div",
{},
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
);
modal(t("keys.form.title"), body, async () => {
@@ -24,6 +26,7 @@ export async function renderKeys() {
key: customIn.value.trim(),
account: accountIn.value.trim() || "default",
rate_per_min: parseInt(rateIn.value) || 0,
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
allow_models: [],
});
reload();
@@ -51,18 +54,76 @@ export async function renderKeys() {
function snippetsCard(openMode) {
const origin = location.origin;
const base = origin + "/v1";
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
// Every scripty snippet exists in two OS flavors: bash heredocs for
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
// (python/js) and GUI instructions are OS-independent.
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
const snips = {
"Claude Code": `# ${t("keys.h.claude")}
export ANTHROPIC_BASE_URL="${origin}"
const buildSnips = (osKind) => osKind === "ps" ? {
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
$env:ANTHROPIC_MODEL = "cortiq-auto"
claude`,
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
Add-Content "$HOME\\.codex\\config.toml" @'
model = "cortiq-auto"
model_provider = "cortiq"
[model_providers.cortiq]
name = "Cortiq Gateway"
base_url = "${base}"
env_key = "CORTIQ_API_KEY"
wire_api = "chat"
'@
$env:CORTIQ_API_KEY = "${key}"
codex`,
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"cortiq": {
"npm": "@ai-sdk/openai-compatible",
"name": "Cortiq Gateway",
"options": { "baseURL": "${base}", "apiKey": "${key}" },
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
}
}
}
'@
opencode`,
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
Add-Content "$HOME\\.continue\\config.yaml" @'
models:
- name: Cortiq Auto
provider: openai
model: cortiq-auto
apiBase: ${base}
apiKey: ${key}
'@`,
Cline: `${t("keys.h.cline1")}
API Provider: OpenAI Compatible
Base URL: ${base}
API Key: ${key}
Model ID: cortiq-auto
${t("keys.h.cline2")}`,
curl: `curl.exe ${base}/chat/completions \`
-H "Authorization: Bearer ${key}" \`
-H "Content-Type: application/json" \`
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
} : {
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
export ANTHROPIC_AUTH_TOKEN="${key}"
export ANTHROPIC_MODEL="cortiq-auto"
claude`,
Codex: `# ${t("keys.h.codex")}
mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
model = "cortiq-auto"
model_provider = "cortiq"
@@ -75,8 +136,7 @@ EOF
export CORTIQ_API_KEY="${key}"
codex`,
OpenCode: `# ${t("keys.h.opencode")}
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
@@ -91,8 +151,7 @@ mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
EOF
opencode`,
Continue: `# ${t("keys.h.continue")}
mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
models:
- name: Cortiq Auto
provider: openai
@@ -128,8 +187,29 @@ const resp = await client.chat.completions.create({
});
console.log(resp.choices[0].message.content);`,
};
let shSnips, psSnips, snips;
function rebuildSnips() {
shSnips = buildSnips("sh");
psSnips = buildSnips("ps");
// python/js/Cline одинаковы — дополняем ps-набор из sh
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
snips = os === "ps" ? psSnips : shSnips;
}
rebuildSnips();
let cur = "Claude Code";
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
// in a pasted snippet is not a comment in default interactive zsh, and a
// "!" inside it triggers history expansion ("event not found")
const HINTS = {
"Claude Code": () => t("keys.h.claude"),
Codex: () => t("keys.h.codex"),
OpenCode: () => t("keys.h.opencode"),
Continue: () => t("keys.h.continue"),
};
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
paintHint();
const seg = h(
"div",
{ class: "seg" },
@@ -137,6 +217,7 @@ console.log(resp.choices[0].message.content);`,
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
cur = k;
pre.textContent = snips[k];
paintHint();
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
paintStatus();
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
@@ -164,7 +245,15 @@ console.log(resp.choices[0].message.content);`,
}).catch(() => {});
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
// status line under the snippet ("installed" / "already configured").
// status line under the snippet ("installed" / "already configured") —
// and a one-line INSTALL command (per OS) when it's missing.
const INSTALL = {
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
};
let detected = {};
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
function paintStatus() {
@@ -176,7 +265,19 @@ console.log(resp.choices[0].message.content);`,
statusHost.innerHTML = "";
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
else mount(statusHost, h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")));
else {
const cmd = (INSTALL[cur] || {})[os];
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
await navigator.clipboard.writeText(cmd);
toast(t("common.copied"), "good");
} }, "⧉");
mount(statusHost,
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
h("span", { class: "muted" }, t("keys.h.install")),
h("code", { class: "mono small cmd-chip" }, cmd),
copyInstall) : null);
}
}
api.harnesses().then((d) => {
detected = d || {};
@@ -187,6 +288,32 @@ console.log(resp.choices[0].message.content);`,
paintStatus();
}).catch(() => {});
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
const osSeg = h(
"div",
{ class: "seg" },
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
);
osSeg.querySelectorAll("button").forEach((b, i) =>
b.addEventListener("click", () => {
os = i === 0 ? "sh" : "ps";
rebuildSnips();
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
pre.textContent = snips[cur];
paintStatus();
}));
// paste-your-key: substitutes the key into every snippet immediately
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
const applyKey = () => {
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
rebuildSnips();
pre.textContent = snips[cur];
};
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
copy.addEventListener("click", async () => {
await navigator.clipboard.writeText(pre.textContent);
@@ -198,7 +325,8 @@ console.log(resp.choices[0].message.content);`,
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" },
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), copy),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
hintHost,
pre,
statusHost,
protoWarn,
@@ -273,7 +401,12 @@ console.log(resp.choices[0].message.content);`,
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
h("td", { class: "right-align" },
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
k.monthly_budget_usd
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
: null),
h(
"td",
{ class: "right-align" },
+24 -1
View File
@@ -194,8 +194,30 @@ export async function renderSettings() {
const modelsData = await api.listModels();
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
// Live proof-of-life for the cache card: entries/hits/saved — and an
// honest warning when the cache is on but has no embedding model to work
// with (it silently does nothing in that state).
let cacheStats = null;
try {
cacheStats = (await api.stats("24h", "model")).cache || null;
} catch { /* stats off */ }
function cacheStatus() {
const wrap = h("div", { style: "margin-bottom:10px" });
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
return wrap;
}
if (cacheStats && s.cache?.enabled) {
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
}
return wrap;
}
const cacheEmbedSel = h(
"select",
{},
@@ -610,6 +632,7 @@ export async function renderSettings() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
cacheStatus(),
cacheEnabled.node,
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
field(t("settings.cache.embed"), cacheEmbedSel),