Compare commits

...
20 Commits
Author SHA1 Message Date
infosave2007 34093330f6 chore: bump v0.2.44
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 17:58:25 +03:00
infosave2007 faddf945d7 fix(config): a foreign project must not capture the config path
default_config_path treated any directory holding Cargo.toml plus a
config/ folder as a gateway checkout, so running cortiq-gateway inside
an unrelated project died with "cannot read config config/gateway.toml"
instead of using the per-user home. Now ./config/gateway.toml counts
only when the file is actually readable, the repo-checkout heuristic
requires Cargo.toml to be cortiq-gateway's own, and everything else
falls through to $CORTIQ_GATEWAY_HOME / ~/.cortiq-gateway — typing
cortiq-gateway in any directory picks up the same configuration.
2026-08-08 17:57:45 +03:00
infosave2007 255d989e83 feat(admin): power controls — stop everything & power off, restart everything
Settings gains a second service button next to restart: stop every managed
local model server and exit the gateway process (nothing is respawned; in
Docker the restart policy decides). The SPA replaces itself with a static
"gateway is off" screen since the API is gone. The restart button is
relabeled "Restart everything" to say what it always did — the gateway
re-execs itself and managed servers come back with it. Translated in all
seven languages.
2026-08-08 17:33:56 +03:00
infosave2007 0d9d24125a style: rustfmt after resolve() signature change 2026-08-08 16:26:07 +03:00
infosave2007 1c59906b49 chore: bump v0.2.43
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 16:12:17 +03:00
infosave2007 19f666e7b1 feat(cmf): auto-detect tool support of managed local servers
cortiq serve >=0.5.62 advertises the loaded model's capabilities in
/healthz ({"capabilities":{"tools":true}} when the chat template has a
tools branch). The gateway now reads that handshake when the managed
server becomes healthy and treats detected tool support as equivalent
to declared caps in the candidate filter — so a low-tier chain made of
a local model no longer widens agent (tools) traffic to cloud models
once the local server can actually drive tools.

Also adds an optional manual "caps" override to [[cmf.servers]] for
older serve builds that do not advertise capabilities.
2026-08-08 15:52:20 +03:00
infosave2007 dbcd9827f6 fix(pipeline): capability filtering widens the chain instead of killing the request
OpenCode (like every coding agent) sends tool definitions with each request.
The router rated the prompt low → the low-tier chain contained only the
local model → the tools filter emptied the chain → hard failure: "No
healthy model available that satisfies required capabilities", with the
agent stuck in a retry loop.

On an AUTO route whose filtered chain comes up empty, the pipeline now
widens to any pool model satisfying the required capabilities — preferring
the routing table's own medium→high order (an agentic request deserves a
capable model), then the rest of the pool — and logs the widening. Pinned
requests still fail loudly as before. Verified with OpenCode's exact
request shape: cortiq-auto + tools at tier low now answers from
deepseek in 1.2s instead of erroring.
2026-08-08 14:25:00 +03:00
infosave2007 b689180717 fix(keys): snippets survive being pasted into real zsh
The copyable text opened with a "# hint" line — but default interactive
zsh has interactivecomments off, so the line was parsed as a command, and
the "!" inside «(перезапишет!)» fired history expansion: pasting the
OpenCode snippet died with "zsh: event not found: )" before doing
anything. Hints now live in the UI above the snippet; the clipboard gets
commands only. Verified by piping the actual rendered snippet through an
interactive zsh: the config file lands intact.
2026-08-08 14:16:22 +03:00
infosave2007 f385c2254b fix(ui): install-command chip and bands slider readable in the light theme
The chip used var(--panel-2, #23252c) — a variable that doesn't exist with
a hard dark fallback, so in the light theme the install command was a dark
box with dark text. Same class of bug in the bands slider track and handle
labels (invented --panel-2/--fg vars). All moved to the real theme
variables (--panel/--border/--text) via a .cmd-chip class; verified by
computed styles in the light theme (text rgb(15,23,42) on white) and
screenshots of both views.
2026-08-08 14:12:39 +03:00
infosave2007 9607a38792 feat(keys): paste-your-key field — snippets carry the real key instantly
Next to the copy button: a key input and an «Insert key» button (Enter works
too). The pasted key substitutes into every harness/SDK snippet immediately
and survives tab and OS switches; a freshly created key pre-fills the field.
No more copying a snippet and then hand-editing the placeholder. 2 strings
×7 languages.
2026-08-08 14:07:11 +03:00
infosave2007 239ddc767c feat(keys): harness snippets for every OS + install command when missing
The harness auto-setup scripts were bash-only (heredocs, mkdir -p) — useless
on Windows. Every scripty snippet now exists in two flavors behind a
🐧🍎/🪟 switch (auto-detected from the browser): bash heredocs for
Linux/macOS and PowerShell here-strings ($env:, New-Item, @'..'@,
curl.exe with backtick continuations) for Windows. SDK snippets and GUI
instructions stay OS-independent.

And when detection says a harness is NOT installed on the gateway machine,
the status line now offers its one-line install command for the selected OS
(claude.ai install.sh / install.ps1, npm -g for Codex and OpenCode-on-
Windows, code --install-extension for Continue/Cline) with a copy button.
1 string ×7 languages.
2026-08-08 14:05:47 +03:00
infosave2007 057abaa95a feat!: API keys actually gate the gateway — auth, attribution, rate limits, budgets
The office-readiness audit found the foundation missing: configured API keys
were decorative. Any bearer — or none — passed every model endpoint, per-key
rate limits existed only in the form, and all real traffic was recorded as
"anonymous", leaving the per-key spend table permanently blank.

A middleware on the model surface (/v1/*, /route, /mcp) now enforces keys:
- no keys configured → open mode as before (the console already warns);
- keys exist → Bearer or x-api-key must match (401 invalid_api_key);
- per-key rate_per_min → fixed one-minute window, 429 with Retry-After;
- NEW per-key monthly_budget_usd → 429 insufficient_quota once the account's
  month-to-date spend crosses the ceiling;
- the matched account rides the request into stats — the keys table's spend
  columns finally see real traffic.

Month-to-date spend is tracked per account in stats and persisted in the
totals sidecar with a watermark (no double counting across restarts). The
key dialog gains a budget field; the table shows "budget $N/mo" in red once
exhausted. All verified live: 401 wrong/missing key, 200 + attribution with
the right key, 429 + Retry-After over the limit, 429 insufficient_quota on
the second request past a $0.000001 budget.

Also: the dashboard shows the OpenRouter credit balance (GET
/api/v1/credits via the stored key, cached 5 min, hidden without a key, red
under $5 — live: $16.18), the server drains connections on Ctrl-C instead
of dropping them, request bodies are capped at 32 MB, and docs/DEPLOY(.ru)
cover office deployment: systemd/launchd/Windows service, nginx/caddy TLS,
security checklist, backups. 10 strings ×7 languages.
2026-08-08 14:01:16 +03:00
infosave2007 dd864eed22 fix(cache): honest settings card — clean threshold, live stats, no-embedder warning
Full audit of the semantic cache against a live instance (mock embedder,
identical requests): the mechanism works — miss at 3.1s, then hits at 2ms
with entries/hits/saved-$ counting. Two honest gaps fixed:

- the threshold round-tripped through f32 and rendered as
  0.9200000166893005 in both the TOML and the settings input — the config
  field is f64 now (writes exactly what you typed) and the input rounds
  legacy values to 2 decimals;
- the card now shows proof of life (entries / hits / hit-rate / saved $)
  and, when the cache is enabled with NO embedding model available, a
  warning that it is silently doing nothing and how to fix it — previously
  that state looked identical to a working cache.

Also audited in the same pass, no changes needed: Prometheus /metrics
(toggle on/off applies live, 404 when off), and all 7 protocol switches
(each gates its endpoint instantly both ways). 4 strings ×7 languages.
2026-08-08 13:21:43 +03:00
infosave2007 c1ff675d80 chore: release 0.2.42
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 12:18:04 +03:00
infosave2007 9ef1af3ae1 feat(ui): the continuous complexity score shows next to the tier
Benchmark (routing mode) and the dashboard's recent-requests list now print
the router's raw 0..1 score under/next to the tier badge — people can see
what different prompts actually weigh and tune the boundary sliders from
real data. Tooltip explains the link to the Routing sliders (7 languages).

Caught before commit: naming the field row.score collided with the answer-
correctness fraction from evaluate() and dragged the benchmark's quality KPI
down to the average complexity — renamed to cscore, KPI verified back at
100%. Dashboard tier badges also render translated names now, and the
Russian locale calls the dashboard «Рабочий стол».
2026-08-08 12:11:07 +03:00
infosave2007 6f4d83ce13 chore: release 0.2.41
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 09:55:20 +03:00
infosave2007 0040bfd7e9 fix(routing): the high zone is unmistakably red
At 35% opacity over the dark background the amber medium zone and the
light-red high zone read as the same color right of the medium handle.
Saturation stepped up per zone (45/55/65%) and high uses a deeper red —
green / amber / red now read at a glance.
2026-08-08 09:53:25 +03:00
infosave2007 10b5101757 fix(i18n): complexity tier names are translated everywhere they're shown
low/medium/high leaked as raw English into the bands legend, the tier badges
on the Routing page, the pair complexity selects, the benchmark's tier column
and the Playground decision panel. New trTier() helper renders the human name
(ru: лёгкий/средний/сложный, de: leicht/mittel/schwer, fr/es/zh/tr likewise);
custom tier names pass through untouched, badge colors stay keyed by the raw
tier. Verified in the browser in Russian and German.
2026-08-08 09:51:38 +03:00
infosave2007 788c87c565 feat(routing): linked complexity-boundary sliders — your own low/medium/high
The tier boundaries lived on the router and were global for every customer
of a hosted router; per-gateway tuning meant hand-editing score ranges. The
Routing page now has one track with two draggable handles (the low|medium
and medium|high boundaries) over three colored zones: handles push each
other (low can't cross medium — it shoves it along, and vice versa), the
numbers ride on the handles and the legend updates live, so nobody types
ranges by hand. Unchecked = the router's own bands, untouched behavior.

Under the hood the gateway re-maps the router's CONTINUOUS complexity score
with the user's boundaries ([routing.bands] low/medium in gateway.toml,
validated 0 < low < medium < 1) before tier chains, escalation and stats —
verified live: with bands at 0.70/0.72 a score-0.27 request that used to be
medium routed low, and back after restoring the defaults.

2 new strings in all 7 languages.
2026-08-08 09:48:36 +03:00
infosave2007 179c414a0e fix(benchmark): retry the SAME hard prompt after a pause before reaching for a fresh one
Difficulty is judged in the background: the first-ever occurrence of a hard
prompt routes by prior (often low/medium), the oracle's judgement lands in
the cache a couple of seconds later. The route-judged retry fired instantly
with a FRESH template — which has the same first-encounter fate — so the
test could go red exactly when the system was working as designed. The retry
now re-asks the same prompt after 4s (steady-state answer from the cache),
and only then tries a fresh template.
2026-08-08 09:23:53 +03:00
37 changed files with 1325 additions and 100 deletions
+83
View File
@@ -3,6 +3,89 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.44] - 2026-08-08
### Added
- **Power controls in Settings**: «Stop everything & power off» stops every managed local
model server and exits the gateway (the panel shows a clear "gateway is off" screen);
«Restart everything» (relabeled) re-execs the gateway with its local servers. Both behind
confirm dialogs, translated in all seven languages.
### Fixed
- `cortiq-gateway` typed in any directory picks up your configuration. Resolution order:
`--config` → a readable `./config/gateway.toml` → `$CORTIQ_GATEWAY_HOME` →
`~/.cortiq-gateway/gateway.toml`. An unrelated project that merely contains `Cargo.toml`
and a `config/` folder no longer hijacks the config path ("cannot read config
config/gateway.toml").
## [0.2.43] - 2026-08-08
### Added
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
configured = open mode, as before.
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
when the server becomes healthy, so a local model whose chat template can drive tool
calling serves agent (tools) traffic with zero manual configuration. Manual override:
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
remaining credit is a KPI card, red under $5.
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
per harness, an install command chip when the harness is missing, and a paste-your-key
field that substitutes the real key into every snippet.
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
Caddy TLS fronting, security checklist, backup list.
### Fixed
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
route's chain has no model with the required capabilities (tools), the gateway escalates
through medium→high chains and then the rest of the pool instead of returning
"No healthy model available". Explicit pins still fail loudly.
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
no history-expansion `!`), and the install chips are readable in the light theme.
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
into the sidecar totals file — per-account all-time and month spend survive compaction
and restarts.
### Breaking
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
will now be rejected with 401 — either remove the keys from the config or start sending
`Authorization: Bearer <key>`.
## [0.2.42] - 2026-08-08
### Added
- **The continuous complexity score is visible**: the benchmark's routing table and the
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
see what different prompts actually weigh and tune the boundary sliders from real traffic.
### Changed
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
«Рабочий стол».
## [0.2.41] - 2026-08-08
### Added
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
(the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
other, the numbers ride on the handles and the legend updates live — no hand-typed score
ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
(`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
stats; unchecked = the router's own bands, untouched behavior.
### Fixed
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
fresh template — difficulty is judged in the background, so the second occurrence is the
honest steady-state answer.
## [0.2.40] - 2026-08-08
### Added
Generated
+1 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "cortiq-gateway"
version = "0.2.40"
version = "0.2.44"
dependencies = [
"anyhow",
"arc-swap",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "cortiq-gateway"
version = "0.2.40"
version = "0.2.44"
edition = "2021"
rust-version = "1.75"
license = "Apache-2.0"
+2
View File
@@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
---
## 🖥️ Admin console
+2
View File
@@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
---
## 🖥️ Панель управления
+122
View File
@@ -0,0 +1,122 @@
# Deploying cortiq-gateway for a team / office
One gateway on a small server, API keys per department, local models for the
cheap tiers, cloud models for the hard ones. This page covers the ops side.
## 1. Run as a service
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Task Scheduler → Create Task → "Run whether user is logged on or not",
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS and the outside world
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
and bind the gateway to localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE streaming
proxy_read_timeout 3600s; # long generations
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
same with automatic certificates.)
## 3. Security checklist
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
the console warns that it is open to the network.
- **API keys**: create one per department/developer on the keys page. With at
least one key configured the model APIs require Bearer / x-api-key —
requests without a valid key get 401.
- **Rate limits**: per-key requests/min → 429 with Retry-After.
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
account's month-to-date spend crosses it.
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
## 4. Backups
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
- `secrets.toml` — provider API keys (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
cumulative per-key totals and month-to-date budget spend.
Copy those three and you can rebuild the instance anywhere.
## 5. Observability
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
OpenRouter balance (red under $5).
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
+123
View File
@@ -0,0 +1,123 @@
# Развёртывание cortiq-gateway для команды / офиса
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
## 1. Запуск как сервис
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
--no-port-search`. Либо [NSSM](https://nssm.cc):
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS и внешний мир
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
привяжите к localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE-стриминг
proxy_read_timeout 3600s; # долгие генерации
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
с автоматическими сертификатами.)
## 3. Чек-лист безопасности
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
него консоль предупреждает, что открыта сети.
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
запросы без валидного ключа получают 401.
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
траты аккаунта за месяц превысят лимит.
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
## 4. Бэкапы
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
- `secrets.toml` — ключи провайдеров (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
тоталы по ключам и месячные траты для бюджетов.
Скопируйте эти три файла — и инстанс поднимается где угодно.
## 5. Наблюдаемость
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
сложности, баланс OpenRouter (красный ниже $5).
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
+70
View File
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
get(get_protocols).put(put_protocols),
)
.route("/admin/api/harnesses", get(get_harnesses))
.route("/admin/api/balance", get(get_balance))
.route("/admin/api/settings", get(get_settings).put(put_settings))
.route("/admin/api/keys", get(list_keys).post(create_key))
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
@@ -183,6 +184,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
.route("/admin/api/logs", get(get_logs).delete(clear_logs))
.route("/admin/api/system", get(get_system))
.route("/admin/api/restart", post(restart_gateway))
.route("/admin/api/shutdown", post(shutdown_gateway))
.route("/admin/api/bench", get(bench_list).post(bench_start))
.route("/admin/api/config/export", get(config_export))
.route("/admin/api/config/import", post(config_import))
@@ -931,6 +933,7 @@ async fn get_routing(State(state): State<SharedState>) -> ApiResult {
.collect();
ok(json!({
"tiers": tiers_to_map(&live.cfg.routing),
"bands": live.cfg.routing.bands,
"tasks": tasks,
"default": live.cfg.routing.default,
"policy": live.cfg.routing.policy,
@@ -978,6 +981,8 @@ struct RoutingBody {
tiers: HashMap<String, Vec<String>>,
#[serde(default)]
tasks: HashMap<String, TierTargets>,
#[serde(default)]
bands: Option<crate::config::TierBands>,
default: String,
#[serde(default)]
policy: RoutingPolicy,
@@ -997,6 +1002,7 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
// empty rule (no models AND no forced tier) = "по сложности"
.filter(|(_, v)| !v.models().is_empty())
.collect(),
bands: body.bands,
default: body.default,
policy: body.policy,
};
@@ -1004,6 +1010,51 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
ok(json!({ "ok": true }))
}
// ─────────────────────────────── balance ───────────────────────────────────
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
// (null) when no OpenRouter key is configured — the card hides itself.
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
std::sync::OnceLock::new();
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
if let Some((at, v)) = *cache.lock().unwrap() {
if at.elapsed() < std::time::Duration::from_secs(300) {
return ok(json!({ "openrouter_usd": v }));
}
}
let key = state
.secrets
.resolve("OPENROUTER_API_KEY")
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
let Some(key) = key.filter(|k| !k.is_empty()) else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(8))
.build()
.unwrap_or_default();
let resp = client
.get("https://openrouter.ai/api/v1/credits")
.bearer_auth(&key)
.send()
.await;
let balance = match resp {
Ok(r) if r.status().is_success() => {
let v: serde_json::Value = r.json().await.unwrap_or_default();
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
Some(total - used)
}
_ => None,
};
let Some(balance) = balance else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
ok(json!({ "openrouter_usd": balance }))
}
// ──────────────────────────── harness detection ────────────────────────────
// Which coding harnesses are present on the machine the gateway runs on, so the
// API-keys view can mark tabs "installed" / "already configured for cortiq".
@@ -1353,6 +1404,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
"key_masked": mask_key(&k.key),
"account": k.account,
"rate_per_min": k.rate_per_min,
"monthly_budget_usd": k.monthly_budget_usd,
"allow_models": k.allow_models,
})
})
@@ -1368,6 +1420,8 @@ struct NewKey {
#[serde(default)]
rate_per_min: u32,
#[serde(default)]
monthly_budget_usd: Option<f64>,
#[serde(default)]
allow_models: Vec<String>,
}
@@ -1386,6 +1440,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
account: b.account,
rate_per_min: b.rate_per_min,
allow_models: b.allow_models,
monthly_budget_usd: b.monthly_budget_usd,
});
state.reload(cfg)?;
// show the full key only once
@@ -2143,6 +2198,7 @@ fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<
o1_window: None,
o1_sink: None,
skip_mtp: false,
caps: Vec::new(),
});
Some(id)
}
@@ -2217,6 +2273,20 @@ async fn get_system(State(state): State<SharedState>) -> ApiResult {
ok(serde_json::to_value(snap).unwrap_or_else(|_| json!({})))
}
/// Stop every managed local model server and power the gateway off. The 200
/// goes out first so the SPA can show its "gateway is off" screen; nothing is
/// respawned — the operator starts the gateway again by hand (or the Docker
/// restart policy does, in which case this behaves like a restart).
async fn shutdown_gateway(State(state): State<SharedState>) -> ApiResult {
state.cmf.stop().await;
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
tracing::info!("shutdown requested from the admin console");
std::process::exit(0);
});
ok(json!({ "ok": true }))
}
/// Restart the gateway in place: stop managed model servers, spawn a fresh
/// copy of ourselves (it retries the bind while our port frees), then exit.
/// In Docker we only exit — we are PID 1, a spawned child would die with the
+1 -1
View File
@@ -55,7 +55,7 @@ impl SemanticCache {
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
Arc::new(Self {
enabled: cfg.enabled,
threshold: cfg.threshold,
threshold: cfg.threshold as f32,
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
max_entries: cfg.max_entries.max(1),
embed_model: cfg.embed_model.clone(),
+27 -5
View File
@@ -56,6 +56,9 @@ struct Managed {
pub struct CmfRuntime {
status: Mutex<CmfStatus>,
children: Mutex<Vec<Managed>>,
/// Server ids whose healthz handshake advertised tool support — lets the
/// pipeline route tools traffic to local models with no manual caps.
pub detected_tools: Mutex<std::collections::HashSet<String>>,
}
impl CmfRuntime {
@@ -63,6 +66,7 @@ impl CmfRuntime {
Arc::new(Self {
status: Mutex::new(CmfStatus::default()),
children: Mutex::new(Vec::new()),
detected_tools: Mutex::new(std::collections::HashSet::new()),
})
}
@@ -193,6 +197,7 @@ impl Default for CmfRuntime {
Self {
status: Mutex::new(CmfStatus::default()),
children: Mutex::new(Vec::new()),
detected_tools: Mutex::new(std::collections::HashSet::new()),
}
}
}
@@ -332,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
.map_err(|e| e.to_string())
}
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
/// Poll `/healthz` until the local server answers 200 (or the deadline
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
/// caller can register them; older servers return `None` there.
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
let url = format!("http://{host}:{port}/healthz");
let client = reqwest::Client::new();
for _ in 0..tries {
@@ -344,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
.await
{
if r.status().is_success() {
return true;
let tools = r
.json::<serde_json::Value>()
.await
.ok()
.and_then(|v| v["capabilities"]["tools"].as_bool());
return (true, tools);
}
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
false
(false, None)
}
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
@@ -432,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
if !rt.is_running(&server.id) {
continue; // failed to spawn / file missing
}
if wait_healthy(&cfg.local_host, server.port, 240).await {
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
if ok {
rt.set_healthy(&server.id, true);
// capability handshake: the server said whether its model's chat
// template can drive tool calling — no manual caps needed
if tools == Some(true) {
rt.detected_tools.lock().unwrap().insert(server.id.clone());
tracing::info!(id = %server.id, "local server advertises tool support");
} else {
rt.detected_tools.lock().unwrap().remove(&server.id);
}
rt.log(format!(
"local CMF server ready — '{}' at http://{}:{}/v1",
server.id, cfg.local_host, server.port
+60 -7
View File
@@ -130,6 +130,10 @@ pub struct CmfServer {
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
#[serde(default)]
pub skip_mtp: bool,
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
/// gateway auto-detects tool support from the server's healthz handshake.
#[serde(default)]
pub caps: Vec<String>,
}
/// CMF-format model factory: import a HuggingFace model, convert it to a
@@ -229,6 +233,7 @@ impl CmfCfg {
o1_window: None,
o1_sink: None,
skip_mtp: false,
caps: Vec::new(),
}];
}
Vec::new()
@@ -404,6 +409,31 @@ pub struct RoutingCfg {
/// called "tasks" — serde extracts it before the flattened tier map.
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
pub tasks: HashMap<String, TierTargets>,
/// Local complexity bands: re-map the router's continuous score into
/// low/medium/high with YOUR OWN boundaries. The hosted router's bands
/// are global for all customers; these are per-gateway. `None` = keep
/// the router's tier verdict as delivered.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bands: Option<TierBands>,
}
/// Score boundaries: `score ≤ low` → low, `≤ medium` → medium, else high.
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
pub struct TierBands {
pub low: f32,
pub medium: f32,
}
impl TierBands {
pub fn tier_of(&self, score: f32) -> &'static str {
if score <= self.low {
"low"
} else if score <= self.medium {
"medium"
} else {
"high"
}
}
}
/// A routing tier is either a list of model_ids or the string `default`.
@@ -595,6 +625,9 @@ pub struct ApiKeyCfg {
pub account: String,
#[serde(default)]
pub rate_per_min: u32,
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub monthly_budget_usd: Option<f64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub allow_models: Vec<String>,
}
@@ -804,9 +837,10 @@ fn default_shadow_soak() -> usize {
pub struct CacheCfg {
#[serde(default)]
pub enabled: bool,
/// Cosine-similarity threshold for a hit (0..1).
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
/// the number you typed (an f32 round-trips as 0.9200000166893005).
#[serde(default = "default_semcache_threshold")]
pub threshold: f32,
pub threshold: f64,
/// Entry time-to-live, e.g. `1h`.
#[serde(default = "default_semcache_ttl")]
pub ttl: String,
@@ -828,7 +862,7 @@ impl Default for CacheCfg {
}
}
}
fn default_semcache_threshold() -> f32 {
fn default_semcache_threshold() -> f64 {
0.92
}
fn default_semcache_ttl() -> String {
@@ -861,6 +895,7 @@ impl Default for Config {
default: String::new(),
policy: RoutingPolicy::default(),
tasks: HashMap::new(),
bands: None,
},
breaker: BreakerCfg::default(),
protocols: ProtocolsCfg {
@@ -889,14 +924,23 @@ impl Default for Config {
}
/// Where the config lives when `--config` is not given.
/// Repo checkout (`config/gateway.toml` exists or `config/` dir exists) → keep it there;
/// installed binary (no repo) → `$data_dir/gateway.toml`.
/// A readable `./config/gateway.toml` (a real gateway checkout / install dir)
/// wins; otherwise the per-user home (`$CORTIQ_GATEWAY_HOME` or
/// `~/.cortiq-gateway`) — so `cortiq-gateway` typed in any directory picks up
/// the same configuration it was set up with. A foreign project that merely
/// has `Cargo.toml` and a `config/` folder must NOT capture the config
/// (that used to shadow the user's real config with a broken path).
pub fn default_config_path() -> String {
let repo_path = "config/gateway.toml";
if std::path::Path::new(repo_path).exists() {
// the file itself, readable — not just an entry that exists()
if std::fs::read_to_string(repo_path).is_ok() {
return repo_path.to_string();
}
if std::path::Path::new("Cargo.toml").exists() && std::path::Path::new("config").exists() {
// our own repo checkout before first run: config/ exists, file not yet
let ours = std::fs::read_to_string("Cargo.toml")
.map(|t| t.contains("name = \"cortiq-gateway\""))
.unwrap_or(false);
if ours && std::path::Path::new("config").is_dir() {
return repo_path.to_string();
}
data_dir()
@@ -988,6 +1032,15 @@ impl Config {
}
}
}
if let Some(b) = &self.routing.bands {
if !(b.low > 0.0 && b.low < b.medium && b.medium < 1.0) {
anyhow::bail!(
"config: routing.bands must satisfy 0 < low < medium < 1 (got {} / {})",
b.low,
b.medium
);
}
}
for (task, targets) in &self.routing.tasks {
for id in &targets.models() {
if !ids.contains(id.as_str()) {
+7
View File
@@ -13,6 +13,8 @@ pub enum GatewayError {
Unauthorized(String),
#[error("rate limit exceeded")]
RateLimited { retry_after_secs: u64 },
#[error("monthly budget exhausted (limit ${budget_usd})")]
BudgetExhausted { budget_usd: f64 },
#[error("upstream unavailable: {0}")]
UpstreamUnavailable(String),
#[error("internal error: {0}")]
@@ -37,6 +39,11 @@ impl GatewayError {
"rate_limit_error",
"rate_limited",
),
Self::BudgetExhausted { .. } => (
StatusCode::TOO_MANY_REQUESTS,
"insufficient_quota",
"insufficient_quota",
),
Self::UpstreamUnavailable(_) => (
StatusCode::BAD_GATEWAY,
"upstream_unavailable",
+15 -3
View File
@@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
.route("/health", get(|| async { "ok" }))
.route("/readyz", get(|| async { "ready" }))
.route("/metrics", get(admin::metrics))
.merge(protocols::build_router());
.merge(protocols::build_router(state.clone()));
if admin_enabled {
app = app
@@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> {
.allow_methods(tower_http::cors::Any)
.allow_headers(tower_http::cors::Any);
let app = app.with_state(state).layer(cors);
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
// small enough that a hostile client can't balloon memory
let app =
app.with_state(state)
.layer(cors)
.layer(tower_http::limit::RequestBodyLimitLayer::new(
32 * 1024 * 1024,
));
let listener = {
let mut attempt = 0u32;
@@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("open this URL to finish setup: {url}");
}
}
axum::serve(listener, app).await?;
axum::serve(listener, app)
.with_graceful_shutdown(async {
let _ = tokio::signal::ctrl_c().await;
tracing::info!("shutdown signal received — draining connections");
})
.await?;
Ok(())
}
+69 -10
View File
@@ -183,6 +183,9 @@ impl Pipeline {
&self,
req: &ChatRequest,
live: &crate::state::Live,
// Tool support detected via the local servers' healthz handshake —
// treated as equivalent to declared caps when filtering candidates.
detected_tools: &std::collections::HashSet<String>,
) -> Result<(RouteDecision, String, Vec<String>)> {
// 1. Extract text for classification
let text = self.extract_text(
@@ -231,6 +234,19 @@ impl Pipeline {
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
match live.router.route(&text, prof).await {
Ok(Some(dec)) => {
let mut dec = dec;
// per-gateway complexity bands: re-map the
// router's continuous score with the user's
// own boundaries (sliders on the Routing page)
if let Some(b) = &live.cfg.routing.bands {
if matches!(
dec.complexity_tier.as_str(),
"low" | "medium" | "high"
) {
dec.complexity_tier =
b.tier_of(dec.complexity_score).to_string();
}
}
let src = dec.source.clone();
let (cands, task_rule) = live
.routing
@@ -261,10 +277,13 @@ impl Pipeline {
// Filter candidates by capabilities (e.g. tools support)
let needs_tools = !req.tools.is_empty();
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
prov.caps().tools || detected_tools.contains(id)
};
let mut final_candidates = Vec::new();
for model_id in candidates {
if let Some(prov) = live.registry.get(&model_id) {
if needs_tools && !prov.caps().tools {
if needs_tools && !has_tools(&model_id, &prov) {
tracing::debug!(
"Candidate '{}' skipped: lacks support for requested tools",
model_id
@@ -293,13 +312,49 @@ impl Pipeline {
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
for id in live.local_candidates() {
if let Some(prov) = live.registry.get(&id) {
if needs_tools && !prov.caps().tools {
if needs_tools && !has_tools(&id, &prov) {
continue;
}
final_candidates.push(id);
}
}
}
// Capability widening: a tier chain built of tools-less local models
// must not kill an agent request. When the filtered chain is empty on
// an AUTO route, widen to any pool model that satisfies the required
// capabilities — preferring the routing table's own medium→high order
// (an agentic request is worth a capable model), then the rest of the
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
// this every low-tier verdict was a hard failure.
if final_candidates.is_empty() {
let mut widened: Vec<String> = Vec::new();
for tier in ["medium", "high"] {
for id in live.routing.candidates(tier) {
if !widened.contains(&id) {
widened.push(id);
}
}
}
for id in live.registry.ids() {
if !widened.contains(id) {
widened.push(id.clone());
}
}
for id in widened {
if let Some(prov) = live.registry.get(&id) {
if needs_tools && !has_tools(&id, &prov) {
continue;
}
final_candidates.push(id);
}
}
if !final_candidates.is_empty() {
tracing::info!(
chosen = %final_candidates[0],
"capability widening: tier chain had no model with the required capabilities"
);
}
}
if final_candidates.is_empty() {
return Err(GatewayError::UpstreamUnavailable(
"No healthy model available that satisfies required capabilities (e.g. tools)"
@@ -311,7 +366,9 @@ impl Pipeline {
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
let live = state.live();
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
let (decision, route_source, mut final_candidates) =
self.resolve(&req, &live, &detected_tools).await?;
// Serving-when-promoted (gated): a promoted task-label is served by the
// local model FIRST (failover to cloud on error). Per-request VETO uses
@@ -458,13 +515,15 @@ impl Pipeline {
.to_string();
let live = state.live();
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
Ok(t) => t,
Err(e) => {
record_error(state, &account, &protocol, &directive, started, &e);
return Err(e);
}
};
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
let (decision, route_source, final_candidates) =
match self.resolve(&req, &live, &detected_tools).await {
Ok(t) => t,
Err(e) => {
record_error(state, &account, &protocol, &directive, started, &e);
return Err(e);
}
};
let mut last_err = None;
let mut attempt = 0usize;
+2
View File
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<MessagesRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.anthropic_messages {
@@ -108,6 +109,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "anthropic_messages".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
+51 -1
View File
@@ -12,10 +12,59 @@ pub mod openai_models;
use crate::state::SharedState;
use axum::Router;
/// Account resolved by the API-key middleware, consumed by protocol handlers
/// for attribution (stats, budgets). Present only when keys are configured.
#[derive(Clone)]
pub struct AccountTag(pub String);
/// API-key gate for the model-facing surface. Open mode (no keys configured)
/// passes everything through — the admin console already warns about it.
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
/// enforced with a fixed one-minute window (429 + Retry-After), and a
/// monthly budget, when set, turns into 429 insufficient_quota once the
/// account's month-to-date spend crosses it.
async fn api_auth(
axum::extract::State(state): axum::extract::State<SharedState>,
mut req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
use axum::response::IntoResponse;
let live = state.live();
if live.cfg.api_keys.is_empty() {
return next.run(req).await;
}
let provided = req
.headers()
.get(axum::http::header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|v| v.strip_prefix("Bearer "))
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
.map(str::trim)
.unwrap_or("");
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
.into_response();
};
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
return crate::error::GatewayError::RateLimited {
retry_after_secs: state.rate.retry_after_secs(),
}
.into_response();
}
if let Some(budget) = key.monthly_budget_usd {
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
.into_response();
}
}
req.extensions_mut().insert(AccountTag(key.account.clone()));
next.run(req).await
}
/// Build the protocols router. Implemented adapters are **always** mounted;
/// each handler checks the live `protocols.*` flag from config itself, so
/// toggles in the admin panel take effect without a restart (hot switching).
pub fn build_router() -> Router<SharedState> {
pub fn build_router(state: SharedState) -> Router<SharedState> {
Router::new()
.merge(openai_chat::routes())
.merge(openai_completions::routes())
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
.merge(anthropic_messages::routes())
.merge(mcp::routes())
.merge(native_passthrough::routes())
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
}
+2
View File
@@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<OpenAiChatRequest>,
) -> Result<Response> {
// hot protocol toggle: if the adapter is disabled in config — return 404
@@ -104,6 +105,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_chat".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
+2
View File
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<CompletionsRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.openai_completions {
@@ -89,6 +90,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_completions".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
+1 -1
View File
@@ -48,7 +48,7 @@ impl Registry {
price_out: 0.0,
kind: "chat".into(),
api_key_env: None,
caps: Vec::new(),
caps: s.caps.clone(),
temperature: s.temperature,
top_p: s.top_p,
max_tokens: s.max_tokens,
+39
View File
@@ -66,6 +66,42 @@ impl Live {
}
}
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
/// fine for office-grade fairness and costs one HashMap lookup per request.
#[derive(Default)]
pub struct RateWindows {
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
}
impl RateWindows {
pub fn allow(&self, key: &str, per_min: u32) -> bool {
let minute = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
/ 60;
let mut g = self.inner.lock().unwrap();
let e = g.entry(key.to_string()).or_insert((minute, 0));
if e.0 != minute {
*e = (minute, 0);
}
if e.1 >= per_min {
return false;
}
e.1 += 1;
true
}
/// Seconds until the current minute window rolls over.
pub fn retry_after_secs(&self) -> u64 {
60 - (std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
% 60)
}
}
pub struct AppState {
pub live: ArcSwap<Live>,
pub stats: Arc<Stats>,
@@ -85,6 +121,8 @@ pub struct AppState {
/// Managed local CMF model server (install/update/spawn lifecycle + status).
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
pub cache: Arc<SemanticCache>,
/// Per-API-key fixed-window rate limiter.
pub rate: RateWindows,
pub secrets: SecretStore,
pub config_path: String,
pub pipeline: crate::pipeline::Pipeline,
@@ -164,6 +202,7 @@ impl SharedState {
admin_open_public: std::sync::atomic::AtomicBool::new(false),
cmf,
cache,
rate: RateWindows::default(),
secrets,
config_path,
pipeline: crate::pipeline::Pipeline::new(),
+112 -9
View File
@@ -137,6 +137,11 @@ struct Inner {
/// = folded + by_account_alltime, with no double counting: a record is
/// either in the file or folded, never both.
folded: HashMap<String, Totals>,
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
month_spend: HashMap<String, (String, f64)>,
/// Records with ts ≤ this are already reflected in the persisted
/// month_spend (watermark against double counting on replay).
month_upto_ts: u64,
/// Appends since the last compaction size check.
writes_since_check: u32,
}
@@ -193,6 +198,7 @@ impl Stats {
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
if rec.ts >= cutoff {
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
}
@@ -200,6 +206,33 @@ impl Stats {
}
}
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
return;
}
let m = month_of(rec.ts);
let e = inner
.month_spend
.entry(rec.account.clone())
.or_insert((m.clone(), 0.0));
if e.0 != m {
*e = (m, 0.0);
}
e.1 += rec.cost_usd;
}
/// Month-to-date spend for an account (budget checks).
pub fn month_spend(&self, account: &str) -> f64 {
let inner = self.inner.lock().unwrap();
let m = month_of(now_secs());
inner
.month_spend
.get(account)
.filter(|(mm, _)| *mm == m)
.map(|(_, v)| *v)
.unwrap_or(0.0)
}
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
inner.recent.push_back(rec.clone());
while inner.recent.len() > ring_size {
@@ -256,6 +289,7 @@ impl Stats {
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
inner.writes_since_check += 1;
}
if let Some(path) = &self.file {
@@ -283,11 +317,42 @@ impl Stats {
let Ok(content) = std::fs::read_to_string(&path) else {
return;
};
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
#[derive(serde::Deserialize)]
struct Sidecar {
#[serde(default)]
by_account: HashMap<String, Totals>,
#[serde(default)]
month_spend: HashMap<String, (String, f64)>,
#[serde(default)]
month_upto_ts: u64,
}
// new format {by_account, month_spend, month_upto_ts}; the original
// file held the bare by_account map — accept both
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
let mut inner = self.inner.lock().unwrap();
inner.folded = sc.by_account;
inner.month_spend = sc.month_spend;
inner.month_upto_ts = sc.month_upto_ts;
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
self.inner.lock().unwrap().folded = m;
}
}
fn write_sidecar(&self, inner: &Inner) {
let Some(tpath) = self.totals_path() else {
return;
};
let payload = serde_json::json!({
"by_account": inner.folded,
"month_spend": inner.month_spend,
"month_upto_ts": now_secs(),
});
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, payload.to_string()).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
/// older than the retention window fold into the cumulative per-account
/// sidecar and are dropped from the file (tmp + rename on both files, so
@@ -305,6 +370,10 @@ impl Stats {
return;
}
inner.writes_since_check = 0;
// piggyback: persist month-to-date spend so budgets survive
// restarts even between compactions
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
}
}
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
@@ -359,14 +428,8 @@ impl Stats {
}
}
// persist the folded totals FIRST, then swap the trimmed file in
if let Some(tpath) = self.totals_path() {
if let Ok(json) = serde_json::to_string(&inner.folded) {
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
}
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
let tmp = format!("{path}.tmp");
if std::fs::write(&tmp, kept).is_ok() {
let _ = std::fs::rename(&tmp, path);
@@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
}
fn month_of(ts: u64) -> String {
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
let days = ts / 86_400;
let mut year = 1970u64;
let mut d = days;
loop {
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let len = if leap { 366 } else { 365 };
if d < len {
break;
}
d -= len;
year += 1;
}
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let ml = [
31,
if leap { 29 } else { 28 },
31,
30,
31,
30,
31,
31,
30,
31,
30,
31,
];
let mut month = 1;
for len in ml {
if d < len {
break;
}
d -= len;
month += 1;
}
format!("{year:04}-{month:02}")
}
fn avg(sum: u64, n: u64) -> u64 {
sum.checked_div(n).unwrap_or(0)
}
+25
View File
@@ -597,6 +597,26 @@ body.modal-open { overflow: hidden; }
.pair-model { min-width: 220px; max-width: 340px; padding: 4px 8px; font-size: 12.5px; }
.pair-tier { max-width: 150px; padding: 4px 8px; font-size: 12.5px; }
/* Complexity bands slider */
.bands-wrap { margin: 4px 0 16px; }
/* one-line command chip (install commands etc.) — theme-aware */
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
.bands-track.disabled { opacity: .45; pointer-events: none; }
.bands-zone { position: absolute; top: 0; bottom: 0; }
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
.bands-handle:active { cursor: grabbing; }
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
.bands-val.raised { top: -46px; }
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
/* Benchmark view */
@keyframes bm-in { from { opacity: 0; transform: translateY(-3px); } to { opacity: 1; transform: none; } }
tbody tr:last-child { animation: bm-in .25s ease; }
@@ -686,3 +706,8 @@ tbody tr:last-child { animation: bm-in .25s ease; }
@media (prefers-reduced-motion: reduce) {
.sidebar, .mobile-nav-backdrop { transition: none; }
}
/* full-screen farewell after "stop everything & power off" */
.shutdown-screen { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100vh; gap: 12px; text-align: center; padding: 24px; }
.shutdown-screen .shutdown-icon { font-size: 64px; color: var(--bad); }
.shutdown-screen .hint { max-width: 480px; }
+2
View File
@@ -74,6 +74,7 @@ export const api = {
// protocols
getProtocols: () => j("GET", "/protocols"),
harnesses: () => j("GET", "/harnesses"),
balance: () => j("GET", "/balance"),
putProtocols: (b) => j("PUT", "/protocols", b),
// settings
@@ -133,6 +134,7 @@ export const api = {
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
clearLogs: () => j("DELETE", "/logs"),
restart: () => j("POST", "/restart"),
shutdown: () => j("POST", "/shutdown"),
benchStart: (model) => j("POST", "/bench", { model }),
benchList: () => j("GET", "/bench"),
+5
View File
@@ -48,6 +48,11 @@ export function onLangChange(fn) {
return () => listeners.delete(fn);
}
// Human name for a complexity tier; custom tier names pass through as-is.
export function trTier(tier) {
return ["low", "medium", "high"].includes(tier) ? t("tier." + tier) : tier;
}
export function t(key, vars) {
const d = DICTS[lang] || en;
let s = d[key] ?? en[key] ?? key;
+23 -2
View File
@@ -536,9 +536,13 @@ export default {
"keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
"keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
"keys.snip.keyPh": "dein API-Schlüssel",
"settings.restart": "Gateway neu starten",
"settings.restartConfirm": "Gateway neu starten? Laufende Anfragen brechen ab; lokale Modellserver starten ebenfalls neu.",
"settings.restart": "Alles neu starten",
"settings.restartConfirm": "Gateway und alle lokalen Modellserver neu starten? Laufende Anfragen werden abgebrochen.",
"settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
"settings.shutdown": "Alles stoppen & ausschalten",
"settings.shutdownConfirm": "Alle lokalen Modellserver stoppen und das Gateway ausschalten? API und dieses Panel antworten erst wieder, wenn Sie das Gateway neu starten.",
"settings.shutdownDone": "Das Gateway ist ausgeschaltet.",
"settings.shutdownHint": "Alle lokalen Modellserver sind gestoppt, der Prozess wurde beendet. Starten Sie das Gateway auf dem Server neu, um das Panel zurückzubekommen.",
"bench.title": "Geschwindigkeit lokaler Modelle",
"bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
"bench.run": "Messen",
@@ -629,4 +633,21 @@ export default {
"routing.tasks.tierHint": "Welches Modell diese Aufgabe bei welcher Komplexität bedient; \"jede\" führt auf allen Stufen. Reihenfolge = Fallback-Reihenfolge. High eskaliert weiterhin über \"jede\"-Einträge.",
"routing.tasks.tierAny": "jede Komplexität",
"routing.tasks.addModel": "Modell",
"routing.bands.enable": "Eigene Komplexitätsgrenzen",
"routing.bands.hint": "Ziehen Sie die Griffe: wo low endet und high beginnt — die Griffe schieben einander, die Zahlen aktualisieren sich live. Der kontinuierliche Router-Score wird mit IHREN Grenzen neu eingeteilt; ohne Häkchen gelten die Grenzen des Routers.",
"tier.low": "leicht",
"tier.medium": "mittel",
"tier.high": "schwer",
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
"settings.cache.entries": "Einträge",
"settings.cache.hits": "Treffer",
"settings.cache.saved": "gespart",
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
"keys.budgetOf": "Budget {b}/Mon.",
"dash.kpi.orBalance": "OpenRouter-Guthaben",
"keys.h.install": "Installieren:",
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
"keys.snip.apply": "Schlüssel einsetzen",
};
+23 -2
View File
@@ -529,9 +529,13 @@ export default {
"keys.snip.withKey": "Snippets below include the key you just created.",
"keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
"keys.snip.keyPh": "your API key",
"settings.restart": "Restart gateway",
"settings.restartConfirm": "Restart the gateway? Requests in flight will be dropped; local model servers restart too.",
"settings.restart": "Restart everything",
"settings.restartConfirm": "Restart the gateway and all local model servers? Requests in flight will be dropped.",
"settings.restarting": "Restarting — the page reloads when the gateway is back…",
"settings.shutdown": "Stop everything & power off",
"settings.shutdownConfirm": "Stop all local model servers and power the gateway off? The API and this panel stop answering until you start the gateway again.",
"settings.shutdownDone": "The gateway is off.",
"settings.shutdownHint": "All local model servers are stopped and the process has exited. Start the gateway again on the server to bring the panel back.",
"bench.title": "Local model speed",
"bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
"bench.run": "Measure",
@@ -622,4 +626,21 @@ export default {
"routing.tasks.tierHint": "Which model serves this task at which complexity; \"any\" leads at every tier. Order = fallback order. High verdicts still escalate over any-tier pins.",
"routing.tasks.tierAny": "any complexity",
"routing.tasks.addModel": "model",
"routing.bands.enable": "My complexity boundaries",
"routing.bands.hint": "Drag the handles to set where low ends and where high begins — the handles push each other, the numbers update live. The router's continuous score is re-mapped with YOUR boundaries; unchecked = the router's own bands.",
"tier.low": "low",
"tier.medium": "medium",
"tier.high": "high",
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
"settings.cache.entries": "entries",
"settings.cache.hits": "hits",
"settings.cache.saved": "saved",
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
"keys.budgetOf": "budget {b}/mo",
"dash.kpi.orBalance": "OpenRouter balance",
"keys.h.install": "Install:",
"keys.snip.keyInputPh": "paste your API key",
"keys.snip.apply": "Insert key",
};
+23 -2
View File
@@ -536,9 +536,13 @@ export default {
"keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
"keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
"keys.snip.keyPh": "tu clave API",
"settings.restart": "Reiniciar la pasarela",
"settings.restartConfirm": "¿Reiniciar la pasarela? Las peticiones en curso se cortarán; los servidores de modelos locales también se reinician.",
"settings.restart": "Reiniciar todo",
"settings.restartConfirm": "¿Reiniciar la pasarela y todos los servidores de modelos locales? Las solicitudes en curso se interrumpirán.",
"settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
"settings.shutdown": "Detener todo y apagar",
"settings.shutdownConfirm": "¿Detener todos los servidores de modelos locales y apagar la pasarela? La API y este panel dejarán de responder hasta que vuelva a iniciar la pasarela.",
"settings.shutdownDone": "La pasarela está apagada.",
"settings.shutdownHint": "Todos los servidores de modelos locales están detenidos y el proceso ha terminado. Inicie de nuevo la pasarela en el servidor para recuperar el panel.",
"bench.title": "Velocidad de modelos locales",
"bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
"bench.run": "Medir",
@@ -629,4 +633,21 @@ export default {
"routing.tasks.tierHint": "Qué modelo atiende esta tarea en qué complejidad; «cualquiera» lidera en todos los niveles. El orden = orden de respaldo. Un veredicto high sigue escalando sobre las entradas «cualquiera».",
"routing.tasks.tierAny": "cualquier complejidad",
"routing.tasks.addModel": "modelo",
"routing.bands.enable": "Mis límites de complejidad",
"routing.bands.hint": "Arrastra los tiradores: dónde termina low y dónde empieza high — se empujan entre sí y las cifras se actualizan en vivo. La puntuación continua del router se re-mapea con TUS límites; sin marcar = las bandas del propio router.",
"tier.low": "fácil",
"tier.medium": "medio",
"tier.high": "difícil",
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
"settings.cache.entries": "entradas",
"settings.cache.hits": "aciertos",
"settings.cache.saved": "ahorrado",
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
"keys.budgetOf": "presupuesto {b}/mes",
"dash.kpi.orBalance": "Saldo OpenRouter",
"keys.h.install": "Instalar:",
"keys.snip.keyInputPh": "pega tu clave API",
"keys.snip.apply": "Insertar clave",
};
+23 -2
View File
@@ -536,9 +536,13 @@ export default {
"keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
"keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
"keys.snip.keyPh": "votre clé API",
"settings.restart": "Redémarrer la passerelle",
"settings.restartConfirm": "Redémarrer la passerelle ? Les requêtes en cours seront coupées ; les serveurs de modèles locaux redémarrent aussi.",
"settings.restart": "Tout redémarrer",
"settings.restartConfirm": "Redémarrer la passerelle et tous les serveurs de modèles locaux ? Les requêtes en cours seront interrompues.",
"settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
"settings.shutdown": "Tout arrêter et éteindre",
"settings.shutdownConfirm": "Arrêter tous les serveurs de modèles locaux et éteindre la passerelle ? L'API et ce panneau ne répondront plus tant que vous n'aurez pas relancé la passerelle.",
"settings.shutdownDone": "La passerelle est éteinte.",
"settings.shutdownHint": "Tous les serveurs de modèles locaux sont arrêtés et le processus est terminé. Relancez la passerelle sur le serveur pour retrouver le panneau.",
"bench.title": "Vitesse des modèles locaux",
"bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
"bench.run": "Mesurer",
@@ -629,4 +633,21 @@ export default {
"routing.tasks.tierHint": "Quel modèle sert cette tâche à quelle complexité ; « toute » mène à tous les niveaux. L'ordre = ordre de repli. Un verdict high escalade toujours au-dessus des entrées « toute ».",
"routing.tasks.tierAny": "toute complexité",
"routing.tasks.addModel": "modèle",
"routing.bands.enable": "Mes limites de complexité",
"routing.bands.hint": "Faites glisser les poignées : où finit low et où commence high — elles se poussent l'une l'autre, les chiffres se mettent à jour en direct. Le score continu du routeur est re-découpé avec VOS limites ; décoché = les bandes du routeur.",
"tier.low": "facile",
"tier.medium": "moyen",
"tier.high": "difficile",
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
"settings.cache.entries": "entrées",
"settings.cache.hits": "hits",
"settings.cache.saved": "économisé",
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
"keys.budgetOf": "budget {b}/mois",
"dash.kpi.orBalance": "Solde OpenRouter",
"keys.h.install": "Installer :",
"keys.snip.keyInputPh": "collez votre clé API",
"keys.snip.apply": "Insérer la clé",
};
+25 -4
View File
@@ -25,7 +25,7 @@ export default {
"common.all": "все",
// nav
"nav.dashboard": "Дашборд",
"nav.dashboard": "Рабочий стол",
"nav.models": "Модели",
"nav.import": "Импорт модели",
"nav.routing": "Роутинг",
@@ -56,7 +56,7 @@ export default {
"login.failed": "Неверный токен администратора.",
// dashboard
"dash.title": "Дашборд",
"dash.title": "Рабочий стол",
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
"dash.clear": "Очистить логи",
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
@@ -528,9 +528,13 @@ export default {
"keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
"keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
"keys.snip.keyPh": "ваш API-ключ",
"settings.restart": "Перезапустить шлюз",
"settings.restartConfirm": "Перезапустить шлюз? Текущие запросы оборвутся; серверы локальных моделей тоже перезапустятся.",
"settings.restart": "Перезапустить всё",
"settings.restartConfirm": "Перезапустить шлюз и все серверы локальных моделей? Текущие запросы оборвутся.",
"settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
"settings.shutdown": "Остановить всё и выключить",
"settings.shutdownConfirm": "Остановить все серверы локальных моделей и выключить шлюз? API и эта панель перестанут отвечать, пока вы не запустите шлюз заново.",
"settings.shutdownDone": "Шлюз выключен.",
"settings.shutdownHint": "Все серверы локальных моделей остановлены, процесс завершён. Запустите шлюз на сервере заново, чтобы вернуть панель.",
"bench.title": "Скорость локальных моделей",
"bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
"bench.run": "Замерить",
@@ -621,4 +625,21 @@ export default {
"routing.tasks.tierHint": "Какая модель обслуживает задачу на какой сложности; «любая» ведёт на всех уровнях. Порядок = порядок фолбэка. Вердикт high по-прежнему эскалирует поверх «любых».",
"routing.tasks.tierAny": "любая сложность",
"routing.tasks.addModel": "модель",
"routing.bands.enable": "Мои границы сложности",
"routing.bands.hint": "Тяните ручки: где кончается low и где начинается high — ручки толкают друг друга, цифры обновляются на лету. Непрерывный score роутера пересчитывается ВАШИМИ границами; галочка снята — действуют пороги самого роутера.",
"tier.low": "лёгкий",
"tier.medium": "средний",
"tier.high": "сложный",
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
"settings.cache.entries": "записей",
"settings.cache.hits": "попаданий",
"settings.cache.saved": "сэкономлено",
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
"keys.budgetOf": "бюджет {b}/мес",
"dash.kpi.orBalance": "Баланс OpenRouter",
"keys.h.install": "Установить:",
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
"keys.snip.apply": "Вставить ключ",
};
+23 -2
View File
@@ -536,9 +536,13 @@ export default {
"keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
"keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
"keys.snip.keyPh": "API anahtarın",
"settings.restart": "Ağ geçidini yeniden başlat",
"settings.restartConfirm": "Ağ geçidi yeniden başlatılsın mı? Devam eden istekler kesilir; yerel model sunucuları da yeniden başlar.",
"settings.restart": "Tümünü yeniden başlat",
"settings.restartConfirm": "Ağ geçidi ve tüm yerel model sunucuları yeniden başlatılsın mı? Devam eden istekler kesilecek.",
"settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
"settings.shutdown": "Tümünü durdur ve kapat",
"settings.shutdownConfirm": "Tüm yerel model sunucuları durdurulup ağ geçidi kapatılsın mı? Ağ geçidini yeniden başlatana kadar API ve bu panel yanıt vermez.",
"settings.shutdownDone": "Ağ geçidi kapalı.",
"settings.shutdownHint": "Tüm yerel model sunucuları durduruldu ve süreç sonlandı. Paneli geri getirmek için ağ geçidini sunucuda yeniden başlatın.",
"bench.title": "Yerel model hızı",
"bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
"bench.run": "Ölç",
@@ -629,4 +633,21 @@ export default {
"routing.tasks.tierHint": "Bu görevi hangi karmaşıklıkta hangi model karşılar; \"her\" tüm katmanlarda önde gider. Sıra = yedekleme sırası. High kararı \"her\" girdilerinin üzerine yine tırmanır.",
"routing.tasks.tierAny": "her karmaşıklık",
"routing.tasks.addModel": "model",
"routing.bands.enable": "Kendi karmaşıklık sınırlarım",
"routing.bands.hint": "Tutamaçları sürükleyin: low nerede biter, high nerede başlar — tutamaçlar birbirini iter, sayılar anlık güncellenir. Yönlendiricinin sürekli puanı SİZİN sınırlarınızla yeniden bölümlenir; işaret kaldırılırsa yönlendiricinin kendi bantları geçerli.",
"tier.low": "kolay",
"tier.medium": "orta",
"tier.high": "zor",
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
"settings.cache.entries": "kayıt",
"settings.cache.hits": "isabet",
"settings.cache.saved": "tasarruf",
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
"keys.budgetOf": "bütçe {b}/ay",
"dash.kpi.orBalance": "OpenRouter bakiyesi",
"keys.h.install": "Kurulum:",
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
"keys.snip.apply": "Anahtarı ekle",
};
+23 -2
View File
@@ -536,9 +536,13 @@ export default {
"keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
"keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
"keys.snip.keyPh": "你的 API 密钥",
"settings.restart": "重启网关",
"settings.restartConfirm": "重启网关?进行中的请求会中断;本地模型服务器也会重启。",
"settings.restart": "全部重启",
"settings.restartConfirm": "重启网关和所有本地模型服务器?正在处理的请求将被中断。",
"settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
"settings.shutdown": "全部停止并关机",
"settings.shutdownConfirm": "停止所有本地模型服务器并关闭网关?在你重新启动网关之前,API 和此面板都将无法响应。",
"settings.shutdownDone": "网关已关闭。",
"settings.shutdownHint": "所有本地模型服务器已停止,进程已退出。请在服务器上重新启动网关以恢复面板。",
"bench.title": "本地模型速度",
"bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s,一次一个模型。",
"bench.run": "测速",
@@ -629,4 +633,21 @@ export default {
"routing.tasks.tierHint": "哪个模型在哪个复杂度上处理该任务;「任意」在所有层级领先。顺序即回退顺序。high 判定仍会越过「任意」条目升级。",
"routing.tasks.tierAny": "任意复杂度",
"routing.tasks.addModel": "模型",
"routing.bands.enable": "自定义复杂度边界",
"routing.bands.hint": "拖动手柄设定 low 到哪里结束、high 从哪里开始 — 手柄会相互推动,数字实时更新。路由器的连续得分将按你的边界重新划分;取消勾选则使用路由器自身的边界。",
"tier.low": "简单",
"tier.medium": "中等",
"tier.high": "困难",
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
"settings.cache.entries": "条目",
"settings.cache.hits": "命中",
"settings.cache.saved": "已节省",
"keys.form.budget": "每月预算,$(0 = 不限)",
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
"keys.budgetOf": "预算 {b}/月",
"dash.kpi.orBalance": "OpenRouter 余额",
"keys.h.install": "安装:",
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
"keys.snip.apply": "插入密钥",
};
+34 -19
View File
@@ -8,7 +8,7 @@
// (keyword checks with | synonyms, think-tag stripping, anti-loop detection,
// pass/partial/fail), now embedded, localized and CDN-free.
import { h, mount, toast, ms, money, moneyEl } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api, testStream } from "../api.js";
// ── built-in suite ──────────────────────────────────────────────────────────
@@ -216,24 +216,35 @@ export async function renderBenchmark() {
let ev;
if (test.judge === "route") {
// the test asserts the ROUTER's decision: did it rate this high?
// One retry with a FRESH template: the router deliberately scores
// familiar text lower (novelty), so a single burnt wording must
// not fail the whole assertion.
// Two honest retries. (1) The SAME prompt after a pause: the
// router judges difficulty in the background — the first-ever
// occurrence routes by prior, the judgement lands in the cache a
// couple of seconds later, so the second occurrence is the real
// steady-state answer. (2) A fresh template, in case this one's
// wording clustered into a burnt region.
if (info.tier !== "high" && test.gen) {
const p2 = hardPrompt(test.gen);
try {
const second = await testStream(
{ model, messages: [{ role: "user", content: p2 }], temperature: 0, max_tokens: (test.max || 128) + 256 },
() => {}
);
if (second.tier === "high") {
info.tier = second.tier;
info.selected_model = second.selected_model;
info.task_label = second.task_label;
info.route_source = second.route_source;
info.cost_usd = (info.cost_usd || 0) + (second.cost_usd || 0);
}
} catch { /* keep first attempt */ }
const attempts = [
{ prompt, delayMs: 4000 },
{ prompt: hardPrompt(test.gen), delayMs: 0 },
];
for (const a of attempts) {
if (info.tier === "high") break;
if (a.delayMs) await new Promise((r) => setTimeout(r, a.delayMs));
try {
const again = await testStream(
{ model, messages: [{ role: "user", content: a.prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
() => {}
);
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
if (again.tier === "high") {
info.tier = again.tier;
info.score = again.score;
info.selected_model = again.selected_model;
info.task_label = again.task_label;
info.route_source = again.route_source;
}
} catch { /* keep previous attempt */ }
}
}
ev = info.tier === "high"
? { status: "pass", score: 1, note: "high ✓" }
@@ -259,6 +270,8 @@ export async function renderBenchmark() {
|| String(info.route_source || "").includes("degraded");
row.task = info.task_label || "—";
row.tier = info.tier || "—";
row.cscore = info.score || 0; // непрерывный коэффициент сложности
// (row.score занят долей правильности ответа из evaluate())
row.chosen = info.selected_model || "—";
row.source = info.route_source || "—";
row.expected = test.label; // null = classification not judged
@@ -392,7 +405,9 @@ export async function renderBenchmark() {
: h("span", { class: r.taskOk ? "" : "err" },
(r.taskOk ? "✓ " : "✗ ") + r.task,
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
h("td", {},
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
h("td", {},
h("span", { class: "mono small" }, r.chosen || "—"),
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
+12 -4
View File
@@ -1,6 +1,6 @@
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
import { SITE_URL } from "../app.js";
@@ -110,12 +110,13 @@ export async function renderDashboard() {
async function load() {
const rangeSecs = range;
let stats, recent, health;
let stats, recent, health, balance;
try {
[stats, recent, health] = await Promise.all([
[stats, recent, health, balance] = await Promise.all([
api.stats(rangeSecs, groupby),
api.requests(40, 0),
api.health().catch(() => null),
api.balance().catch(() => null),
]);
} catch (e) {
mount(root, h("div", { class: "callout" }, String(e.message || e)));
@@ -162,6 +163,11 @@ export async function renderDashboard() {
kpi(num(tot.requests), t("dash.kpi.requests")),
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
balance && balance.openrouter_usd != null
? kpi(
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
t("dash.kpi.orBalance"))
: null,
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
kpi(pct(tot.success_rate), t("dash.kpi.success")),
kpi(num(tot.failovers), t("dash.kpi.failovers"))
@@ -271,7 +277,9 @@ function recentTable(rows) {
{},
h("td", { class: "mono" }, timeAgo(r.ts)),
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
h("td", {},
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
h("td", {}, r.task_label || "—"),
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
h("td", { class: "mono" }, money(r.cost_usd)),
+147 -14
View File
@@ -10,12 +10,14 @@ export async function renderKeys() {
function openCreate(reload) {
const accountIn = h("input", { placeholder: "team-a" });
const rateIn = h("input", { type: "number", value: 600 });
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
const customIn = h("input", { placeholder: "sk-gw-…" });
const body = h(
"div",
{},
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
);
modal(t("keys.form.title"), body, async () => {
@@ -24,6 +26,7 @@ export async function renderKeys() {
key: customIn.value.trim(),
account: accountIn.value.trim() || "default",
rate_per_min: parseInt(rateIn.value) || 0,
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
allow_models: [],
});
reload();
@@ -51,18 +54,76 @@ export async function renderKeys() {
function snippetsCard(openMode) {
const origin = location.origin;
const base = origin + "/v1";
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
// Every scripty snippet exists in two OS flavors: bash heredocs for
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
// (python/js) and GUI instructions are OS-independent.
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
const snips = {
"Claude Code": `# ${t("keys.h.claude")}
export ANTHROPIC_BASE_URL="${origin}"
const buildSnips = (osKind) => osKind === "ps" ? {
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
$env:ANTHROPIC_MODEL = "cortiq-auto"
claude`,
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
Add-Content "$HOME\\.codex\\config.toml" @'
model = "cortiq-auto"
model_provider = "cortiq"
[model_providers.cortiq]
name = "Cortiq Gateway"
base_url = "${base}"
env_key = "CORTIQ_API_KEY"
wire_api = "chat"
'@
$env:CORTIQ_API_KEY = "${key}"
codex`,
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"cortiq": {
"npm": "@ai-sdk/openai-compatible",
"name": "Cortiq Gateway",
"options": { "baseURL": "${base}", "apiKey": "${key}" },
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
}
}
}
'@
opencode`,
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
Add-Content "$HOME\\.continue\\config.yaml" @'
models:
- name: Cortiq Auto
provider: openai
model: cortiq-auto
apiBase: ${base}
apiKey: ${key}
'@`,
Cline: `${t("keys.h.cline1")}
API Provider: OpenAI Compatible
Base URL: ${base}
API Key: ${key}
Model ID: cortiq-auto
${t("keys.h.cline2")}`,
curl: `curl.exe ${base}/chat/completions \`
-H "Authorization: Bearer ${key}" \`
-H "Content-Type: application/json" \`
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
} : {
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
export ANTHROPIC_AUTH_TOKEN="${key}"
export ANTHROPIC_MODEL="cortiq-auto"
claude`,
Codex: `# ${t("keys.h.codex")}
mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
model = "cortiq-auto"
model_provider = "cortiq"
@@ -75,8 +136,7 @@ EOF
export CORTIQ_API_KEY="${key}"
codex`,
OpenCode: `# ${t("keys.h.opencode")}
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
@@ -91,8 +151,7 @@ mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
EOF
opencode`,
Continue: `# ${t("keys.h.continue")}
mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
models:
- name: Cortiq Auto
provider: openai
@@ -128,8 +187,29 @@ const resp = await client.chat.completions.create({
});
console.log(resp.choices[0].message.content);`,
};
let shSnips, psSnips, snips;
function rebuildSnips() {
shSnips = buildSnips("sh");
psSnips = buildSnips("ps");
// python/js/Cline одинаковы — дополняем ps-набор из sh
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
snips = os === "ps" ? psSnips : shSnips;
}
rebuildSnips();
let cur = "Claude Code";
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
// in a pasted snippet is not a comment in default interactive zsh, and a
// "!" inside it triggers history expansion ("event not found")
const HINTS = {
"Claude Code": () => t("keys.h.claude"),
Codex: () => t("keys.h.codex"),
OpenCode: () => t("keys.h.opencode"),
Continue: () => t("keys.h.continue"),
};
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
paintHint();
const seg = h(
"div",
{ class: "seg" },
@@ -137,6 +217,7 @@ console.log(resp.choices[0].message.content);`,
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
cur = k;
pre.textContent = snips[k];
paintHint();
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
paintStatus();
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
@@ -164,7 +245,15 @@ console.log(resp.choices[0].message.content);`,
}).catch(() => {});
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
// status line under the snippet ("installed" / "already configured").
// status line under the snippet ("installed" / "already configured") —
// and a one-line INSTALL command (per OS) when it's missing.
const INSTALL = {
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
};
let detected = {};
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
function paintStatus() {
@@ -176,7 +265,19 @@ console.log(resp.choices[0].message.content);`,
statusHost.innerHTML = "";
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
else mount(statusHost, h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")));
else {
const cmd = (INSTALL[cur] || {})[os];
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
await navigator.clipboard.writeText(cmd);
toast(t("common.copied"), "good");
} }, "⧉");
mount(statusHost,
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
h("span", { class: "muted" }, t("keys.h.install")),
h("code", { class: "mono small cmd-chip" }, cmd),
copyInstall) : null);
}
}
api.harnesses().then((d) => {
detected = d || {};
@@ -187,6 +288,32 @@ console.log(resp.choices[0].message.content);`,
paintStatus();
}).catch(() => {});
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
const osSeg = h(
"div",
{ class: "seg" },
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
);
osSeg.querySelectorAll("button").forEach((b, i) =>
b.addEventListener("click", () => {
os = i === 0 ? "sh" : "ps";
rebuildSnips();
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
pre.textContent = snips[cur];
paintStatus();
}));
// paste-your-key: substitutes the key into every snippet immediately
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
const applyKey = () => {
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
rebuildSnips();
pre.textContent = snips[cur];
};
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
copy.addEventListener("click", async () => {
await navigator.clipboard.writeText(pre.textContent);
@@ -198,7 +325,8 @@ console.log(resp.choices[0].message.content);`,
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" },
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), copy),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
hintHost,
pre,
statusHost,
protoWarn,
@@ -273,7 +401,12 @@ console.log(resp.choices[0].message.content);`,
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
h("td", { class: "right-align" },
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
k.monthly_budget_usd
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
: null),
h(
"td",
{ class: "right-align" },
+2 -2
View File
@@ -5,7 +5,7 @@
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
import { h, mount, toast, money, moneyEl, ms } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
import { appState } from "../app.js";
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
@@ -206,7 +206,7 @@ export async function renderPlayground() {
decision,
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
kv(t("pg.task"), c.task_label || "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, trTier(c.complexity.tier)) : "—"),
h(
"div",
{ class: "kv", style: "display:block" },
+92 -4
View File
@@ -1,6 +1,6 @@
// Routing — tier editor (ordered model chips), default model, and policy.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
@@ -71,6 +71,7 @@ export async function renderRouting() {
const st = {
tiers,
taskPairs,
bands: routing.bands || null, // null = пороги роутера как есть
default: routing.default || ids[0] || "",
policy: routing.policy || { mode: "fixed_table", min_class: {} },
};
@@ -120,11 +121,97 @@ export async function renderRouting() {
);
}
// Linked boundary sliders: one track, two handles (low|medium boundaries),
// three colored zones. Handles push each other, numbers update live —
// nobody types score ranges by hand.
function bandsCard() {
const DEF = { low: 0.33, medium: 0.48 };
const enabled = !!st.bands;
const b = st.bands || { ...DEF };
const MIN_GAP = 0.02;
const track = h("div", { class: "bands-track" });
const zoneLow = h("div", { class: "bands-zone z-low" });
const zoneMed = h("div", { class: "bands-zone z-med" });
const zoneHigh = h("div", { class: "bands-zone z-high" });
const hLow = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
const hMed = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
const legend = h("div", { class: "bands-legend" });
track.append(zoneLow, zoneMed, zoneHigh, hLow, hMed);
function paint() {
const lo = b.low, me = b.medium;
zoneLow.style.width = lo * 100 + "%";
zoneMed.style.left = lo * 100 + "%";
zoneMed.style.width = (me - lo) * 100 + "%";
zoneHigh.style.left = me * 100 + "%";
zoneHigh.style.width = (1 - me) * 100 + "%";
hLow.style.left = lo * 100 + "%";
hMed.style.left = me * 100 + "%";
hLow.querySelector(".bands-val").textContent = lo.toFixed(2);
hMed.querySelector(".bands-val").textContent = me.toFixed(2);
// подписи не наезжают, когда ручки рядом
hMed.querySelector(".bands-val").classList.toggle("raised", me - lo < 0.09);
mount(legend,
h("span", { class: "badge tier-low" }, `${trTier("low")} ≤ ${lo.toFixed(2)}`),
h("span", { class: "badge tier-medium" }, `${trTier("medium")} ≤ ${me.toFixed(2)}`),
h("span", { class: "badge tier-high" }, `${trTier("high")} > ${me.toFixed(2)}`));
track.classList.toggle("disabled", !st.bands);
}
function drag(handle, which) {
handle.addEventListener("pointerdown", (e) => {
if (!st.bands) return;
handle.setPointerCapture(e.pointerId);
const rect = track.getBoundingClientRect();
const move = (ev) => {
let v = (ev.clientX - rect.left) / rect.width;
v = Math.min(0.98, Math.max(0.02, Math.round(v * 100) / 100));
if (which === "low") {
b.low = v;
// связанность: low толкает medium перед собой
if (b.medium < b.low + MIN_GAP) b.medium = Math.min(0.99, b.low + MIN_GAP);
} else {
b.medium = v;
// и medium толкает low назад
if (b.low > b.medium - MIN_GAP) b.low = Math.max(0.01, b.medium - MIN_GAP);
}
st.bands = { low: b.low, medium: b.medium };
paint();
};
const up = () => {
handle.removeEventListener("pointermove", move);
handle.removeEventListener("pointerup", up);
};
handle.addEventListener("pointermove", move);
handle.addEventListener("pointerup", up);
});
}
drag(hLow, "low");
drag(hMed, "medium");
const toggle = h("input", { type: "checkbox", checked: enabled ? "" : null });
toggle.checked = enabled;
toggle.addEventListener("change", () => {
st.bands = toggle.checked ? { low: b.low, medium: b.medium } : null;
paint();
});
paint();
return h(
"div",
{ class: "bands-wrap" },
h("label", { class: "check", style: "margin-bottom:8px" }, toggle, t("routing.bands.enable")),
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.bands.hint")),
track,
legend
);
}
function tierRow(tier) {
return h(
"div",
{ class: "tierrow" },
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, trTier(tier))),
chipEditor(st.tiers, tier)
);
}
@@ -167,7 +254,7 @@ export async function renderRouting() {
"select",
{ class: "pair-tier", title: t("routing.tasks.tierHint") },
h("option", { value: "any" }, t("routing.tasks.tierAny")),
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, tr))
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, trTier(tr)))
);
tierSel.value = p.tier;
tierSel.addEventListener("change", () => { p.tier = tierSel.value; });
@@ -257,7 +344,7 @@ export async function renderRouting() {
}
if (Object.keys(rule).length) tasks[k] = rule;
}
await api.putRouting({ tiers: st.tiers, tasks, default: st.default, policy: st.policy });
await api.putRouting({ tiers: st.tiers, tasks, bands: st.bands, default: st.default, policy: st.policy });
toast(t("toast.saved"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
@@ -282,6 +369,7 @@ export async function renderRouting() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
bandsCard(),
...Object.keys(st.tiers).map(tierRow)
),
tasksCard(),
+51 -2
View File
@@ -194,8 +194,30 @@ export async function renderSettings() {
const modelsData = await api.listModels();
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
// Live proof-of-life for the cache card: entries/hits/saved — and an
// honest warning when the cache is on but has no embedding model to work
// with (it silently does nothing in that state).
let cacheStats = null;
try {
cacheStats = (await api.stats("24h", "model")).cache || null;
} catch { /* stats off */ }
function cacheStatus() {
const wrap = h("div", { style: "margin-bottom:10px" });
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
return wrap;
}
if (cacheStats && s.cache?.enabled) {
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
}
return wrap;
}
const cacheEmbedSel = h(
"select",
{},
@@ -537,6 +559,31 @@ export async function renderSettings() {
}
});
const stopBtn = h("button", { class: "btn danger" }, "\u23fb " + t("settings.shutdown"));
stopBtn.addEventListener("click", async () => {
if (!(await confirmDialog(t("settings.shutdownConfirm")))) return;
stopBtn.disabled = true;
restartBtn.disabled = true;
try {
await api.shutdown();
// the server is gone in ~300ms — replace the app with a static farewell
document.body.innerHTML = "";
document.body.appendChild(
h(
"div",
{ class: "shutdown-screen" },
h("div", { class: "shutdown-icon" }, "\u23fb"),
h("h2", {}, t("settings.shutdownDone")),
h("p", { class: "hint" }, t("settings.shutdownHint"))
)
);
} catch (e) {
toast(String(e.message || e), "bad");
stopBtn.disabled = false;
restartBtn.disabled = false;
}
});
mount(
root,
h(
@@ -610,6 +657,7 @@ export async function renderSettings() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
cacheStatus(),
cacheEnabled.node,
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
field(t("settings.cache.embed"), cacheEmbedSel),
@@ -659,7 +707,8 @@ export async function renderSettings() {
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
fileIn,
h("div", { class: "grow" }),
restartBtn
restartBtn,
stopBtn
)
)
);