Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1c59906b49 | ||
|
|
19f666e7b1 | ||
|
|
dbcd9827f6 | ||
|
|
b689180717 | ||
|
|
f385c2254b | ||
|
|
9607a38792 | ||
|
|
239ddc767c | ||
|
|
057abaa95a | ||
|
|
dd864eed22 | ||
|
|
c1ff675d80 | ||
|
|
9ef1af3ae1 | ||
|
|
6f4d83ce13 | ||
|
|
0040bfd7e9 | ||
|
|
10b5101757 | ||
|
|
788c87c565 | ||
|
|
179c414a0e |
@@ -3,6 +3,74 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.43] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
|
||||
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
|
||||
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
|
||||
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
|
||||
configured = open mode, as before.
|
||||
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
|
||||
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
|
||||
when the server becomes healthy, so a local model whose chat template can drive tool
|
||||
calling serves agent (tools) traffic with zero manual configuration. Manual override:
|
||||
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
|
||||
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
|
||||
remaining credit is a KPI card, red under $5.
|
||||
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
|
||||
per harness, an install command chip when the harness is missing, and a paste-your-key
|
||||
field that substitutes the real key into every snippet.
|
||||
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
|
||||
Caddy TLS fronting, security checklist, backup list.
|
||||
|
||||
### Fixed
|
||||
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
|
||||
route's chain has no model with the required capabilities (tools), the gateway escalates
|
||||
through medium→high chains and then the rest of the pool instead of returning
|
||||
"No healthy model available". Explicit pins still fail loudly.
|
||||
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
|
||||
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
|
||||
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
|
||||
no history-expansion `!`), and the install chips are readable in the light theme.
|
||||
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
|
||||
into the sidecar totals file — per-account all-time and month spend survive compaction
|
||||
and restarts.
|
||||
|
||||
### Breaking
|
||||
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
|
||||
will now be rejected with 401 — either remove the keys from the config or start sending
|
||||
`Authorization: Bearer <key>`.
|
||||
|
||||
## [0.2.42] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **The continuous complexity score is visible**: the benchmark's routing table and the
|
||||
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
|
||||
see what different prompts actually weigh and tune the boundary sliders from real traffic.
|
||||
|
||||
### Changed
|
||||
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
|
||||
«Рабочий стол».
|
||||
|
||||
## [0.2.41] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
|
||||
(the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
|
||||
other, the numbers ride on the handles and the legend updates live — no hand-typed score
|
||||
ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
|
||||
(`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
|
||||
stats; unchecked = the router's own bands, untouched behavior.
|
||||
|
||||
### Fixed
|
||||
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
|
||||
selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
|
||||
facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
|
||||
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
|
||||
fresh template — difficulty is judged in the background, so the second occurrence is the
|
||||
honest steady-state answer.
|
||||
|
||||
## [0.2.40] - 2026-08-08
|
||||
|
||||
### Added
|
||||
|
||||
Generated
+1
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.40"
|
||||
version = "0.2.43"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.40"
|
||||
version = "0.2.43"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
|
||||
@@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
|
||||
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
|
||||
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
|
||||
|
||||
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Admin console
|
||||
|
||||
@@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
||||
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
|
||||
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
|
||||
|
||||
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Панель управления
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
# Deploying cortiq-gateway for a team / office
|
||||
|
||||
One gateway on a small server, API keys per department, local models for the
|
||||
cheap tiers, cloud models for the hard ones. This page covers the ops side.
|
||||
|
||||
## 1. Run as a service
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Task Scheduler → Create Task → "Run whether user is logged on or not",
|
||||
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
|
||||
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS and the outside world
|
||||
|
||||
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
|
||||
and bind the gateway to localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE streaming
|
||||
proxy_read_timeout 3600s; # long generations
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
|
||||
same with automatic certificates.)
|
||||
|
||||
## 3. Security checklist
|
||||
|
||||
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
|
||||
the console warns that it is open to the network.
|
||||
- **API keys**: create one per department/developer on the keys page. With at
|
||||
least one key configured the model APIs require Bearer / x-api-key —
|
||||
requests without a valid key get 401.
|
||||
- **Rate limits**: per-key requests/min → 429 with Retry-After.
|
||||
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
|
||||
account's month-to-date spend crosses it.
|
||||
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
|
||||
|
||||
## 4. Backups
|
||||
|
||||
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
|
||||
- `secrets.toml` — provider API keys (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
|
||||
cumulative per-key totals and month-to-date budget spend.
|
||||
|
||||
Copy those three and you can rebuild the instance anywhere.
|
||||
|
||||
## 5. Observability
|
||||
|
||||
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
|
||||
OpenRouter balance (red under $5).
|
||||
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
|
||||
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Развёртывание cortiq-gateway для команды / офиса
|
||||
|
||||
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
|
||||
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
|
||||
|
||||
## 1. Запуск как сервис
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
|
||||
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
|
||||
--no-port-search`. Либо [NSSM](https://nssm.cc):
|
||||
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS и внешний мир
|
||||
|
||||
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
|
||||
привяжите к localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE-стриминг
|
||||
proxy_read_timeout 3600s; # долгие генерации
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
|
||||
с автоматическими сертификатами.)
|
||||
|
||||
## 3. Чек-лист безопасности
|
||||
|
||||
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
|
||||
него консоль предупреждает, что открыта сети.
|
||||
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
|
||||
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
|
||||
запросы без валидного ключа получают 401.
|
||||
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
|
||||
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
|
||||
траты аккаунта за месяц превысят лимит.
|
||||
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
|
||||
|
||||
## 4. Бэкапы
|
||||
|
||||
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
|
||||
- `secrets.toml` — ключи провайдеров (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
|
||||
тоталы по ключам и месячные траты для бюджетов.
|
||||
|
||||
Скопируйте эти три файла — и инстанс поднимается где угодно.
|
||||
|
||||
## 5. Наблюдаемость
|
||||
|
||||
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
|
||||
сложности, баланс OpenRouter (красный ниже $5).
|
||||
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
|
||||
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
|
||||
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
get(get_protocols).put(put_protocols),
|
||||
)
|
||||
.route("/admin/api/harnesses", get(get_harnesses))
|
||||
.route("/admin/api/balance", get(get_balance))
|
||||
.route("/admin/api/settings", get(get_settings).put(put_settings))
|
||||
.route("/admin/api/keys", get(list_keys).post(create_key))
|
||||
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
|
||||
@@ -931,6 +932,7 @@ async fn get_routing(State(state): State<SharedState>) -> ApiResult {
|
||||
.collect();
|
||||
ok(json!({
|
||||
"tiers": tiers_to_map(&live.cfg.routing),
|
||||
"bands": live.cfg.routing.bands,
|
||||
"tasks": tasks,
|
||||
"default": live.cfg.routing.default,
|
||||
"policy": live.cfg.routing.policy,
|
||||
@@ -978,6 +980,8 @@ struct RoutingBody {
|
||||
tiers: HashMap<String, Vec<String>>,
|
||||
#[serde(default)]
|
||||
tasks: HashMap<String, TierTargets>,
|
||||
#[serde(default)]
|
||||
bands: Option<crate::config::TierBands>,
|
||||
default: String,
|
||||
#[serde(default)]
|
||||
policy: RoutingPolicy,
|
||||
@@ -997,6 +1001,7 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
|
||||
// empty rule (no models AND no forced tier) = "по сложности"
|
||||
.filter(|(_, v)| !v.models().is_empty())
|
||||
.collect(),
|
||||
bands: body.bands,
|
||||
default: body.default,
|
||||
policy: body.policy,
|
||||
};
|
||||
@@ -1004,6 +1009,51 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
|
||||
ok(json!({ "ok": true }))
|
||||
}
|
||||
|
||||
// ─────────────────────────────── balance ───────────────────────────────────
|
||||
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
|
||||
// (null) when no OpenRouter key is configured — the card hides itself.
|
||||
|
||||
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
|
||||
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
|
||||
std::sync::OnceLock::new();
|
||||
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
|
||||
if let Some((at, v)) = *cache.lock().unwrap() {
|
||||
if at.elapsed() < std::time::Duration::from_secs(300) {
|
||||
return ok(json!({ "openrouter_usd": v }));
|
||||
}
|
||||
}
|
||||
let key = state
|
||||
.secrets
|
||||
.resolve("OPENROUTER_API_KEY")
|
||||
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
|
||||
let Some(key) = key.filter(|k| !k.is_empty()) else {
|
||||
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
|
||||
};
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(8))
|
||||
.build()
|
||||
.unwrap_or_default();
|
||||
let resp = client
|
||||
.get("https://openrouter.ai/api/v1/credits")
|
||||
.bearer_auth(&key)
|
||||
.send()
|
||||
.await;
|
||||
let balance = match resp {
|
||||
Ok(r) if r.status().is_success() => {
|
||||
let v: serde_json::Value = r.json().await.unwrap_or_default();
|
||||
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
|
||||
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
|
||||
Some(total - used)
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
let Some(balance) = balance else {
|
||||
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
|
||||
};
|
||||
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
|
||||
ok(json!({ "openrouter_usd": balance }))
|
||||
}
|
||||
|
||||
// ──────────────────────────── harness detection ────────────────────────────
|
||||
// Which coding harnesses are present on the machine the gateway runs on, so the
|
||||
// API-keys view can mark tabs "installed" / "already configured for cortiq".
|
||||
@@ -1353,6 +1403,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
|
||||
"key_masked": mask_key(&k.key),
|
||||
"account": k.account,
|
||||
"rate_per_min": k.rate_per_min,
|
||||
"monthly_budget_usd": k.monthly_budget_usd,
|
||||
"allow_models": k.allow_models,
|
||||
})
|
||||
})
|
||||
@@ -1368,6 +1419,8 @@ struct NewKey {
|
||||
#[serde(default)]
|
||||
rate_per_min: u32,
|
||||
#[serde(default)]
|
||||
monthly_budget_usd: Option<f64>,
|
||||
#[serde(default)]
|
||||
allow_models: Vec<String>,
|
||||
}
|
||||
|
||||
@@ -1386,6 +1439,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
|
||||
account: b.account,
|
||||
rate_per_min: b.rate_per_min,
|
||||
allow_models: b.allow_models,
|
||||
monthly_budget_usd: b.monthly_budget_usd,
|
||||
});
|
||||
state.reload(cfg)?;
|
||||
// show the full key only once
|
||||
@@ -2143,6 +2197,7 @@ fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
});
|
||||
Some(id)
|
||||
}
|
||||
|
||||
+1
-1
@@ -55,7 +55,7 @@ impl SemanticCache {
|
||||
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
enabled: cfg.enabled,
|
||||
threshold: cfg.threshold,
|
||||
threshold: cfg.threshold as f32,
|
||||
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
|
||||
max_entries: cfg.max_entries.max(1),
|
||||
embed_model: cfg.embed_model.clone(),
|
||||
|
||||
+27
-5
@@ -56,6 +56,9 @@ struct Managed {
|
||||
pub struct CmfRuntime {
|
||||
status: Mutex<CmfStatus>,
|
||||
children: Mutex<Vec<Managed>>,
|
||||
/// Server ids whose healthz handshake advertised tool support — lets the
|
||||
/// pipeline route tools traffic to local models with no manual caps.
|
||||
pub detected_tools: Mutex<std::collections::HashSet<String>>,
|
||||
}
|
||||
|
||||
impl CmfRuntime {
|
||||
@@ -63,6 +66,7 @@ impl CmfRuntime {
|
||||
Arc::new(Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -193,6 +197,7 @@ impl Default for CmfRuntime {
|
||||
Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -332,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline
|
||||
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
|
||||
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
|
||||
/// caller can register them; older servers return `None` there.
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
|
||||
let url = format!("http://{host}:{port}/healthz");
|
||||
let client = reqwest::Client::new();
|
||||
for _ in 0..tries {
|
||||
@@ -344,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
.await
|
||||
{
|
||||
if r.status().is_success() {
|
||||
return true;
|
||||
let tools = r
|
||||
.json::<serde_json::Value>()
|
||||
.await
|
||||
.ok()
|
||||
.and_then(|v| v["capabilities"]["tools"].as_bool());
|
||||
return (true, tools);
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
false
|
||||
(false, None)
|
||||
}
|
||||
|
||||
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
||||
@@ -432,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
|
||||
if !rt.is_running(&server.id) {
|
||||
continue; // failed to spawn / file missing
|
||||
}
|
||||
if wait_healthy(&cfg.local_host, server.port, 240).await {
|
||||
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
|
||||
if ok {
|
||||
rt.set_healthy(&server.id, true);
|
||||
// capability handshake: the server said whether its model's chat
|
||||
// template can drive tool calling — no manual caps needed
|
||||
if tools == Some(true) {
|
||||
rt.detected_tools.lock().unwrap().insert(server.id.clone());
|
||||
tracing::info!(id = %server.id, "local server advertises tool support");
|
||||
} else {
|
||||
rt.detected_tools.lock().unwrap().remove(&server.id);
|
||||
}
|
||||
rt.log(format!(
|
||||
"local CMF server ready — '{}' at http://{}:{}/v1",
|
||||
server.id, cfg.local_host, server.port
|
||||
|
||||
+47
-3
@@ -130,6 +130,10 @@ pub struct CmfServer {
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
|
||||
/// gateway auto-detects tool support from the server's healthz handshake.
|
||||
#[serde(default)]
|
||||
pub caps: Vec<String>,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -229,6 +233,7 @@ impl CmfCfg {
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
@@ -404,6 +409,31 @@ pub struct RoutingCfg {
|
||||
/// called "tasks" — serde extracts it before the flattened tier map.
|
||||
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
|
||||
pub tasks: HashMap<String, TierTargets>,
|
||||
/// Local complexity bands: re-map the router's continuous score into
|
||||
/// low/medium/high with YOUR OWN boundaries. The hosted router's bands
|
||||
/// are global for all customers; these are per-gateway. `None` = keep
|
||||
/// the router's tier verdict as delivered.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub bands: Option<TierBands>,
|
||||
}
|
||||
|
||||
/// Score boundaries: `score ≤ low` → low, `≤ medium` → medium, else high.
|
||||
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
|
||||
pub struct TierBands {
|
||||
pub low: f32,
|
||||
pub medium: f32,
|
||||
}
|
||||
|
||||
impl TierBands {
|
||||
pub fn tier_of(&self, score: f32) -> &'static str {
|
||||
if score <= self.low {
|
||||
"low"
|
||||
} else if score <= self.medium {
|
||||
"medium"
|
||||
} else {
|
||||
"high"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A routing tier is either a list of model_ids or the string `default`.
|
||||
@@ -595,6 +625,9 @@ pub struct ApiKeyCfg {
|
||||
pub account: String,
|
||||
#[serde(default)]
|
||||
pub rate_per_min: u32,
|
||||
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub monthly_budget_usd: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub allow_models: Vec<String>,
|
||||
}
|
||||
@@ -804,9 +837,10 @@ fn default_shadow_soak() -> usize {
|
||||
pub struct CacheCfg {
|
||||
#[serde(default)]
|
||||
pub enabled: bool,
|
||||
/// Cosine-similarity threshold for a hit (0..1).
|
||||
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
|
||||
/// the number you typed (an f32 round-trips as 0.9200000166893005).
|
||||
#[serde(default = "default_semcache_threshold")]
|
||||
pub threshold: f32,
|
||||
pub threshold: f64,
|
||||
/// Entry time-to-live, e.g. `1h`.
|
||||
#[serde(default = "default_semcache_ttl")]
|
||||
pub ttl: String,
|
||||
@@ -828,7 +862,7 @@ impl Default for CacheCfg {
|
||||
}
|
||||
}
|
||||
}
|
||||
fn default_semcache_threshold() -> f32 {
|
||||
fn default_semcache_threshold() -> f64 {
|
||||
0.92
|
||||
}
|
||||
fn default_semcache_ttl() -> String {
|
||||
@@ -861,6 +895,7 @@ impl Default for Config {
|
||||
default: String::new(),
|
||||
policy: RoutingPolicy::default(),
|
||||
tasks: HashMap::new(),
|
||||
bands: None,
|
||||
},
|
||||
breaker: BreakerCfg::default(),
|
||||
protocols: ProtocolsCfg {
|
||||
@@ -988,6 +1023,15 @@ impl Config {
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(b) = &self.routing.bands {
|
||||
if !(b.low > 0.0 && b.low < b.medium && b.medium < 1.0) {
|
||||
anyhow::bail!(
|
||||
"config: routing.bands must satisfy 0 < low < medium < 1 (got {} / {})",
|
||||
b.low,
|
||||
b.medium
|
||||
);
|
||||
}
|
||||
}
|
||||
for (task, targets) in &self.routing.tasks {
|
||||
for id in &targets.models() {
|
||||
if !ids.contains(id.as_str()) {
|
||||
|
||||
@@ -13,6 +13,8 @@ pub enum GatewayError {
|
||||
Unauthorized(String),
|
||||
#[error("rate limit exceeded")]
|
||||
RateLimited { retry_after_secs: u64 },
|
||||
#[error("monthly budget exhausted (limit ${budget_usd})")]
|
||||
BudgetExhausted { budget_usd: f64 },
|
||||
#[error("upstream unavailable: {0}")]
|
||||
UpstreamUnavailable(String),
|
||||
#[error("internal error: {0}")]
|
||||
@@ -37,6 +39,11 @@ impl GatewayError {
|
||||
"rate_limit_error",
|
||||
"rate_limited",
|
||||
),
|
||||
Self::BudgetExhausted { .. } => (
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"insufficient_quota",
|
||||
"insufficient_quota",
|
||||
),
|
||||
Self::UpstreamUnavailable(_) => (
|
||||
StatusCode::BAD_GATEWAY,
|
||||
"upstream_unavailable",
|
||||
|
||||
+15
-3
@@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
|
||||
.route("/health", get(|| async { "ok" }))
|
||||
.route("/readyz", get(|| async { "ready" }))
|
||||
.route("/metrics", get(admin::metrics))
|
||||
.merge(protocols::build_router());
|
||||
.merge(protocols::build_router(state.clone()));
|
||||
|
||||
if admin_enabled {
|
||||
app = app
|
||||
@@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> {
|
||||
.allow_methods(tower_http::cors::Any)
|
||||
.allow_headers(tower_http::cors::Any);
|
||||
|
||||
let app = app.with_state(state).layer(cors);
|
||||
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
|
||||
// small enough that a hostile client can't balloon memory
|
||||
let app =
|
||||
app.with_state(state)
|
||||
.layer(cors)
|
||||
.layer(tower_http::limit::RequestBodyLimitLayer::new(
|
||||
32 * 1024 * 1024,
|
||||
));
|
||||
|
||||
let listener = {
|
||||
let mut attempt = 0u32;
|
||||
@@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
|
||||
tracing::info!("open this URL to finish setup: {url}");
|
||||
}
|
||||
}
|
||||
axum::serve(listener, app).await?;
|
||||
axum::serve(listener, app)
|
||||
.with_graceful_shutdown(async {
|
||||
let _ = tokio::signal::ctrl_c().await;
|
||||
tracing::info!("shutdown signal received — draining connections");
|
||||
})
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
+63
-4
@@ -183,6 +183,9 @@ impl Pipeline {
|
||||
&self,
|
||||
req: &ChatRequest,
|
||||
live: &crate::state::Live,
|
||||
// Tool support detected via the local servers' healthz handshake —
|
||||
// treated as equivalent to declared caps when filtering candidates.
|
||||
detected_tools: &std::collections::HashSet<String>,
|
||||
) -> Result<(RouteDecision, String, Vec<String>)> {
|
||||
// 1. Extract text for classification
|
||||
let text = self.extract_text(
|
||||
@@ -231,6 +234,19 @@ impl Pipeline {
|
||||
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
|
||||
match live.router.route(&text, prof).await {
|
||||
Ok(Some(dec)) => {
|
||||
let mut dec = dec;
|
||||
// per-gateway complexity bands: re-map the
|
||||
// router's continuous score with the user's
|
||||
// own boundaries (sliders on the Routing page)
|
||||
if let Some(b) = &live.cfg.routing.bands {
|
||||
if matches!(
|
||||
dec.complexity_tier.as_str(),
|
||||
"low" | "medium" | "high"
|
||||
) {
|
||||
dec.complexity_tier =
|
||||
b.tier_of(dec.complexity_score).to_string();
|
||||
}
|
||||
}
|
||||
let src = dec.source.clone();
|
||||
let (cands, task_rule) = live
|
||||
.routing
|
||||
@@ -261,10 +277,13 @@ impl Pipeline {
|
||||
|
||||
// Filter candidates by capabilities (e.g. tools support)
|
||||
let needs_tools = !req.tools.is_empty();
|
||||
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
|
||||
prov.caps().tools || detected_tools.contains(id)
|
||||
};
|
||||
let mut final_candidates = Vec::new();
|
||||
for model_id in candidates {
|
||||
if let Some(prov) = live.registry.get(&model_id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&model_id, &prov) {
|
||||
tracing::debug!(
|
||||
"Candidate '{}' skipped: lacks support for requested tools",
|
||||
model_id
|
||||
@@ -293,13 +312,49 @@ impl Pipeline {
|
||||
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
|
||||
for id in live.local_candidates() {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Capability widening: a tier chain built of tools-less local models
|
||||
// must not kill an agent request. When the filtered chain is empty on
|
||||
// an AUTO route, widen to any pool model that satisfies the required
|
||||
// capabilities — preferring the routing table's own medium→high order
|
||||
// (an agentic request is worth a capable model), then the rest of the
|
||||
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
|
||||
// this every low-tier verdict was a hard failure.
|
||||
if final_candidates.is_empty() {
|
||||
let mut widened: Vec<String> = Vec::new();
|
||||
for tier in ["medium", "high"] {
|
||||
for id in live.routing.candidates(tier) {
|
||||
if !widened.contains(&id) {
|
||||
widened.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in live.registry.ids() {
|
||||
if !widened.contains(id) {
|
||||
widened.push(id.clone());
|
||||
}
|
||||
}
|
||||
for id in widened {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
if !final_candidates.is_empty() {
|
||||
tracing::info!(
|
||||
chosen = %final_candidates[0],
|
||||
"capability widening: tier chain had no model with the required capabilities"
|
||||
);
|
||||
}
|
||||
}
|
||||
if final_candidates.is_empty() {
|
||||
return Err(GatewayError::UpstreamUnavailable(
|
||||
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
||||
@@ -311,7 +366,9 @@ impl Pipeline {
|
||||
|
||||
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
||||
let live = state.live();
|
||||
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, mut final_candidates) =
|
||||
self.resolve(&req, &live, &detected_tools).await?;
|
||||
|
||||
// Serving-when-promoted (gated): a promoted task-label is served by the
|
||||
// local model FIRST (failover to cloud on error). Per-request VETO uses
|
||||
@@ -458,7 +515,9 @@ impl Pipeline {
|
||||
.to_string();
|
||||
|
||||
let live = state.live();
|
||||
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, final_candidates) =
|
||||
match self.resolve(&req, &live, &detected_tools).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
|
||||
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<MessagesRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.anthropic_messages {
|
||||
@@ -108,6 +109,7 @@ async fn handler(
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "anthropic_messages".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
+51
-1
@@ -12,10 +12,59 @@ pub mod openai_models;
|
||||
use crate::state::SharedState;
|
||||
use axum::Router;
|
||||
|
||||
/// Account resolved by the API-key middleware, consumed by protocol handlers
|
||||
/// for attribution (stats, budgets). Present only when keys are configured.
|
||||
#[derive(Clone)]
|
||||
pub struct AccountTag(pub String);
|
||||
|
||||
/// API-key gate for the model-facing surface. Open mode (no keys configured)
|
||||
/// passes everything through — the admin console already warns about it.
|
||||
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
|
||||
/// enforced with a fixed one-minute window (429 + Retry-After), and a
|
||||
/// monthly budget, when set, turns into 429 insufficient_quota once the
|
||||
/// account's month-to-date spend crosses it.
|
||||
async fn api_auth(
|
||||
axum::extract::State(state): axum::extract::State<SharedState>,
|
||||
mut req: axum::extract::Request,
|
||||
next: axum::middleware::Next,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
let live = state.live();
|
||||
if live.cfg.api_keys.is_empty() {
|
||||
return next.run(req).await;
|
||||
}
|
||||
let provided = req
|
||||
.headers()
|
||||
.get(axum::http::header::AUTHORIZATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
|
||||
.map(str::trim)
|
||||
.unwrap_or("");
|
||||
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
|
||||
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
|
||||
.into_response();
|
||||
};
|
||||
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
|
||||
return crate::error::GatewayError::RateLimited {
|
||||
retry_after_secs: state.rate.retry_after_secs(),
|
||||
}
|
||||
.into_response();
|
||||
}
|
||||
if let Some(budget) = key.monthly_budget_usd {
|
||||
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
|
||||
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
|
||||
.into_response();
|
||||
}
|
||||
}
|
||||
req.extensions_mut().insert(AccountTag(key.account.clone()));
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
/// Build the protocols router. Implemented adapters are **always** mounted;
|
||||
/// each handler checks the live `protocols.*` flag from config itself, so
|
||||
/// toggles in the admin panel take effect without a restart (hot switching).
|
||||
pub fn build_router() -> Router<SharedState> {
|
||||
pub fn build_router(state: SharedState) -> Router<SharedState> {
|
||||
Router::new()
|
||||
.merge(openai_chat::routes())
|
||||
.merge(openai_completions::routes())
|
||||
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
|
||||
.merge(anthropic_messages::routes())
|
||||
.merge(mcp::routes())
|
||||
.merge(native_passthrough::routes())
|
||||
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
|
||||
}
|
||||
|
||||
@@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<OpenAiChatRequest>,
|
||||
) -> Result<Response> {
|
||||
// hot protocol toggle: if the adapter is disabled in config — return 404
|
||||
@@ -104,6 +105,7 @@ async fn handler(
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_chat".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<CompletionsRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.openai_completions {
|
||||
@@ -89,6 +90,7 @@ async fn handler(
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_completions".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
+1
-1
@@ -48,7 +48,7 @@ impl Registry {
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
caps: s.caps.clone(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
|
||||
@@ -66,6 +66,42 @@ impl Live {
|
||||
}
|
||||
}
|
||||
|
||||
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
|
||||
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
|
||||
/// fine for office-grade fairness and costs one HashMap lookup per request.
|
||||
#[derive(Default)]
|
||||
pub struct RateWindows {
|
||||
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
|
||||
}
|
||||
|
||||
impl RateWindows {
|
||||
pub fn allow(&self, key: &str, per_min: u32) -> bool {
|
||||
let minute = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
/ 60;
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
let e = g.entry(key.to_string()).or_insert((minute, 0));
|
||||
if e.0 != minute {
|
||||
*e = (minute, 0);
|
||||
}
|
||||
if e.1 >= per_min {
|
||||
return false;
|
||||
}
|
||||
e.1 += 1;
|
||||
true
|
||||
}
|
||||
/// Seconds until the current minute window rolls over.
|
||||
pub fn retry_after_secs(&self) -> u64 {
|
||||
60 - (std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
% 60)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct AppState {
|
||||
pub live: ArcSwap<Live>,
|
||||
pub stats: Arc<Stats>,
|
||||
@@ -85,6 +121,8 @@ pub struct AppState {
|
||||
/// Managed local CMF model server (install/update/spawn lifecycle + status).
|
||||
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
|
||||
pub cache: Arc<SemanticCache>,
|
||||
/// Per-API-key fixed-window rate limiter.
|
||||
pub rate: RateWindows,
|
||||
pub secrets: SecretStore,
|
||||
pub config_path: String,
|
||||
pub pipeline: crate::pipeline::Pipeline,
|
||||
@@ -164,6 +202,7 @@ impl SharedState {
|
||||
admin_open_public: std::sync::atomic::AtomicBool::new(false),
|
||||
cmf,
|
||||
cache,
|
||||
rate: RateWindows::default(),
|
||||
secrets,
|
||||
config_path,
|
||||
pipeline: crate::pipeline::Pipeline::new(),
|
||||
|
||||
+112
-9
@@ -137,6 +137,11 @@ struct Inner {
|
||||
/// = folded + by_account_alltime, with no double counting: a record is
|
||||
/// either in the file or folded, never both.
|
||||
folded: HashMap<String, Totals>,
|
||||
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
/// Records with ts ≤ this are already reflected in the persisted
|
||||
/// month_spend (watermark against double counting on replay).
|
||||
month_upto_ts: u64,
|
||||
/// Appends since the last compaction size check.
|
||||
writes_since_check: u32,
|
||||
}
|
||||
@@ -193,6 +198,7 @@ impl Stats {
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
if rec.ts >= cutoff {
|
||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||
}
|
||||
@@ -200,6 +206,33 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
|
||||
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
|
||||
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
|
||||
return;
|
||||
}
|
||||
let m = month_of(rec.ts);
|
||||
let e = inner
|
||||
.month_spend
|
||||
.entry(rec.account.clone())
|
||||
.or_insert((m.clone(), 0.0));
|
||||
if e.0 != m {
|
||||
*e = (m, 0.0);
|
||||
}
|
||||
e.1 += rec.cost_usd;
|
||||
}
|
||||
|
||||
/// Month-to-date spend for an account (budget checks).
|
||||
pub fn month_spend(&self, account: &str) -> f64 {
|
||||
let inner = self.inner.lock().unwrap();
|
||||
let m = month_of(now_secs());
|
||||
inner
|
||||
.month_spend
|
||||
.get(account)
|
||||
.filter(|(mm, _)| *mm == m)
|
||||
.map(|(_, v)| *v)
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
|
||||
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
|
||||
inner.recent.push_back(rec.clone());
|
||||
while inner.recent.len() > ring_size {
|
||||
@@ -256,6 +289,7 @@ impl Stats {
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
inner.writes_since_check += 1;
|
||||
}
|
||||
if let Some(path) = &self.file {
|
||||
@@ -283,11 +317,42 @@ impl Stats {
|
||||
let Ok(content) = std::fs::read_to_string(&path) else {
|
||||
return;
|
||||
};
|
||||
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||
#[derive(serde::Deserialize)]
|
||||
struct Sidecar {
|
||||
#[serde(default)]
|
||||
by_account: HashMap<String, Totals>,
|
||||
#[serde(default)]
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
#[serde(default)]
|
||||
month_upto_ts: u64,
|
||||
}
|
||||
// new format {by_account, month_spend, month_upto_ts}; the original
|
||||
// file held the bare by_account map — accept both
|
||||
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
inner.folded = sc.by_account;
|
||||
inner.month_spend = sc.month_spend;
|
||||
inner.month_upto_ts = sc.month_upto_ts;
|
||||
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||
self.inner.lock().unwrap().folded = m;
|
||||
}
|
||||
}
|
||||
|
||||
fn write_sidecar(&self, inner: &Inner) {
|
||||
let Some(tpath) = self.totals_path() else {
|
||||
return;
|
||||
};
|
||||
let payload = serde_json::json!({
|
||||
"by_account": inner.folded,
|
||||
"month_spend": inner.month_spend,
|
||||
"month_upto_ts": now_secs(),
|
||||
});
|
||||
let tmp = format!("{tpath}.tmp");
|
||||
if std::fs::write(&tmp, payload.to_string()).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &tpath);
|
||||
}
|
||||
}
|
||||
|
||||
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
|
||||
/// older than the retention window fold into the cumulative per-account
|
||||
/// sidecar and are dropped from the file (tmp + rename on both files, so
|
||||
@@ -305,6 +370,10 @@ impl Stats {
|
||||
return;
|
||||
}
|
||||
inner.writes_since_check = 0;
|
||||
// piggyback: persist month-to-date spend so budgets survive
|
||||
// restarts even between compactions
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
}
|
||||
}
|
||||
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
|
||||
@@ -359,14 +428,8 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
// persist the folded totals FIRST, then swap the trimmed file in
|
||||
if let Some(tpath) = self.totals_path() {
|
||||
if let Ok(json) = serde_json::to_string(&inner.folded) {
|
||||
let tmp = format!("{tpath}.tmp");
|
||||
if std::fs::write(&tmp, json).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &tpath);
|
||||
}
|
||||
}
|
||||
}
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
let tmp = format!("{path}.tmp");
|
||||
if std::fs::write(&tmp, kept).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, path);
|
||||
@@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
|
||||
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
|
||||
}
|
||||
|
||||
fn month_of(ts: u64) -> String {
|
||||
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
|
||||
let days = ts / 86_400;
|
||||
let mut year = 1970u64;
|
||||
let mut d = days;
|
||||
loop {
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let len = if leap { 366 } else { 365 };
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
year += 1;
|
||||
}
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let ml = [
|
||||
31,
|
||||
if leap { 29 } else { 28 },
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
];
|
||||
let mut month = 1;
|
||||
for len in ml {
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
month += 1;
|
||||
}
|
||||
format!("{year:04}-{month:02}")
|
||||
}
|
||||
|
||||
fn avg(sum: u64, n: u64) -> u64 {
|
||||
sum.checked_div(n).unwrap_or(0)
|
||||
}
|
||||
|
||||
Vendored
+20
@@ -597,6 +597,26 @@ body.modal-open { overflow: hidden; }
|
||||
.pair-model { min-width: 220px; max-width: 340px; padding: 4px 8px; font-size: 12.5px; }
|
||||
.pair-tier { max-width: 150px; padding: 4px 8px; font-size: 12.5px; }
|
||||
|
||||
/* Complexity bands slider */
|
||||
.bands-wrap { margin: 4px 0 16px; }
|
||||
/* one-line command chip (install commands etc.) — theme-aware */
|
||||
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
|
||||
|
||||
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
|
||||
.bands-track.disabled { opacity: .45; pointer-events: none; }
|
||||
.bands-zone { position: absolute; top: 0; bottom: 0; }
|
||||
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
|
||||
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
|
||||
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
|
||||
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
|
||||
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
|
||||
.bands-handle:active { cursor: grabbing; }
|
||||
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
|
||||
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
|
||||
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
|
||||
.bands-val.raised { top: -46px; }
|
||||
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
|
||||
|
||||
/* Benchmark view */
|
||||
@keyframes bm-in { from { opacity: 0; transform: translateY(-3px); } to { opacity: 1; transform: none; } }
|
||||
tbody tr:last-child { animation: bm-in .25s ease; }
|
||||
|
||||
Vendored
+1
@@ -74,6 +74,7 @@ export const api = {
|
||||
// protocols
|
||||
getProtocols: () => j("GET", "/protocols"),
|
||||
harnesses: () => j("GET", "/harnesses"),
|
||||
balance: () => j("GET", "/balance"),
|
||||
putProtocols: (b) => j("PUT", "/protocols", b),
|
||||
|
||||
// settings
|
||||
|
||||
Vendored
+5
@@ -48,6 +48,11 @@ export function onLangChange(fn) {
|
||||
return () => listeners.delete(fn);
|
||||
}
|
||||
|
||||
// Human name for a complexity tier; custom tier names pass through as-is.
|
||||
export function trTier(tier) {
|
||||
return ["low", "medium", "high"].includes(tier) ? t("tier." + tier) : tier;
|
||||
}
|
||||
|
||||
export function t(key, vars) {
|
||||
const d = DICTS[lang] || en;
|
||||
let s = d[key] ?? en[key] ?? key;
|
||||
|
||||
Vendored
+17
@@ -629,4 +629,21 @@ export default {
|
||||
"routing.tasks.tierHint": "Welches Modell diese Aufgabe bei welcher Komplexität bedient; \"jede\" führt auf allen Stufen. Reihenfolge = Fallback-Reihenfolge. High eskaliert weiterhin über \"jede\"-Einträge.",
|
||||
"routing.tasks.tierAny": "jede Komplexität",
|
||||
"routing.tasks.addModel": "Modell",
|
||||
"routing.bands.enable": "Eigene Komplexitätsgrenzen",
|
||||
"routing.bands.hint": "Ziehen Sie die Griffe: wo low endet und high beginnt — die Griffe schieben einander, die Zahlen aktualisieren sich live. Der kontinuierliche Router-Score wird mit IHREN Grenzen neu eingeteilt; ohne Häkchen gelten die Grenzen des Routers.",
|
||||
"tier.low": "leicht",
|
||||
"tier.medium": "mittel",
|
||||
"tier.high": "schwer",
|
||||
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
|
||||
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
|
||||
"settings.cache.entries": "Einträge",
|
||||
"settings.cache.hits": "Treffer",
|
||||
"settings.cache.saved": "gespart",
|
||||
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
|
||||
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
|
||||
"keys.budgetOf": "Budget {b}/Mon.",
|
||||
"dash.kpi.orBalance": "OpenRouter-Guthaben",
|
||||
"keys.h.install": "Installieren:",
|
||||
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
|
||||
"keys.snip.apply": "Schlüssel einsetzen",
|
||||
};
|
||||
|
||||
Vendored
+17
@@ -622,4 +622,21 @@ export default {
|
||||
"routing.tasks.tierHint": "Which model serves this task at which complexity; \"any\" leads at every tier. Order = fallback order. High verdicts still escalate over any-tier pins.",
|
||||
"routing.tasks.tierAny": "any complexity",
|
||||
"routing.tasks.addModel": "model",
|
||||
"routing.bands.enable": "My complexity boundaries",
|
||||
"routing.bands.hint": "Drag the handles to set where low ends and where high begins — the handles push each other, the numbers update live. The router's continuous score is re-mapped with YOUR boundaries; unchecked = the router's own bands.",
|
||||
"tier.low": "low",
|
||||
"tier.medium": "medium",
|
||||
"tier.high": "high",
|
||||
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
|
||||
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
|
||||
"settings.cache.entries": "entries",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "saved",
|
||||
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
|
||||
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
|
||||
"keys.budgetOf": "budget {b}/mo",
|
||||
"dash.kpi.orBalance": "OpenRouter balance",
|
||||
"keys.h.install": "Install:",
|
||||
"keys.snip.keyInputPh": "paste your API key",
|
||||
"keys.snip.apply": "Insert key",
|
||||
};
|
||||
|
||||
Vendored
+17
@@ -629,4 +629,21 @@ export default {
|
||||
"routing.tasks.tierHint": "Qué modelo atiende esta tarea en qué complejidad; «cualquiera» lidera en todos los niveles. El orden = orden de respaldo. Un veredicto high sigue escalando sobre las entradas «cualquiera».",
|
||||
"routing.tasks.tierAny": "cualquier complejidad",
|
||||
"routing.tasks.addModel": "modelo",
|
||||
"routing.bands.enable": "Mis límites de complejidad",
|
||||
"routing.bands.hint": "Arrastra los tiradores: dónde termina low y dónde empieza high — se empujan entre sí y las cifras se actualizan en vivo. La puntuación continua del router se re-mapea con TUS límites; sin marcar = las bandas del propio router.",
|
||||
"tier.low": "fácil",
|
||||
"tier.medium": "medio",
|
||||
"tier.high": "difícil",
|
||||
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
|
||||
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
|
||||
"settings.cache.entries": "entradas",
|
||||
"settings.cache.hits": "aciertos",
|
||||
"settings.cache.saved": "ahorrado",
|
||||
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
|
||||
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
|
||||
"keys.budgetOf": "presupuesto {b}/mes",
|
||||
"dash.kpi.orBalance": "Saldo OpenRouter",
|
||||
"keys.h.install": "Instalar:",
|
||||
"keys.snip.keyInputPh": "pega tu clave API",
|
||||
"keys.snip.apply": "Insertar clave",
|
||||
};
|
||||
|
||||
Vendored
+17
@@ -629,4 +629,21 @@ export default {
|
||||
"routing.tasks.tierHint": "Quel modèle sert cette tâche à quelle complexité ; « toute » mène à tous les niveaux. L'ordre = ordre de repli. Un verdict high escalade toujours au-dessus des entrées « toute ».",
|
||||
"routing.tasks.tierAny": "toute complexité",
|
||||
"routing.tasks.addModel": "modèle",
|
||||
"routing.bands.enable": "Mes limites de complexité",
|
||||
"routing.bands.hint": "Faites glisser les poignées : où finit low et où commence high — elles se poussent l'une l'autre, les chiffres se mettent à jour en direct. Le score continu du routeur est re-découpé avec VOS limites ; décoché = les bandes du routeur.",
|
||||
"tier.low": "facile",
|
||||
"tier.medium": "moyen",
|
||||
"tier.high": "difficile",
|
||||
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
|
||||
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
|
||||
"settings.cache.entries": "entrées",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "économisé",
|
||||
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
|
||||
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
|
||||
"keys.budgetOf": "budget {b}/mois",
|
||||
"dash.kpi.orBalance": "Solde OpenRouter",
|
||||
"keys.h.install": "Installer :",
|
||||
"keys.snip.keyInputPh": "collez votre clé API",
|
||||
"keys.snip.apply": "Insérer la clé",
|
||||
};
|
||||
|
||||
Vendored
+19
-2
@@ -25,7 +25,7 @@ export default {
|
||||
"common.all": "все",
|
||||
|
||||
// nav
|
||||
"nav.dashboard": "Дашборд",
|
||||
"nav.dashboard": "Рабочий стол",
|
||||
"nav.models": "Модели",
|
||||
"nav.import": "Импорт модели",
|
||||
"nav.routing": "Роутинг",
|
||||
@@ -56,7 +56,7 @@ export default {
|
||||
"login.failed": "Неверный токен администратора.",
|
||||
|
||||
// dashboard
|
||||
"dash.title": "Дашборд",
|
||||
"dash.title": "Рабочий стол",
|
||||
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
|
||||
"dash.clear": "Очистить логи",
|
||||
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
|
||||
@@ -621,4 +621,21 @@ export default {
|
||||
"routing.tasks.tierHint": "Какая модель обслуживает задачу на какой сложности; «любая» ведёт на всех уровнях. Порядок = порядок фолбэка. Вердикт high по-прежнему эскалирует поверх «любых».",
|
||||
"routing.tasks.tierAny": "любая сложность",
|
||||
"routing.tasks.addModel": "модель",
|
||||
"routing.bands.enable": "Мои границы сложности",
|
||||
"routing.bands.hint": "Тяните ручки: где кончается low и где начинается high — ручки толкают друг друга, цифры обновляются на лету. Непрерывный score роутера пересчитывается ВАШИМИ границами; галочка снята — действуют пороги самого роутера.",
|
||||
"tier.low": "лёгкий",
|
||||
"tier.medium": "средний",
|
||||
"tier.high": "сложный",
|
||||
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
|
||||
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
|
||||
"settings.cache.entries": "записей",
|
||||
"settings.cache.hits": "попаданий",
|
||||
"settings.cache.saved": "сэкономлено",
|
||||
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
|
||||
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
|
||||
"keys.budgetOf": "бюджет {b}/мес",
|
||||
"dash.kpi.orBalance": "Баланс OpenRouter",
|
||||
"keys.h.install": "Установить:",
|
||||
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
|
||||
"keys.snip.apply": "Вставить ключ",
|
||||
};
|
||||
|
||||
Vendored
+17
@@ -629,4 +629,21 @@ export default {
|
||||
"routing.tasks.tierHint": "Bu görevi hangi karmaşıklıkta hangi model karşılar; \"her\" tüm katmanlarda önde gider. Sıra = yedekleme sırası. High kararı \"her\" girdilerinin üzerine yine tırmanır.",
|
||||
"routing.tasks.tierAny": "her karmaşıklık",
|
||||
"routing.tasks.addModel": "model",
|
||||
"routing.bands.enable": "Kendi karmaşıklık sınırlarım",
|
||||
"routing.bands.hint": "Tutamaçları sürükleyin: low nerede biter, high nerede başlar — tutamaçlar birbirini iter, sayılar anlık güncellenir. Yönlendiricinin sürekli puanı SİZİN sınırlarınızla yeniden bölümlenir; işaret kaldırılırsa yönlendiricinin kendi bantları geçerli.",
|
||||
"tier.low": "kolay",
|
||||
"tier.medium": "orta",
|
||||
"tier.high": "zor",
|
||||
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
|
||||
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
|
||||
"settings.cache.entries": "kayıt",
|
||||
"settings.cache.hits": "isabet",
|
||||
"settings.cache.saved": "tasarruf",
|
||||
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
|
||||
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
|
||||
"keys.budgetOf": "bütçe {b}/ay",
|
||||
"dash.kpi.orBalance": "OpenRouter bakiyesi",
|
||||
"keys.h.install": "Kurulum:",
|
||||
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
|
||||
"keys.snip.apply": "Anahtarı ekle",
|
||||
};
|
||||
|
||||
Vendored
+17
@@ -629,4 +629,21 @@ export default {
|
||||
"routing.tasks.tierHint": "哪个模型在哪个复杂度上处理该任务;「任意」在所有层级领先。顺序即回退顺序。high 判定仍会越过「任意」条目升级。",
|
||||
"routing.tasks.tierAny": "任意复杂度",
|
||||
"routing.tasks.addModel": "模型",
|
||||
"routing.bands.enable": "自定义复杂度边界",
|
||||
"routing.bands.hint": "拖动手柄设定 low 到哪里结束、high 从哪里开始 — 手柄会相互推动,数字实时更新。路由器的连续得分将按你的边界重新划分;取消勾选则使用路由器自身的边界。",
|
||||
"tier.low": "简单",
|
||||
"tier.medium": "中等",
|
||||
"tier.high": "困难",
|
||||
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
|
||||
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
|
||||
"settings.cache.entries": "条目",
|
||||
"settings.cache.hits": "命中",
|
||||
"settings.cache.saved": "已节省",
|
||||
"keys.form.budget": "每月预算,$(0 = 不限)",
|
||||
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
|
||||
"keys.budgetOf": "预算 {b}/月",
|
||||
"dash.kpi.orBalance": "OpenRouter 余额",
|
||||
"keys.h.install": "安装:",
|
||||
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
|
||||
"keys.snip.apply": "插入密钥",
|
||||
};
|
||||
|
||||
Vendored
+34
-19
@@ -8,7 +8,7 @@
|
||||
// (keyword checks with | synonyms, think-tag stripping, anti-loop detection,
|
||||
// pass/partial/fail), now embedded, localized and CDN-free.
|
||||
import { h, mount, toast, ms, money, moneyEl } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api, testStream } from "../api.js";
|
||||
|
||||
// ── built-in suite ──────────────────────────────────────────────────────────
|
||||
@@ -216,24 +216,35 @@ export async function renderBenchmark() {
|
||||
let ev;
|
||||
if (test.judge === "route") {
|
||||
// the test asserts the ROUTER's decision: did it rate this high?
|
||||
// One retry with a FRESH template: the router deliberately scores
|
||||
// familiar text lower (novelty), so a single burnt wording must
|
||||
// not fail the whole assertion.
|
||||
// Two honest retries. (1) The SAME prompt after a pause: the
|
||||
// router judges difficulty in the background — the first-ever
|
||||
// occurrence routes by prior, the judgement lands in the cache a
|
||||
// couple of seconds later, so the second occurrence is the real
|
||||
// steady-state answer. (2) A fresh template, in case this one's
|
||||
// wording clustered into a burnt region.
|
||||
if (info.tier !== "high" && test.gen) {
|
||||
const p2 = hardPrompt(test.gen);
|
||||
try {
|
||||
const second = await testStream(
|
||||
{ model, messages: [{ role: "user", content: p2 }], temperature: 0, max_tokens: (test.max || 128) + 256 },
|
||||
() => {}
|
||||
);
|
||||
if (second.tier === "high") {
|
||||
info.tier = second.tier;
|
||||
info.selected_model = second.selected_model;
|
||||
info.task_label = second.task_label;
|
||||
info.route_source = second.route_source;
|
||||
info.cost_usd = (info.cost_usd || 0) + (second.cost_usd || 0);
|
||||
}
|
||||
} catch { /* keep first attempt */ }
|
||||
const attempts = [
|
||||
{ prompt, delayMs: 4000 },
|
||||
{ prompt: hardPrompt(test.gen), delayMs: 0 },
|
||||
];
|
||||
for (const a of attempts) {
|
||||
if (info.tier === "high") break;
|
||||
if (a.delayMs) await new Promise((r) => setTimeout(r, a.delayMs));
|
||||
try {
|
||||
const again = await testStream(
|
||||
{ model, messages: [{ role: "user", content: a.prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
|
||||
() => {}
|
||||
);
|
||||
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
|
||||
if (again.tier === "high") {
|
||||
info.tier = again.tier;
|
||||
info.score = again.score;
|
||||
info.selected_model = again.selected_model;
|
||||
info.task_label = again.task_label;
|
||||
info.route_source = again.route_source;
|
||||
}
|
||||
} catch { /* keep previous attempt */ }
|
||||
}
|
||||
}
|
||||
ev = info.tier === "high"
|
||||
? { status: "pass", score: 1, note: "high ✓" }
|
||||
@@ -259,6 +270,8 @@ export async function renderBenchmark() {
|
||||
|| String(info.route_source || "").includes("degraded");
|
||||
row.task = info.task_label || "—";
|
||||
row.tier = info.tier || "—";
|
||||
row.cscore = info.score || 0; // непрерывный коэффициент сложности
|
||||
// (row.score занят долей правильности ответа из evaluate())
|
||||
row.chosen = info.selected_model || "—";
|
||||
row.source = info.route_source || "—";
|
||||
row.expected = test.label; // null = classification not judged
|
||||
@@ -392,7 +405,9 @@ export async function renderBenchmark() {
|
||||
: h("span", { class: r.taskOk ? "" : "err" },
|
||||
(r.taskOk ? "✓ " : "✗ ") + r.task,
|
||||
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
|
||||
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
|
||||
h("td", {},
|
||||
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
|
||||
h("td", {},
|
||||
h("span", { class: "mono small" }, r.chosen || "—"),
|
||||
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
|
||||
|
||||
Vendored
+12
-4
@@ -1,6 +1,6 @@
|
||||
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
|
||||
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { lineChart, barList } from "../charts.js";
|
||||
import { SITE_URL } from "../app.js";
|
||||
@@ -110,12 +110,13 @@ export async function renderDashboard() {
|
||||
|
||||
async function load() {
|
||||
const rangeSecs = range;
|
||||
let stats, recent, health;
|
||||
let stats, recent, health, balance;
|
||||
try {
|
||||
[stats, recent, health] = await Promise.all([
|
||||
[stats, recent, health, balance] = await Promise.all([
|
||||
api.stats(rangeSecs, groupby),
|
||||
api.requests(40, 0),
|
||||
api.health().catch(() => null),
|
||||
api.balance().catch(() => null),
|
||||
]);
|
||||
} catch (e) {
|
||||
mount(root, h("div", { class: "callout" }, String(e.message || e)));
|
||||
@@ -162,6 +163,11 @@ export async function renderDashboard() {
|
||||
kpi(num(tot.requests), t("dash.kpi.requests")),
|
||||
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
|
||||
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
|
||||
balance && balance.openrouter_usd != null
|
||||
? kpi(
|
||||
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
|
||||
t("dash.kpi.orBalance"))
|
||||
: null,
|
||||
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
|
||||
kpi(pct(tot.success_rate), t("dash.kpi.success")),
|
||||
kpi(num(tot.failovers), t("dash.kpi.failovers"))
|
||||
@@ -271,7 +277,9 @@ function recentTable(rows) {
|
||||
{},
|
||||
h("td", { class: "mono" }, timeAgo(r.ts)),
|
||||
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
|
||||
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
|
||||
h("td", {},
|
||||
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
|
||||
h("td", {}, r.task_label || "—"),
|
||||
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
|
||||
h("td", { class: "mono" }, money(r.cost_usd)),
|
||||
|
||||
Vendored
+147
-14
@@ -10,12 +10,14 @@ export async function renderKeys() {
|
||||
function openCreate(reload) {
|
||||
const accountIn = h("input", { placeholder: "team-a" });
|
||||
const rateIn = h("input", { type: "number", value: 600 });
|
||||
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
|
||||
const customIn = h("input", { placeholder: "sk-gw-…" });
|
||||
const body = h(
|
||||
"div",
|
||||
{},
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
|
||||
);
|
||||
modal(t("keys.form.title"), body, async () => {
|
||||
@@ -24,6 +26,7 @@ export async function renderKeys() {
|
||||
key: customIn.value.trim(),
|
||||
account: accountIn.value.trim() || "default",
|
||||
rate_per_min: parseInt(rateIn.value) || 0,
|
||||
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
|
||||
allow_models: [],
|
||||
});
|
||||
reload();
|
||||
@@ -51,18 +54,76 @@ export async function renderKeys() {
|
||||
function snippetsCard(openMode) {
|
||||
const origin = location.origin;
|
||||
const base = origin + "/v1";
|
||||
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
// Every scripty snippet exists in two OS flavors: bash heredocs for
|
||||
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
|
||||
// (python/js) and GUI instructions are OS-independent.
|
||||
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
|
||||
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
|
||||
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
|
||||
const snips = {
|
||||
"Claude Code": `# ${t("keys.h.claude")}
|
||||
export ANTHROPIC_BASE_URL="${origin}"
|
||||
const buildSnips = (osKind) => osKind === "ps" ? {
|
||||
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
|
||||
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
|
||||
$env:ANTHROPIC_MODEL = "cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
|
||||
Add-Content "$HOME\\.codex\\config.toml" @'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
[model_providers.cortiq]
|
||||
name = "Cortiq Gateway"
|
||||
base_url = "${base}"
|
||||
env_key = "CORTIQ_API_KEY"
|
||||
wire_api = "chat"
|
||||
'@
|
||||
$env:CORTIQ_API_KEY = "${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
|
||||
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"cortiq": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Cortiq Gateway",
|
||||
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
opencode`,
|
||||
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
|
||||
Add-Content "$HOME\\.continue\\config.yaml" @'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
model: cortiq-auto
|
||||
apiBase: ${base}
|
||||
apiKey: ${key}
|
||||
'@`,
|
||||
Cline: `${t("keys.h.cline1")}
|
||||
API Provider: OpenAI Compatible
|
||||
Base URL: ${base}
|
||||
API Key: ${key}
|
||||
Model ID: cortiq-auto
|
||||
|
||||
${t("keys.h.cline2")}`,
|
||||
curl: `curl.exe ${base}/chat/completions \`
|
||||
-H "Authorization: Bearer ${key}" \`
|
||||
-H "Content-Type: application/json" \`
|
||||
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||
} : {
|
||||
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
|
||||
export ANTHROPIC_AUTH_TOKEN="${key}"
|
||||
export ANTHROPIC_MODEL="cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `# ${t("keys.h.codex")}
|
||||
mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
@@ -75,8 +136,7 @@ EOF
|
||||
export CORTIQ_API_KEY="${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `# ${t("keys.h.opencode")}
|
||||
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
@@ -91,8 +151,7 @@ mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
EOF
|
||||
|
||||
opencode`,
|
||||
Continue: `# ${t("keys.h.continue")}
|
||||
mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
@@ -128,8 +187,29 @@ const resp = await client.chat.completions.create({
|
||||
});
|
||||
console.log(resp.choices[0].message.content);`,
|
||||
};
|
||||
let shSnips, psSnips, snips;
|
||||
function rebuildSnips() {
|
||||
shSnips = buildSnips("sh");
|
||||
psSnips = buildSnips("ps");
|
||||
// python/js/Cline одинаковы — дополняем ps-набор из sh
|
||||
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
|
||||
snips = os === "ps" ? psSnips : shSnips;
|
||||
}
|
||||
rebuildSnips();
|
||||
let cur = "Claude Code";
|
||||
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
|
||||
// in a pasted snippet is not a comment in default interactive zsh, and a
|
||||
// "!" inside it triggers history expansion ("event not found")
|
||||
const HINTS = {
|
||||
"Claude Code": () => t("keys.h.claude"),
|
||||
Codex: () => t("keys.h.codex"),
|
||||
OpenCode: () => t("keys.h.opencode"),
|
||||
Continue: () => t("keys.h.continue"),
|
||||
};
|
||||
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
|
||||
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
|
||||
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
|
||||
paintHint();
|
||||
const seg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
@@ -137,6 +217,7 @@ console.log(resp.choices[0].message.content);`,
|
||||
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
|
||||
cur = k;
|
||||
pre.textContent = snips[k];
|
||||
paintHint();
|
||||
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
|
||||
paintStatus();
|
||||
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
|
||||
@@ -164,7 +245,15 @@ console.log(resp.choices[0].message.content);`,
|
||||
}).catch(() => {});
|
||||
|
||||
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
|
||||
// status line under the snippet ("installed" / "already configured").
|
||||
// status line under the snippet ("installed" / "already configured") —
|
||||
// and a one-line INSTALL command (per OS) when it's missing.
|
||||
const INSTALL = {
|
||||
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
|
||||
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
|
||||
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
|
||||
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
|
||||
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
|
||||
};
|
||||
let detected = {};
|
||||
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
|
||||
function paintStatus() {
|
||||
@@ -176,7 +265,19 @@ console.log(resp.choices[0].message.content);`,
|
||||
statusHost.innerHTML = "";
|
||||
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
|
||||
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
|
||||
else mount(statusHost, h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")));
|
||||
else {
|
||||
const cmd = (INSTALL[cur] || {})[os];
|
||||
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
|
||||
await navigator.clipboard.writeText(cmd);
|
||||
toast(t("common.copied"), "good");
|
||||
} }, "⧉");
|
||||
mount(statusHost,
|
||||
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
|
||||
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
|
||||
h("span", { class: "muted" }, t("keys.h.install")),
|
||||
h("code", { class: "mono small cmd-chip" }, cmd),
|
||||
copyInstall) : null);
|
||||
}
|
||||
}
|
||||
api.harnesses().then((d) => {
|
||||
detected = d || {};
|
||||
@@ -187,6 +288,32 @@ console.log(resp.choices[0].message.content);`,
|
||||
paintStatus();
|
||||
}).catch(() => {});
|
||||
|
||||
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
|
||||
const osSeg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
|
||||
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
|
||||
);
|
||||
osSeg.querySelectorAll("button").forEach((b, i) =>
|
||||
b.addEventListener("click", () => {
|
||||
os = i === 0 ? "sh" : "ps";
|
||||
rebuildSnips();
|
||||
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
|
||||
pre.textContent = snips[cur];
|
||||
paintStatus();
|
||||
}));
|
||||
|
||||
// paste-your-key: substitutes the key into every snippet immediately
|
||||
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
|
||||
const applyKey = () => {
|
||||
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
rebuildSnips();
|
||||
pre.textContent = snips[cur];
|
||||
};
|
||||
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
|
||||
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
|
||||
|
||||
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||
copy.addEventListener("click", async () => {
|
||||
await navigator.clipboard.writeText(pre.textContent);
|
||||
@@ -198,7 +325,8 @@ console.log(resp.choices[0].message.content);`,
|
||||
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" },
|
||||
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), copy),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
|
||||
hintHost,
|
||||
pre,
|
||||
statusHost,
|
||||
protoWarn,
|
||||
@@ -273,7 +401,12 @@ console.log(resp.choices[0].message.content);`,
|
||||
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
|
||||
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
|
||||
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
|
||||
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
|
||||
h("td", { class: "right-align" },
|
||||
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
|
||||
k.monthly_budget_usd
|
||||
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
|
||||
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
|
||||
: null),
|
||||
h(
|
||||
"td",
|
||||
{ class: "right-align" },
|
||||
|
||||
Vendored
+2
-2
@@ -5,7 +5,7 @@
|
||||
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
|
||||
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
|
||||
import { h, mount, toast, money, moneyEl, ms } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
|
||||
@@ -206,7 +206,7 @@ export async function renderPlayground() {
|
||||
decision,
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
|
||||
kv(t("pg.task"), c.task_label || "—"),
|
||||
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
|
||||
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, trTier(c.complexity.tier)) : "—"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "kv", style: "display:block" },
|
||||
|
||||
Vendored
+92
-4
@@ -1,6 +1,6 @@
|
||||
// Routing — tier editor (ordered model chips), default model, and policy.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
|
||||
@@ -71,6 +71,7 @@ export async function renderRouting() {
|
||||
const st = {
|
||||
tiers,
|
||||
taskPairs,
|
||||
bands: routing.bands || null, // null = пороги роутера как есть
|
||||
default: routing.default || ids[0] || "",
|
||||
policy: routing.policy || { mode: "fixed_table", min_class: {} },
|
||||
};
|
||||
@@ -120,11 +121,97 @@ export async function renderRouting() {
|
||||
);
|
||||
}
|
||||
|
||||
// Linked boundary sliders: one track, two handles (low|medium boundaries),
|
||||
// three colored zones. Handles push each other, numbers update live —
|
||||
// nobody types score ranges by hand.
|
||||
function bandsCard() {
|
||||
const DEF = { low: 0.33, medium: 0.48 };
|
||||
const enabled = !!st.bands;
|
||||
const b = st.bands || { ...DEF };
|
||||
const MIN_GAP = 0.02;
|
||||
|
||||
const track = h("div", { class: "bands-track" });
|
||||
const zoneLow = h("div", { class: "bands-zone z-low" });
|
||||
const zoneMed = h("div", { class: "bands-zone z-med" });
|
||||
const zoneHigh = h("div", { class: "bands-zone z-high" });
|
||||
const hLow = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
|
||||
const hMed = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
|
||||
const legend = h("div", { class: "bands-legend" });
|
||||
track.append(zoneLow, zoneMed, zoneHigh, hLow, hMed);
|
||||
|
||||
function paint() {
|
||||
const lo = b.low, me = b.medium;
|
||||
zoneLow.style.width = lo * 100 + "%";
|
||||
zoneMed.style.left = lo * 100 + "%";
|
||||
zoneMed.style.width = (me - lo) * 100 + "%";
|
||||
zoneHigh.style.left = me * 100 + "%";
|
||||
zoneHigh.style.width = (1 - me) * 100 + "%";
|
||||
hLow.style.left = lo * 100 + "%";
|
||||
hMed.style.left = me * 100 + "%";
|
||||
hLow.querySelector(".bands-val").textContent = lo.toFixed(2);
|
||||
hMed.querySelector(".bands-val").textContent = me.toFixed(2);
|
||||
// подписи не наезжают, когда ручки рядом
|
||||
hMed.querySelector(".bands-val").classList.toggle("raised", me - lo < 0.09);
|
||||
mount(legend,
|
||||
h("span", { class: "badge tier-low" }, `${trTier("low")} ≤ ${lo.toFixed(2)}`),
|
||||
h("span", { class: "badge tier-medium" }, `${trTier("medium")} ≤ ${me.toFixed(2)}`),
|
||||
h("span", { class: "badge tier-high" }, `${trTier("high")} > ${me.toFixed(2)}`));
|
||||
track.classList.toggle("disabled", !st.bands);
|
||||
}
|
||||
|
||||
function drag(handle, which) {
|
||||
handle.addEventListener("pointerdown", (e) => {
|
||||
if (!st.bands) return;
|
||||
handle.setPointerCapture(e.pointerId);
|
||||
const rect = track.getBoundingClientRect();
|
||||
const move = (ev) => {
|
||||
let v = (ev.clientX - rect.left) / rect.width;
|
||||
v = Math.min(0.98, Math.max(0.02, Math.round(v * 100) / 100));
|
||||
if (which === "low") {
|
||||
b.low = v;
|
||||
// связанность: low толкает medium перед собой
|
||||
if (b.medium < b.low + MIN_GAP) b.medium = Math.min(0.99, b.low + MIN_GAP);
|
||||
} else {
|
||||
b.medium = v;
|
||||
// и medium толкает low назад
|
||||
if (b.low > b.medium - MIN_GAP) b.low = Math.max(0.01, b.medium - MIN_GAP);
|
||||
}
|
||||
st.bands = { low: b.low, medium: b.medium };
|
||||
paint();
|
||||
};
|
||||
const up = () => {
|
||||
handle.removeEventListener("pointermove", move);
|
||||
handle.removeEventListener("pointerup", up);
|
||||
};
|
||||
handle.addEventListener("pointermove", move);
|
||||
handle.addEventListener("pointerup", up);
|
||||
});
|
||||
}
|
||||
drag(hLow, "low");
|
||||
drag(hMed, "medium");
|
||||
|
||||
const toggle = h("input", { type: "checkbox", checked: enabled ? "" : null });
|
||||
toggle.checked = enabled;
|
||||
toggle.addEventListener("change", () => {
|
||||
st.bands = toggle.checked ? { low: b.low, medium: b.medium } : null;
|
||||
paint();
|
||||
});
|
||||
paint();
|
||||
return h(
|
||||
"div",
|
||||
{ class: "bands-wrap" },
|
||||
h("label", { class: "check", style: "margin-bottom:8px" }, toggle, t("routing.bands.enable")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.bands.hint")),
|
||||
track,
|
||||
legend
|
||||
);
|
||||
}
|
||||
|
||||
function tierRow(tier) {
|
||||
return h(
|
||||
"div",
|
||||
{ class: "tierrow" },
|
||||
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
|
||||
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, trTier(tier))),
|
||||
chipEditor(st.tiers, tier)
|
||||
);
|
||||
}
|
||||
@@ -167,7 +254,7 @@ export async function renderRouting() {
|
||||
"select",
|
||||
{ class: "pair-tier", title: t("routing.tasks.tierHint") },
|
||||
h("option", { value: "any" }, t("routing.tasks.tierAny")),
|
||||
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, tr))
|
||||
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, trTier(tr)))
|
||||
);
|
||||
tierSel.value = p.tier;
|
||||
tierSel.addEventListener("change", () => { p.tier = tierSel.value; });
|
||||
@@ -257,7 +344,7 @@ export async function renderRouting() {
|
||||
}
|
||||
if (Object.keys(rule).length) tasks[k] = rule;
|
||||
}
|
||||
await api.putRouting({ tiers: st.tiers, tasks, default: st.default, policy: st.policy });
|
||||
await api.putRouting({ tiers: st.tiers, tasks, bands: st.bands, default: st.default, policy: st.policy });
|
||||
toast(t("toast.saved"), "good");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
@@ -282,6 +369,7 @@ export async function renderRouting() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
|
||||
bandsCard(),
|
||||
...Object.keys(st.tiers).map(tierRow)
|
||||
),
|
||||
tasksCard(),
|
||||
|
||||
Vendored
+24
-1
@@ -194,8 +194,30 @@ export async function renderSettings() {
|
||||
const modelsData = await api.listModels();
|
||||
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
|
||||
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
|
||||
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
|
||||
// Live proof-of-life for the cache card: entries/hits/saved — and an
|
||||
// honest warning when the cache is on but has no embedding model to work
|
||||
// with (it silently does nothing in that state).
|
||||
let cacheStats = null;
|
||||
try {
|
||||
cacheStats = (await api.stats("24h", "model")).cache || null;
|
||||
} catch { /* stats off */ }
|
||||
function cacheStatus() {
|
||||
const wrap = h("div", { style: "margin-bottom:10px" });
|
||||
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
|
||||
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
|
||||
return wrap;
|
||||
}
|
||||
if (cacheStats && s.cache?.enabled) {
|
||||
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
|
||||
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
|
||||
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
|
||||
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
|
||||
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
|
||||
}
|
||||
return wrap;
|
||||
}
|
||||
const cacheEmbedSel = h(
|
||||
"select",
|
||||
{},
|
||||
@@ -610,6 +632,7 @@ export async function renderSettings() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
|
||||
cacheStatus(),
|
||||
cacheEnabled.node,
|
||||
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
|
||||
field(t("settings.cache.embed"), cacheEmbedSel),
|
||||
|
||||
Reference in New Issue
Block a user