Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
34093330f6 | ||
|
|
faddf945d7 | ||
|
|
255d989e83 | ||
|
|
0d9d24125a | ||
|
|
1c59906b49 | ||
|
|
19f666e7b1 | ||
|
|
dbcd9827f6 | ||
|
|
b689180717 | ||
|
|
f385c2254b | ||
|
|
9607a38792 | ||
|
|
239ddc767c | ||
|
|
057abaa95a | ||
|
|
dd864eed22 |
@@ -3,6 +3,60 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.44] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Power controls in Settings**: «Stop everything & power off» stops every managed local
|
||||
model server and exits the gateway (the panel shows a clear "gateway is off" screen);
|
||||
«Restart everything» (relabeled) re-execs the gateway with its local servers. Both behind
|
||||
confirm dialogs, translated in all seven languages.
|
||||
|
||||
### Fixed
|
||||
- `cortiq-gateway` typed in any directory picks up your configuration. Resolution order:
|
||||
`--config` → a readable `./config/gateway.toml` → `$CORTIQ_GATEWAY_HOME` →
|
||||
`~/.cortiq-gateway/gateway.toml`. An unrelated project that merely contains `Cargo.toml`
|
||||
and a `config/` folder no longer hijacks the config path ("cannot read config
|
||||
config/gateway.toml").
|
||||
|
||||
## [0.2.43] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
|
||||
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
|
||||
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
|
||||
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
|
||||
configured = open mode, as before.
|
||||
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
|
||||
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
|
||||
when the server becomes healthy, so a local model whose chat template can drive tool
|
||||
calling serves agent (tools) traffic with zero manual configuration. Manual override:
|
||||
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
|
||||
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
|
||||
remaining credit is a KPI card, red under $5.
|
||||
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
|
||||
per harness, an install command chip when the harness is missing, and a paste-your-key
|
||||
field that substitutes the real key into every snippet.
|
||||
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
|
||||
Caddy TLS fronting, security checklist, backup list.
|
||||
|
||||
### Fixed
|
||||
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
|
||||
route's chain has no model with the required capabilities (tools), the gateway escalates
|
||||
through medium→high chains and then the rest of the pool instead of returning
|
||||
"No healthy model available". Explicit pins still fail loudly.
|
||||
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
|
||||
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
|
||||
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
|
||||
no history-expansion `!`), and the install chips are readable in the light theme.
|
||||
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
|
||||
into the sidecar totals file — per-account all-time and month spend survive compaction
|
||||
and restarts.
|
||||
|
||||
### Breaking
|
||||
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
|
||||
will now be rejected with 401 — either remove the keys from the config or start sending
|
||||
`Authorization: Bearer <key>`.
|
||||
|
||||
## [0.2.42] - 2026-08-08
|
||||
|
||||
### Added
|
||||
|
||||
Generated
+1
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.42"
|
||||
version = "0.2.44"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.42"
|
||||
version = "0.2.44"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
|
||||
@@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
|
||||
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
|
||||
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
|
||||
|
||||
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Admin console
|
||||
|
||||
@@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
||||
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
|
||||
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
|
||||
|
||||
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Панель управления
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
# Deploying cortiq-gateway for a team / office
|
||||
|
||||
One gateway on a small server, API keys per department, local models for the
|
||||
cheap tiers, cloud models for the hard ones. This page covers the ops side.
|
||||
|
||||
## 1. Run as a service
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Task Scheduler → Create Task → "Run whether user is logged on or not",
|
||||
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
|
||||
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS and the outside world
|
||||
|
||||
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
|
||||
and bind the gateway to localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE streaming
|
||||
proxy_read_timeout 3600s; # long generations
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
|
||||
same with automatic certificates.)
|
||||
|
||||
## 3. Security checklist
|
||||
|
||||
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
|
||||
the console warns that it is open to the network.
|
||||
- **API keys**: create one per department/developer on the keys page. With at
|
||||
least one key configured the model APIs require Bearer / x-api-key —
|
||||
requests without a valid key get 401.
|
||||
- **Rate limits**: per-key requests/min → 429 with Retry-After.
|
||||
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
|
||||
account's month-to-date spend crosses it.
|
||||
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
|
||||
|
||||
## 4. Backups
|
||||
|
||||
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
|
||||
- `secrets.toml` — provider API keys (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
|
||||
cumulative per-key totals and month-to-date budget spend.
|
||||
|
||||
Copy those three and you can rebuild the instance anywhere.
|
||||
|
||||
## 5. Observability
|
||||
|
||||
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
|
||||
OpenRouter balance (red under $5).
|
||||
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
|
||||
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Развёртывание cortiq-gateway для команды / офиса
|
||||
|
||||
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
|
||||
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
|
||||
|
||||
## 1. Запуск как сервис
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
|
||||
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
|
||||
--no-port-search`. Либо [NSSM](https://nssm.cc):
|
||||
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS и внешний мир
|
||||
|
||||
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
|
||||
привяжите к localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE-стриминг
|
||||
proxy_read_timeout 3600s; # долгие генерации
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
|
||||
с автоматическими сертификатами.)
|
||||
|
||||
## 3. Чек-лист безопасности
|
||||
|
||||
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
|
||||
него консоль предупреждает, что открыта сети.
|
||||
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
|
||||
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
|
||||
запросы без валидного ключа получают 401.
|
||||
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
|
||||
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
|
||||
траты аккаунта за месяц превысят лимит.
|
||||
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
|
||||
|
||||
## 4. Бэкапы
|
||||
|
||||
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
|
||||
- `secrets.toml` — ключи провайдеров (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
|
||||
тоталы по ключам и месячные траты для бюджетов.
|
||||
|
||||
Скопируйте эти три файла — и инстанс поднимается где угодно.
|
||||
|
||||
## 5. Наблюдаемость
|
||||
|
||||
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
|
||||
сложности, баланс OpenRouter (красный ниже $5).
|
||||
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
|
||||
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
|
||||
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
get(get_protocols).put(put_protocols),
|
||||
)
|
||||
.route("/admin/api/harnesses", get(get_harnesses))
|
||||
.route("/admin/api/balance", get(get_balance))
|
||||
.route("/admin/api/settings", get(get_settings).put(put_settings))
|
||||
.route("/admin/api/keys", get(list_keys).post(create_key))
|
||||
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
|
||||
@@ -183,6 +184,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
.route("/admin/api/logs", get(get_logs).delete(clear_logs))
|
||||
.route("/admin/api/system", get(get_system))
|
||||
.route("/admin/api/restart", post(restart_gateway))
|
||||
.route("/admin/api/shutdown", post(shutdown_gateway))
|
||||
.route("/admin/api/bench", get(bench_list).post(bench_start))
|
||||
.route("/admin/api/config/export", get(config_export))
|
||||
.route("/admin/api/config/import", post(config_import))
|
||||
@@ -1008,6 +1010,51 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
|
||||
ok(json!({ "ok": true }))
|
||||
}
|
||||
|
||||
// ─────────────────────────────── balance ───────────────────────────────────
|
||||
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
|
||||
// (null) when no OpenRouter key is configured — the card hides itself.
|
||||
|
||||
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
|
||||
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
|
||||
std::sync::OnceLock::new();
|
||||
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
|
||||
if let Some((at, v)) = *cache.lock().unwrap() {
|
||||
if at.elapsed() < std::time::Duration::from_secs(300) {
|
||||
return ok(json!({ "openrouter_usd": v }));
|
||||
}
|
||||
}
|
||||
let key = state
|
||||
.secrets
|
||||
.resolve("OPENROUTER_API_KEY")
|
||||
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
|
||||
let Some(key) = key.filter(|k| !k.is_empty()) else {
|
||||
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
|
||||
};
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(8))
|
||||
.build()
|
||||
.unwrap_or_default();
|
||||
let resp = client
|
||||
.get("https://openrouter.ai/api/v1/credits")
|
||||
.bearer_auth(&key)
|
||||
.send()
|
||||
.await;
|
||||
let balance = match resp {
|
||||
Ok(r) if r.status().is_success() => {
|
||||
let v: serde_json::Value = r.json().await.unwrap_or_default();
|
||||
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
|
||||
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
|
||||
Some(total - used)
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
let Some(balance) = balance else {
|
||||
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
|
||||
};
|
||||
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
|
||||
ok(json!({ "openrouter_usd": balance }))
|
||||
}
|
||||
|
||||
// ──────────────────────────── harness detection ────────────────────────────
|
||||
// Which coding harnesses are present on the machine the gateway runs on, so the
|
||||
// API-keys view can mark tabs "installed" / "already configured for cortiq".
|
||||
@@ -1357,6 +1404,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
|
||||
"key_masked": mask_key(&k.key),
|
||||
"account": k.account,
|
||||
"rate_per_min": k.rate_per_min,
|
||||
"monthly_budget_usd": k.monthly_budget_usd,
|
||||
"allow_models": k.allow_models,
|
||||
})
|
||||
})
|
||||
@@ -1372,6 +1420,8 @@ struct NewKey {
|
||||
#[serde(default)]
|
||||
rate_per_min: u32,
|
||||
#[serde(default)]
|
||||
monthly_budget_usd: Option<f64>,
|
||||
#[serde(default)]
|
||||
allow_models: Vec<String>,
|
||||
}
|
||||
|
||||
@@ -1390,6 +1440,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
|
||||
account: b.account,
|
||||
rate_per_min: b.rate_per_min,
|
||||
allow_models: b.allow_models,
|
||||
monthly_budget_usd: b.monthly_budget_usd,
|
||||
});
|
||||
state.reload(cfg)?;
|
||||
// show the full key only once
|
||||
@@ -2147,6 +2198,7 @@ fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
});
|
||||
Some(id)
|
||||
}
|
||||
@@ -2221,6 +2273,20 @@ async fn get_system(State(state): State<SharedState>) -> ApiResult {
|
||||
ok(serde_json::to_value(snap).unwrap_or_else(|_| json!({})))
|
||||
}
|
||||
|
||||
/// Stop every managed local model server and power the gateway off. The 200
|
||||
/// goes out first so the SPA can show its "gateway is off" screen; nothing is
|
||||
/// respawned — the operator starts the gateway again by hand (or the Docker
|
||||
/// restart policy does, in which case this behaves like a restart).
|
||||
async fn shutdown_gateway(State(state): State<SharedState>) -> ApiResult {
|
||||
state.cmf.stop().await;
|
||||
tokio::spawn(async move {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
|
||||
tracing::info!("shutdown requested from the admin console");
|
||||
std::process::exit(0);
|
||||
});
|
||||
ok(json!({ "ok": true }))
|
||||
}
|
||||
|
||||
/// Restart the gateway in place: stop managed model servers, spawn a fresh
|
||||
/// copy of ourselves (it retries the bind while our port frees), then exit.
|
||||
/// In Docker we only exit — we are PID 1, a spawned child would die with the
|
||||
|
||||
+1
-1
@@ -55,7 +55,7 @@ impl SemanticCache {
|
||||
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
enabled: cfg.enabled,
|
||||
threshold: cfg.threshold,
|
||||
threshold: cfg.threshold as f32,
|
||||
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
|
||||
max_entries: cfg.max_entries.max(1),
|
||||
embed_model: cfg.embed_model.clone(),
|
||||
|
||||
+27
-5
@@ -56,6 +56,9 @@ struct Managed {
|
||||
pub struct CmfRuntime {
|
||||
status: Mutex<CmfStatus>,
|
||||
children: Mutex<Vec<Managed>>,
|
||||
/// Server ids whose healthz handshake advertised tool support — lets the
|
||||
/// pipeline route tools traffic to local models with no manual caps.
|
||||
pub detected_tools: Mutex<std::collections::HashSet<String>>,
|
||||
}
|
||||
|
||||
impl CmfRuntime {
|
||||
@@ -63,6 +66,7 @@ impl CmfRuntime {
|
||||
Arc::new(Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -193,6 +197,7 @@ impl Default for CmfRuntime {
|
||||
Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -332,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline
|
||||
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
|
||||
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
|
||||
/// caller can register them; older servers return `None` there.
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
|
||||
let url = format!("http://{host}:{port}/healthz");
|
||||
let client = reqwest::Client::new();
|
||||
for _ in 0..tries {
|
||||
@@ -344,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
.await
|
||||
{
|
||||
if r.status().is_success() {
|
||||
return true;
|
||||
let tools = r
|
||||
.json::<serde_json::Value>()
|
||||
.await
|
||||
.ok()
|
||||
.and_then(|v| v["capabilities"]["tools"].as_bool());
|
||||
return (true, tools);
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
false
|
||||
(false, None)
|
||||
}
|
||||
|
||||
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
||||
@@ -432,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
|
||||
if !rt.is_running(&server.id) {
|
||||
continue; // failed to spawn / file missing
|
||||
}
|
||||
if wait_healthy(&cfg.local_host, server.port, 240).await {
|
||||
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
|
||||
if ok {
|
||||
rt.set_healthy(&server.id, true);
|
||||
// capability handshake: the server said whether its model's chat
|
||||
// template can drive tool calling — no manual caps needed
|
||||
if tools == Some(true) {
|
||||
rt.detected_tools.lock().unwrap().insert(server.id.clone());
|
||||
tracing::info!(id = %server.id, "local server advertises tool support");
|
||||
} else {
|
||||
rt.detected_tools.lock().unwrap().remove(&server.id);
|
||||
}
|
||||
rt.log(format!(
|
||||
"local CMF server ready — '{}' at http://{}:{}/v1",
|
||||
server.id, cfg.local_host, server.port
|
||||
|
||||
+25
-7
@@ -130,6 +130,10 @@ pub struct CmfServer {
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
|
||||
/// gateway auto-detects tool support from the server's healthz handshake.
|
||||
#[serde(default)]
|
||||
pub caps: Vec<String>,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -229,6 +233,7 @@ impl CmfCfg {
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
@@ -620,6 +625,9 @@ pub struct ApiKeyCfg {
|
||||
pub account: String,
|
||||
#[serde(default)]
|
||||
pub rate_per_min: u32,
|
||||
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub monthly_budget_usd: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub allow_models: Vec<String>,
|
||||
}
|
||||
@@ -829,9 +837,10 @@ fn default_shadow_soak() -> usize {
|
||||
pub struct CacheCfg {
|
||||
#[serde(default)]
|
||||
pub enabled: bool,
|
||||
/// Cosine-similarity threshold for a hit (0..1).
|
||||
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
|
||||
/// the number you typed (an f32 round-trips as 0.9200000166893005).
|
||||
#[serde(default = "default_semcache_threshold")]
|
||||
pub threshold: f32,
|
||||
pub threshold: f64,
|
||||
/// Entry time-to-live, e.g. `1h`.
|
||||
#[serde(default = "default_semcache_ttl")]
|
||||
pub ttl: String,
|
||||
@@ -853,7 +862,7 @@ impl Default for CacheCfg {
|
||||
}
|
||||
}
|
||||
}
|
||||
fn default_semcache_threshold() -> f32 {
|
||||
fn default_semcache_threshold() -> f64 {
|
||||
0.92
|
||||
}
|
||||
fn default_semcache_ttl() -> String {
|
||||
@@ -915,14 +924,23 @@ impl Default for Config {
|
||||
}
|
||||
|
||||
/// Where the config lives when `--config` is not given.
|
||||
/// Repo checkout (`config/gateway.toml` exists or `config/` dir exists) → keep it there;
|
||||
/// installed binary (no repo) → `$data_dir/gateway.toml`.
|
||||
/// A readable `./config/gateway.toml` (a real gateway checkout / install dir)
|
||||
/// wins; otherwise the per-user home (`$CORTIQ_GATEWAY_HOME` or
|
||||
/// `~/.cortiq-gateway`) — so `cortiq-gateway` typed in any directory picks up
|
||||
/// the same configuration it was set up with. A foreign project that merely
|
||||
/// has `Cargo.toml` and a `config/` folder must NOT capture the config
|
||||
/// (that used to shadow the user's real config with a broken path).
|
||||
pub fn default_config_path() -> String {
|
||||
let repo_path = "config/gateway.toml";
|
||||
if std::path::Path::new(repo_path).exists() {
|
||||
// the file itself, readable — not just an entry that exists()
|
||||
if std::fs::read_to_string(repo_path).is_ok() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
if std::path::Path::new("Cargo.toml").exists() && std::path::Path::new("config").exists() {
|
||||
// our own repo checkout before first run: config/ exists, file not yet
|
||||
let ours = std::fs::read_to_string("Cargo.toml")
|
||||
.map(|t| t.contains("name = \"cortiq-gateway\""))
|
||||
.unwrap_or(false);
|
||||
if ours && std::path::Path::new("config").is_dir() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
data_dir()
|
||||
|
||||
@@ -13,6 +13,8 @@ pub enum GatewayError {
|
||||
Unauthorized(String),
|
||||
#[error("rate limit exceeded")]
|
||||
RateLimited { retry_after_secs: u64 },
|
||||
#[error("monthly budget exhausted (limit ${budget_usd})")]
|
||||
BudgetExhausted { budget_usd: f64 },
|
||||
#[error("upstream unavailable: {0}")]
|
||||
UpstreamUnavailable(String),
|
||||
#[error("internal error: {0}")]
|
||||
@@ -37,6 +39,11 @@ impl GatewayError {
|
||||
"rate_limit_error",
|
||||
"rate_limited",
|
||||
),
|
||||
Self::BudgetExhausted { .. } => (
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"insufficient_quota",
|
||||
"insufficient_quota",
|
||||
),
|
||||
Self::UpstreamUnavailable(_) => (
|
||||
StatusCode::BAD_GATEWAY,
|
||||
"upstream_unavailable",
|
||||
|
||||
+15
-3
@@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
|
||||
.route("/health", get(|| async { "ok" }))
|
||||
.route("/readyz", get(|| async { "ready" }))
|
||||
.route("/metrics", get(admin::metrics))
|
||||
.merge(protocols::build_router());
|
||||
.merge(protocols::build_router(state.clone()));
|
||||
|
||||
if admin_enabled {
|
||||
app = app
|
||||
@@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> {
|
||||
.allow_methods(tower_http::cors::Any)
|
||||
.allow_headers(tower_http::cors::Any);
|
||||
|
||||
let app = app.with_state(state).layer(cors);
|
||||
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
|
||||
// small enough that a hostile client can't balloon memory
|
||||
let app =
|
||||
app.with_state(state)
|
||||
.layer(cors)
|
||||
.layer(tower_http::limit::RequestBodyLimitLayer::new(
|
||||
32 * 1024 * 1024,
|
||||
));
|
||||
|
||||
let listener = {
|
||||
let mut attempt = 0u32;
|
||||
@@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
|
||||
tracing::info!("open this URL to finish setup: {url}");
|
||||
}
|
||||
}
|
||||
axum::serve(listener, app).await?;
|
||||
axum::serve(listener, app)
|
||||
.with_graceful_shutdown(async {
|
||||
let _ = tokio::signal::ctrl_c().await;
|
||||
tracing::info!("shutdown signal received — draining connections");
|
||||
})
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
+56
-10
@@ -183,6 +183,9 @@ impl Pipeline {
|
||||
&self,
|
||||
req: &ChatRequest,
|
||||
live: &crate::state::Live,
|
||||
// Tool support detected via the local servers' healthz handshake —
|
||||
// treated as equivalent to declared caps when filtering candidates.
|
||||
detected_tools: &std::collections::HashSet<String>,
|
||||
) -> Result<(RouteDecision, String, Vec<String>)> {
|
||||
// 1. Extract text for classification
|
||||
let text = self.extract_text(
|
||||
@@ -274,10 +277,13 @@ impl Pipeline {
|
||||
|
||||
// Filter candidates by capabilities (e.g. tools support)
|
||||
let needs_tools = !req.tools.is_empty();
|
||||
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
|
||||
prov.caps().tools || detected_tools.contains(id)
|
||||
};
|
||||
let mut final_candidates = Vec::new();
|
||||
for model_id in candidates {
|
||||
if let Some(prov) = live.registry.get(&model_id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&model_id, &prov) {
|
||||
tracing::debug!(
|
||||
"Candidate '{}' skipped: lacks support for requested tools",
|
||||
model_id
|
||||
@@ -306,13 +312,49 @@ impl Pipeline {
|
||||
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
|
||||
for id in live.local_candidates() {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Capability widening: a tier chain built of tools-less local models
|
||||
// must not kill an agent request. When the filtered chain is empty on
|
||||
// an AUTO route, widen to any pool model that satisfies the required
|
||||
// capabilities — preferring the routing table's own medium→high order
|
||||
// (an agentic request is worth a capable model), then the rest of the
|
||||
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
|
||||
// this every low-tier verdict was a hard failure.
|
||||
if final_candidates.is_empty() {
|
||||
let mut widened: Vec<String> = Vec::new();
|
||||
for tier in ["medium", "high"] {
|
||||
for id in live.routing.candidates(tier) {
|
||||
if !widened.contains(&id) {
|
||||
widened.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in live.registry.ids() {
|
||||
if !widened.contains(id) {
|
||||
widened.push(id.clone());
|
||||
}
|
||||
}
|
||||
for id in widened {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
if !final_candidates.is_empty() {
|
||||
tracing::info!(
|
||||
chosen = %final_candidates[0],
|
||||
"capability widening: tier chain had no model with the required capabilities"
|
||||
);
|
||||
}
|
||||
}
|
||||
if final_candidates.is_empty() {
|
||||
return Err(GatewayError::UpstreamUnavailable(
|
||||
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
||||
@@ -324,7 +366,9 @@ impl Pipeline {
|
||||
|
||||
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
||||
let live = state.live();
|
||||
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, mut final_candidates) =
|
||||
self.resolve(&req, &live, &detected_tools).await?;
|
||||
|
||||
// Serving-when-promoted (gated): a promoted task-label is served by the
|
||||
// local model FIRST (failover to cloud on error). Per-request VETO uses
|
||||
@@ -471,13 +515,15 @@ impl Pipeline {
|
||||
.to_string();
|
||||
|
||||
let live = state.live();
|
||||
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, final_candidates) =
|
||||
match self.resolve(&req, &live, &detected_tools).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
|
||||
let mut last_err = None;
|
||||
let mut attempt = 0usize;
|
||||
|
||||
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<MessagesRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.anthropic_messages {
|
||||
@@ -108,6 +109,7 @@ async fn handler(
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "anthropic_messages".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
+51
-1
@@ -12,10 +12,59 @@ pub mod openai_models;
|
||||
use crate::state::SharedState;
|
||||
use axum::Router;
|
||||
|
||||
/// Account resolved by the API-key middleware, consumed by protocol handlers
|
||||
/// for attribution (stats, budgets). Present only when keys are configured.
|
||||
#[derive(Clone)]
|
||||
pub struct AccountTag(pub String);
|
||||
|
||||
/// API-key gate for the model-facing surface. Open mode (no keys configured)
|
||||
/// passes everything through — the admin console already warns about it.
|
||||
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
|
||||
/// enforced with a fixed one-minute window (429 + Retry-After), and a
|
||||
/// monthly budget, when set, turns into 429 insufficient_quota once the
|
||||
/// account's month-to-date spend crosses it.
|
||||
async fn api_auth(
|
||||
axum::extract::State(state): axum::extract::State<SharedState>,
|
||||
mut req: axum::extract::Request,
|
||||
next: axum::middleware::Next,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
let live = state.live();
|
||||
if live.cfg.api_keys.is_empty() {
|
||||
return next.run(req).await;
|
||||
}
|
||||
let provided = req
|
||||
.headers()
|
||||
.get(axum::http::header::AUTHORIZATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
|
||||
.map(str::trim)
|
||||
.unwrap_or("");
|
||||
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
|
||||
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
|
||||
.into_response();
|
||||
};
|
||||
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
|
||||
return crate::error::GatewayError::RateLimited {
|
||||
retry_after_secs: state.rate.retry_after_secs(),
|
||||
}
|
||||
.into_response();
|
||||
}
|
||||
if let Some(budget) = key.monthly_budget_usd {
|
||||
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
|
||||
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
|
||||
.into_response();
|
||||
}
|
||||
}
|
||||
req.extensions_mut().insert(AccountTag(key.account.clone()));
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
/// Build the protocols router. Implemented adapters are **always** mounted;
|
||||
/// each handler checks the live `protocols.*` flag from config itself, so
|
||||
/// toggles in the admin panel take effect without a restart (hot switching).
|
||||
pub fn build_router() -> Router<SharedState> {
|
||||
pub fn build_router(state: SharedState) -> Router<SharedState> {
|
||||
Router::new()
|
||||
.merge(openai_chat::routes())
|
||||
.merge(openai_completions::routes())
|
||||
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
|
||||
.merge(anthropic_messages::routes())
|
||||
.merge(mcp::routes())
|
||||
.merge(native_passthrough::routes())
|
||||
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
|
||||
}
|
||||
|
||||
@@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<OpenAiChatRequest>,
|
||||
) -> Result<Response> {
|
||||
// hot protocol toggle: if the adapter is disabled in config — return 404
|
||||
@@ -104,6 +105,7 @@ async fn handler(
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_chat".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<CompletionsRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.openai_completions {
|
||||
@@ -89,6 +90,7 @@ async fn handler(
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_completions".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
+1
-1
@@ -48,7 +48,7 @@ impl Registry {
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
caps: s.caps.clone(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
|
||||
@@ -66,6 +66,42 @@ impl Live {
|
||||
}
|
||||
}
|
||||
|
||||
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
|
||||
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
|
||||
/// fine for office-grade fairness and costs one HashMap lookup per request.
|
||||
#[derive(Default)]
|
||||
pub struct RateWindows {
|
||||
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
|
||||
}
|
||||
|
||||
impl RateWindows {
|
||||
pub fn allow(&self, key: &str, per_min: u32) -> bool {
|
||||
let minute = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
/ 60;
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
let e = g.entry(key.to_string()).or_insert((minute, 0));
|
||||
if e.0 != minute {
|
||||
*e = (minute, 0);
|
||||
}
|
||||
if e.1 >= per_min {
|
||||
return false;
|
||||
}
|
||||
e.1 += 1;
|
||||
true
|
||||
}
|
||||
/// Seconds until the current minute window rolls over.
|
||||
pub fn retry_after_secs(&self) -> u64 {
|
||||
60 - (std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
% 60)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct AppState {
|
||||
pub live: ArcSwap<Live>,
|
||||
pub stats: Arc<Stats>,
|
||||
@@ -85,6 +121,8 @@ pub struct AppState {
|
||||
/// Managed local CMF model server (install/update/spawn lifecycle + status).
|
||||
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
|
||||
pub cache: Arc<SemanticCache>,
|
||||
/// Per-API-key fixed-window rate limiter.
|
||||
pub rate: RateWindows,
|
||||
pub secrets: SecretStore,
|
||||
pub config_path: String,
|
||||
pub pipeline: crate::pipeline::Pipeline,
|
||||
@@ -164,6 +202,7 @@ impl SharedState {
|
||||
admin_open_public: std::sync::atomic::AtomicBool::new(false),
|
||||
cmf,
|
||||
cache,
|
||||
rate: RateWindows::default(),
|
||||
secrets,
|
||||
config_path,
|
||||
pipeline: crate::pipeline::Pipeline::new(),
|
||||
|
||||
+112
-9
@@ -137,6 +137,11 @@ struct Inner {
|
||||
/// = folded + by_account_alltime, with no double counting: a record is
|
||||
/// either in the file or folded, never both.
|
||||
folded: HashMap<String, Totals>,
|
||||
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
/// Records with ts ≤ this are already reflected in the persisted
|
||||
/// month_spend (watermark against double counting on replay).
|
||||
month_upto_ts: u64,
|
||||
/// Appends since the last compaction size check.
|
||||
writes_since_check: u32,
|
||||
}
|
||||
@@ -193,6 +198,7 @@ impl Stats {
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
if rec.ts >= cutoff {
|
||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||
}
|
||||
@@ -200,6 +206,33 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
|
||||
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
|
||||
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
|
||||
return;
|
||||
}
|
||||
let m = month_of(rec.ts);
|
||||
let e = inner
|
||||
.month_spend
|
||||
.entry(rec.account.clone())
|
||||
.or_insert((m.clone(), 0.0));
|
||||
if e.0 != m {
|
||||
*e = (m, 0.0);
|
||||
}
|
||||
e.1 += rec.cost_usd;
|
||||
}
|
||||
|
||||
/// Month-to-date spend for an account (budget checks).
|
||||
pub fn month_spend(&self, account: &str) -> f64 {
|
||||
let inner = self.inner.lock().unwrap();
|
||||
let m = month_of(now_secs());
|
||||
inner
|
||||
.month_spend
|
||||
.get(account)
|
||||
.filter(|(mm, _)| *mm == m)
|
||||
.map(|(_, v)| *v)
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
|
||||
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
|
||||
inner.recent.push_back(rec.clone());
|
||||
while inner.recent.len() > ring_size {
|
||||
@@ -256,6 +289,7 @@ impl Stats {
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
inner.writes_since_check += 1;
|
||||
}
|
||||
if let Some(path) = &self.file {
|
||||
@@ -283,11 +317,42 @@ impl Stats {
|
||||
let Ok(content) = std::fs::read_to_string(&path) else {
|
||||
return;
|
||||
};
|
||||
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||
#[derive(serde::Deserialize)]
|
||||
struct Sidecar {
|
||||
#[serde(default)]
|
||||
by_account: HashMap<String, Totals>,
|
||||
#[serde(default)]
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
#[serde(default)]
|
||||
month_upto_ts: u64,
|
||||
}
|
||||
// new format {by_account, month_spend, month_upto_ts}; the original
|
||||
// file held the bare by_account map — accept both
|
||||
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
inner.folded = sc.by_account;
|
||||
inner.month_spend = sc.month_spend;
|
||||
inner.month_upto_ts = sc.month_upto_ts;
|
||||
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||
self.inner.lock().unwrap().folded = m;
|
||||
}
|
||||
}
|
||||
|
||||
fn write_sidecar(&self, inner: &Inner) {
|
||||
let Some(tpath) = self.totals_path() else {
|
||||
return;
|
||||
};
|
||||
let payload = serde_json::json!({
|
||||
"by_account": inner.folded,
|
||||
"month_spend": inner.month_spend,
|
||||
"month_upto_ts": now_secs(),
|
||||
});
|
||||
let tmp = format!("{tpath}.tmp");
|
||||
if std::fs::write(&tmp, payload.to_string()).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &tpath);
|
||||
}
|
||||
}
|
||||
|
||||
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
|
||||
/// older than the retention window fold into the cumulative per-account
|
||||
/// sidecar and are dropped from the file (tmp + rename on both files, so
|
||||
@@ -305,6 +370,10 @@ impl Stats {
|
||||
return;
|
||||
}
|
||||
inner.writes_since_check = 0;
|
||||
// piggyback: persist month-to-date spend so budgets survive
|
||||
// restarts even between compactions
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
}
|
||||
}
|
||||
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
|
||||
@@ -359,14 +428,8 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
// persist the folded totals FIRST, then swap the trimmed file in
|
||||
if let Some(tpath) = self.totals_path() {
|
||||
if let Ok(json) = serde_json::to_string(&inner.folded) {
|
||||
let tmp = format!("{tpath}.tmp");
|
||||
if std::fs::write(&tmp, json).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &tpath);
|
||||
}
|
||||
}
|
||||
}
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
let tmp = format!("{path}.tmp");
|
||||
if std::fs::write(&tmp, kept).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, path);
|
||||
@@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
|
||||
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
|
||||
}
|
||||
|
||||
fn month_of(ts: u64) -> String {
|
||||
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
|
||||
let days = ts / 86_400;
|
||||
let mut year = 1970u64;
|
||||
let mut d = days;
|
||||
loop {
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let len = if leap { 366 } else { 365 };
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
year += 1;
|
||||
}
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let ml = [
|
||||
31,
|
||||
if leap { 29 } else { 28 },
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
];
|
||||
let mut month = 1;
|
||||
for len in ml {
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
month += 1;
|
||||
}
|
||||
format!("{year:04}-{month:02}")
|
||||
}
|
||||
|
||||
fn avg(sum: u64, n: u64) -> u64 {
|
||||
sum.checked_div(n).unwrap_or(0)
|
||||
}
|
||||
|
||||
Vendored
+11
-3
@@ -599,17 +599,20 @@ body.modal-open { overflow: hidden; }
|
||||
|
||||
/* Complexity bands slider */
|
||||
.bands-wrap { margin: 4px 0 16px; }
|
||||
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--panel-2, #23252c); margin: 26px 8px 10px; }
|
||||
/* one-line command chip (install commands etc.) — theme-aware */
|
||||
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
|
||||
|
||||
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
|
||||
.bands-track.disabled { opacity: .45; pointer-events: none; }
|
||||
.bands-zone { position: absolute; top: 0; bottom: 0; }
|
||||
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
|
||||
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
|
||||
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
|
||||
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
|
||||
background: var(--fg, #fff); border: 3px solid var(--accent, #6c6cf5); cursor: grab; touch-action: none; z-index: 2; }
|
||||
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
|
||||
.bands-handle:active { cursor: grabbing; }
|
||||
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
|
||||
font-family: var(--mono, monospace); color: var(--fg); background: var(--panel, #1b1d23); padding: 1px 6px;
|
||||
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
|
||||
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
|
||||
.bands-val.raised { top: -46px; }
|
||||
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
|
||||
@@ -703,3 +706,8 @@ tbody tr:last-child { animation: bm-in .25s ease; }
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sidebar, .mobile-nav-backdrop { transition: none; }
|
||||
}
|
||||
|
||||
/* full-screen farewell after "stop everything & power off" */
|
||||
.shutdown-screen { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100vh; gap: 12px; text-align: center; padding: 24px; }
|
||||
.shutdown-screen .shutdown-icon { font-size: 64px; color: var(--bad); }
|
||||
.shutdown-screen .hint { max-width: 480px; }
|
||||
|
||||
Vendored
+2
@@ -74,6 +74,7 @@ export const api = {
|
||||
// protocols
|
||||
getProtocols: () => j("GET", "/protocols"),
|
||||
harnesses: () => j("GET", "/harnesses"),
|
||||
balance: () => j("GET", "/balance"),
|
||||
putProtocols: (b) => j("PUT", "/protocols", b),
|
||||
|
||||
// settings
|
||||
@@ -133,6 +134,7 @@ export const api = {
|
||||
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
|
||||
clearLogs: () => j("DELETE", "/logs"),
|
||||
restart: () => j("POST", "/restart"),
|
||||
shutdown: () => j("POST", "/shutdown"),
|
||||
benchStart: (model) => j("POST", "/bench", { model }),
|
||||
benchList: () => j("GET", "/bench"),
|
||||
|
||||
|
||||
Vendored
+17
-2
@@ -536,9 +536,13 @@ export default {
|
||||
"keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
|
||||
"keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
|
||||
"keys.snip.keyPh": "dein API-Schlüssel",
|
||||
"settings.restart": "Gateway neu starten",
|
||||
"settings.restartConfirm": "Gateway neu starten? Laufende Anfragen brechen ab; lokale Modellserver starten ebenfalls neu.",
|
||||
"settings.restart": "Alles neu starten",
|
||||
"settings.restartConfirm": "Gateway und alle lokalen Modellserver neu starten? Laufende Anfragen werden abgebrochen.",
|
||||
"settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
|
||||
"settings.shutdown": "Alles stoppen & ausschalten",
|
||||
"settings.shutdownConfirm": "Alle lokalen Modellserver stoppen und das Gateway ausschalten? API und dieses Panel antworten erst wieder, wenn Sie das Gateway neu starten.",
|
||||
"settings.shutdownDone": "Das Gateway ist ausgeschaltet.",
|
||||
"settings.shutdownHint": "Alle lokalen Modellserver sind gestoppt, der Prozess wurde beendet. Starten Sie das Gateway auf dem Server neu, um das Panel zurückzubekommen.",
|
||||
"bench.title": "Geschwindigkeit lokaler Modelle",
|
||||
"bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
|
||||
"bench.run": "Messen",
|
||||
@@ -635,4 +639,15 @@ export default {
|
||||
"tier.medium": "mittel",
|
||||
"tier.high": "schwer",
|
||||
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
|
||||
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
|
||||
"settings.cache.entries": "Einträge",
|
||||
"settings.cache.hits": "Treffer",
|
||||
"settings.cache.saved": "gespart",
|
||||
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
|
||||
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
|
||||
"keys.budgetOf": "Budget {b}/Mon.",
|
||||
"dash.kpi.orBalance": "OpenRouter-Guthaben",
|
||||
"keys.h.install": "Installieren:",
|
||||
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
|
||||
"keys.snip.apply": "Schlüssel einsetzen",
|
||||
};
|
||||
|
||||
Vendored
+17
-2
@@ -529,9 +529,13 @@ export default {
|
||||
"keys.snip.withKey": "Snippets below include the key you just created.",
|
||||
"keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
|
||||
"keys.snip.keyPh": "your API key",
|
||||
"settings.restart": "Restart gateway",
|
||||
"settings.restartConfirm": "Restart the gateway? Requests in flight will be dropped; local model servers restart too.",
|
||||
"settings.restart": "Restart everything",
|
||||
"settings.restartConfirm": "Restart the gateway and all local model servers? Requests in flight will be dropped.",
|
||||
"settings.restarting": "Restarting — the page reloads when the gateway is back…",
|
||||
"settings.shutdown": "Stop everything & power off",
|
||||
"settings.shutdownConfirm": "Stop all local model servers and power the gateway off? The API and this panel stop answering until you start the gateway again.",
|
||||
"settings.shutdownDone": "The gateway is off.",
|
||||
"settings.shutdownHint": "All local model servers are stopped and the process has exited. Start the gateway again on the server to bring the panel back.",
|
||||
"bench.title": "Local model speed",
|
||||
"bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
|
||||
"bench.run": "Measure",
|
||||
@@ -628,4 +632,15 @@ export default {
|
||||
"tier.medium": "medium",
|
||||
"tier.high": "high",
|
||||
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
|
||||
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
|
||||
"settings.cache.entries": "entries",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "saved",
|
||||
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
|
||||
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
|
||||
"keys.budgetOf": "budget {b}/mo",
|
||||
"dash.kpi.orBalance": "OpenRouter balance",
|
||||
"keys.h.install": "Install:",
|
||||
"keys.snip.keyInputPh": "paste your API key",
|
||||
"keys.snip.apply": "Insert key",
|
||||
};
|
||||
|
||||
Vendored
+17
-2
@@ -536,9 +536,13 @@ export default {
|
||||
"keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
|
||||
"keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
|
||||
"keys.snip.keyPh": "tu clave API",
|
||||
"settings.restart": "Reiniciar la pasarela",
|
||||
"settings.restartConfirm": "¿Reiniciar la pasarela? Las peticiones en curso se cortarán; los servidores de modelos locales también se reinician.",
|
||||
"settings.restart": "Reiniciar todo",
|
||||
"settings.restartConfirm": "¿Reiniciar la pasarela y todos los servidores de modelos locales? Las solicitudes en curso se interrumpirán.",
|
||||
"settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
|
||||
"settings.shutdown": "Detener todo y apagar",
|
||||
"settings.shutdownConfirm": "¿Detener todos los servidores de modelos locales y apagar la pasarela? La API y este panel dejarán de responder hasta que vuelva a iniciar la pasarela.",
|
||||
"settings.shutdownDone": "La pasarela está apagada.",
|
||||
"settings.shutdownHint": "Todos los servidores de modelos locales están detenidos y el proceso ha terminado. Inicie de nuevo la pasarela en el servidor para recuperar el panel.",
|
||||
"bench.title": "Velocidad de modelos locales",
|
||||
"bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
|
||||
"bench.run": "Medir",
|
||||
@@ -635,4 +639,15 @@ export default {
|
||||
"tier.medium": "medio",
|
||||
"tier.high": "difícil",
|
||||
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
|
||||
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
|
||||
"settings.cache.entries": "entradas",
|
||||
"settings.cache.hits": "aciertos",
|
||||
"settings.cache.saved": "ahorrado",
|
||||
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
|
||||
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
|
||||
"keys.budgetOf": "presupuesto {b}/mes",
|
||||
"dash.kpi.orBalance": "Saldo OpenRouter",
|
||||
"keys.h.install": "Instalar:",
|
||||
"keys.snip.keyInputPh": "pega tu clave API",
|
||||
"keys.snip.apply": "Insertar clave",
|
||||
};
|
||||
|
||||
Vendored
+17
-2
@@ -536,9 +536,13 @@ export default {
|
||||
"keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
|
||||
"keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
|
||||
"keys.snip.keyPh": "votre clé API",
|
||||
"settings.restart": "Redémarrer la passerelle",
|
||||
"settings.restartConfirm": "Redémarrer la passerelle ? Les requêtes en cours seront coupées ; les serveurs de modèles locaux redémarrent aussi.",
|
||||
"settings.restart": "Tout redémarrer",
|
||||
"settings.restartConfirm": "Redémarrer la passerelle et tous les serveurs de modèles locaux ? Les requêtes en cours seront interrompues.",
|
||||
"settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
|
||||
"settings.shutdown": "Tout arrêter et éteindre",
|
||||
"settings.shutdownConfirm": "Arrêter tous les serveurs de modèles locaux et éteindre la passerelle ? L'API et ce panneau ne répondront plus tant que vous n'aurez pas relancé la passerelle.",
|
||||
"settings.shutdownDone": "La passerelle est éteinte.",
|
||||
"settings.shutdownHint": "Tous les serveurs de modèles locaux sont arrêtés et le processus est terminé. Relancez la passerelle sur le serveur pour retrouver le panneau.",
|
||||
"bench.title": "Vitesse des modèles locaux",
|
||||
"bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
|
||||
"bench.run": "Mesurer",
|
||||
@@ -635,4 +639,15 @@ export default {
|
||||
"tier.medium": "moyen",
|
||||
"tier.high": "difficile",
|
||||
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
|
||||
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
|
||||
"settings.cache.entries": "entrées",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "économisé",
|
||||
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
|
||||
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
|
||||
"keys.budgetOf": "budget {b}/mois",
|
||||
"dash.kpi.orBalance": "Solde OpenRouter",
|
||||
"keys.h.install": "Installer :",
|
||||
"keys.snip.keyInputPh": "collez votre clé API",
|
||||
"keys.snip.apply": "Insérer la clé",
|
||||
};
|
||||
|
||||
Vendored
+17
-2
@@ -528,9 +528,13 @@ export default {
|
||||
"keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
|
||||
"keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
|
||||
"keys.snip.keyPh": "ваш API-ключ",
|
||||
"settings.restart": "Перезапустить шлюз",
|
||||
"settings.restartConfirm": "Перезапустить шлюз? Текущие запросы оборвутся; серверы локальных моделей тоже перезапустятся.",
|
||||
"settings.restart": "Перезапустить всё",
|
||||
"settings.restartConfirm": "Перезапустить шлюз и все серверы локальных моделей? Текущие запросы оборвутся.",
|
||||
"settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
|
||||
"settings.shutdown": "Остановить всё и выключить",
|
||||
"settings.shutdownConfirm": "Остановить все серверы локальных моделей и выключить шлюз? API и эта панель перестанут отвечать, пока вы не запустите шлюз заново.",
|
||||
"settings.shutdownDone": "Шлюз выключен.",
|
||||
"settings.shutdownHint": "Все серверы локальных моделей остановлены, процесс завершён. Запустите шлюз на сервере заново, чтобы вернуть панель.",
|
||||
"bench.title": "Скорость локальных моделей",
|
||||
"bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
|
||||
"bench.run": "Замерить",
|
||||
@@ -627,4 +631,15 @@ export default {
|
||||
"tier.medium": "средний",
|
||||
"tier.high": "сложный",
|
||||
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
|
||||
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
|
||||
"settings.cache.entries": "записей",
|
||||
"settings.cache.hits": "попаданий",
|
||||
"settings.cache.saved": "сэкономлено",
|
||||
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
|
||||
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
|
||||
"keys.budgetOf": "бюджет {b}/мес",
|
||||
"dash.kpi.orBalance": "Баланс OpenRouter",
|
||||
"keys.h.install": "Установить:",
|
||||
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
|
||||
"keys.snip.apply": "Вставить ключ",
|
||||
};
|
||||
|
||||
Vendored
+17
-2
@@ -536,9 +536,13 @@ export default {
|
||||
"keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
|
||||
"keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
|
||||
"keys.snip.keyPh": "API anahtarın",
|
||||
"settings.restart": "Ağ geçidini yeniden başlat",
|
||||
"settings.restartConfirm": "Ağ geçidi yeniden başlatılsın mı? Devam eden istekler kesilir; yerel model sunucuları da yeniden başlar.",
|
||||
"settings.restart": "Tümünü yeniden başlat",
|
||||
"settings.restartConfirm": "Ağ geçidi ve tüm yerel model sunucuları yeniden başlatılsın mı? Devam eden istekler kesilecek.",
|
||||
"settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
|
||||
"settings.shutdown": "Tümünü durdur ve kapat",
|
||||
"settings.shutdownConfirm": "Tüm yerel model sunucuları durdurulup ağ geçidi kapatılsın mı? Ağ geçidini yeniden başlatana kadar API ve bu panel yanıt vermez.",
|
||||
"settings.shutdownDone": "Ağ geçidi kapalı.",
|
||||
"settings.shutdownHint": "Tüm yerel model sunucuları durduruldu ve süreç sonlandı. Paneli geri getirmek için ağ geçidini sunucuda yeniden başlatın.",
|
||||
"bench.title": "Yerel model hızı",
|
||||
"bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
|
||||
"bench.run": "Ölç",
|
||||
@@ -635,4 +639,15 @@ export default {
|
||||
"tier.medium": "orta",
|
||||
"tier.high": "zor",
|
||||
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
|
||||
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
|
||||
"settings.cache.entries": "kayıt",
|
||||
"settings.cache.hits": "isabet",
|
||||
"settings.cache.saved": "tasarruf",
|
||||
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
|
||||
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
|
||||
"keys.budgetOf": "bütçe {b}/ay",
|
||||
"dash.kpi.orBalance": "OpenRouter bakiyesi",
|
||||
"keys.h.install": "Kurulum:",
|
||||
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
|
||||
"keys.snip.apply": "Anahtarı ekle",
|
||||
};
|
||||
|
||||
Vendored
+17
-2
@@ -536,9 +536,13 @@ export default {
|
||||
"keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
|
||||
"keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
|
||||
"keys.snip.keyPh": "你的 API 密钥",
|
||||
"settings.restart": "重启网关",
|
||||
"settings.restartConfirm": "重启网关?进行中的请求会中断;本地模型服务器也会重启。",
|
||||
"settings.restart": "全部重启",
|
||||
"settings.restartConfirm": "重启网关和所有本地模型服务器?正在处理的请求将被中断。",
|
||||
"settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
|
||||
"settings.shutdown": "全部停止并关机",
|
||||
"settings.shutdownConfirm": "停止所有本地模型服务器并关闭网关?在你重新启动网关之前,API 和此面板都将无法响应。",
|
||||
"settings.shutdownDone": "网关已关闭。",
|
||||
"settings.shutdownHint": "所有本地模型服务器已停止,进程已退出。请在服务器上重新启动网关以恢复面板。",
|
||||
"bench.title": "本地模型速度",
|
||||
"bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s,一次一个模型。",
|
||||
"bench.run": "测速",
|
||||
@@ -635,4 +639,15 @@ export default {
|
||||
"tier.medium": "中等",
|
||||
"tier.high": "困难",
|
||||
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
|
||||
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
|
||||
"settings.cache.entries": "条目",
|
||||
"settings.cache.hits": "命中",
|
||||
"settings.cache.saved": "已节省",
|
||||
"keys.form.budget": "每月预算,$(0 = 不限)",
|
||||
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
|
||||
"keys.budgetOf": "预算 {b}/月",
|
||||
"dash.kpi.orBalance": "OpenRouter 余额",
|
||||
"keys.h.install": "安装:",
|
||||
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
|
||||
"keys.snip.apply": "插入密钥",
|
||||
};
|
||||
|
||||
Vendored
+8
-2
@@ -110,12 +110,13 @@ export async function renderDashboard() {
|
||||
|
||||
async function load() {
|
||||
const rangeSecs = range;
|
||||
let stats, recent, health;
|
||||
let stats, recent, health, balance;
|
||||
try {
|
||||
[stats, recent, health] = await Promise.all([
|
||||
[stats, recent, health, balance] = await Promise.all([
|
||||
api.stats(rangeSecs, groupby),
|
||||
api.requests(40, 0),
|
||||
api.health().catch(() => null),
|
||||
api.balance().catch(() => null),
|
||||
]);
|
||||
} catch (e) {
|
||||
mount(root, h("div", { class: "callout" }, String(e.message || e)));
|
||||
@@ -162,6 +163,11 @@ export async function renderDashboard() {
|
||||
kpi(num(tot.requests), t("dash.kpi.requests")),
|
||||
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
|
||||
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
|
||||
balance && balance.openrouter_usd != null
|
||||
? kpi(
|
||||
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
|
||||
t("dash.kpi.orBalance"))
|
||||
: null,
|
||||
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
|
||||
kpi(pct(tot.success_rate), t("dash.kpi.success")),
|
||||
kpi(num(tot.failovers), t("dash.kpi.failovers"))
|
||||
|
||||
Vendored
+147
-14
@@ -10,12 +10,14 @@ export async function renderKeys() {
|
||||
function openCreate(reload) {
|
||||
const accountIn = h("input", { placeholder: "team-a" });
|
||||
const rateIn = h("input", { type: "number", value: 600 });
|
||||
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
|
||||
const customIn = h("input", { placeholder: "sk-gw-…" });
|
||||
const body = h(
|
||||
"div",
|
||||
{},
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
|
||||
);
|
||||
modal(t("keys.form.title"), body, async () => {
|
||||
@@ -24,6 +26,7 @@ export async function renderKeys() {
|
||||
key: customIn.value.trim(),
|
||||
account: accountIn.value.trim() || "default",
|
||||
rate_per_min: parseInt(rateIn.value) || 0,
|
||||
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
|
||||
allow_models: [],
|
||||
});
|
||||
reload();
|
||||
@@ -51,18 +54,76 @@ export async function renderKeys() {
|
||||
function snippetsCard(openMode) {
|
||||
const origin = location.origin;
|
||||
const base = origin + "/v1";
|
||||
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
// Every scripty snippet exists in two OS flavors: bash heredocs for
|
||||
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
|
||||
// (python/js) and GUI instructions are OS-independent.
|
||||
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
|
||||
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
|
||||
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
|
||||
const snips = {
|
||||
"Claude Code": `# ${t("keys.h.claude")}
|
||||
export ANTHROPIC_BASE_URL="${origin}"
|
||||
const buildSnips = (osKind) => osKind === "ps" ? {
|
||||
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
|
||||
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
|
||||
$env:ANTHROPIC_MODEL = "cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
|
||||
Add-Content "$HOME\\.codex\\config.toml" @'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
[model_providers.cortiq]
|
||||
name = "Cortiq Gateway"
|
||||
base_url = "${base}"
|
||||
env_key = "CORTIQ_API_KEY"
|
||||
wire_api = "chat"
|
||||
'@
|
||||
$env:CORTIQ_API_KEY = "${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
|
||||
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"cortiq": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Cortiq Gateway",
|
||||
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
opencode`,
|
||||
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
|
||||
Add-Content "$HOME\\.continue\\config.yaml" @'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
model: cortiq-auto
|
||||
apiBase: ${base}
|
||||
apiKey: ${key}
|
||||
'@`,
|
||||
Cline: `${t("keys.h.cline1")}
|
||||
API Provider: OpenAI Compatible
|
||||
Base URL: ${base}
|
||||
API Key: ${key}
|
||||
Model ID: cortiq-auto
|
||||
|
||||
${t("keys.h.cline2")}`,
|
||||
curl: `curl.exe ${base}/chat/completions \`
|
||||
-H "Authorization: Bearer ${key}" \`
|
||||
-H "Content-Type: application/json" \`
|
||||
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||
} : {
|
||||
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
|
||||
export ANTHROPIC_AUTH_TOKEN="${key}"
|
||||
export ANTHROPIC_MODEL="cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `# ${t("keys.h.codex")}
|
||||
mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
@@ -75,8 +136,7 @@ EOF
|
||||
export CORTIQ_API_KEY="${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `# ${t("keys.h.opencode")}
|
||||
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
@@ -91,8 +151,7 @@ mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
EOF
|
||||
|
||||
opencode`,
|
||||
Continue: `# ${t("keys.h.continue")}
|
||||
mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
@@ -128,8 +187,29 @@ const resp = await client.chat.completions.create({
|
||||
});
|
||||
console.log(resp.choices[0].message.content);`,
|
||||
};
|
||||
let shSnips, psSnips, snips;
|
||||
function rebuildSnips() {
|
||||
shSnips = buildSnips("sh");
|
||||
psSnips = buildSnips("ps");
|
||||
// python/js/Cline одинаковы — дополняем ps-набор из sh
|
||||
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
|
||||
snips = os === "ps" ? psSnips : shSnips;
|
||||
}
|
||||
rebuildSnips();
|
||||
let cur = "Claude Code";
|
||||
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
|
||||
// in a pasted snippet is not a comment in default interactive zsh, and a
|
||||
// "!" inside it triggers history expansion ("event not found")
|
||||
const HINTS = {
|
||||
"Claude Code": () => t("keys.h.claude"),
|
||||
Codex: () => t("keys.h.codex"),
|
||||
OpenCode: () => t("keys.h.opencode"),
|
||||
Continue: () => t("keys.h.continue"),
|
||||
};
|
||||
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
|
||||
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
|
||||
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
|
||||
paintHint();
|
||||
const seg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
@@ -137,6 +217,7 @@ console.log(resp.choices[0].message.content);`,
|
||||
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
|
||||
cur = k;
|
||||
pre.textContent = snips[k];
|
||||
paintHint();
|
||||
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
|
||||
paintStatus();
|
||||
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
|
||||
@@ -164,7 +245,15 @@ console.log(resp.choices[0].message.content);`,
|
||||
}).catch(() => {});
|
||||
|
||||
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
|
||||
// status line under the snippet ("installed" / "already configured").
|
||||
// status line under the snippet ("installed" / "already configured") —
|
||||
// and a one-line INSTALL command (per OS) when it's missing.
|
||||
const INSTALL = {
|
||||
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
|
||||
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
|
||||
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
|
||||
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
|
||||
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
|
||||
};
|
||||
let detected = {};
|
||||
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
|
||||
function paintStatus() {
|
||||
@@ -176,7 +265,19 @@ console.log(resp.choices[0].message.content);`,
|
||||
statusHost.innerHTML = "";
|
||||
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
|
||||
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
|
||||
else mount(statusHost, h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")));
|
||||
else {
|
||||
const cmd = (INSTALL[cur] || {})[os];
|
||||
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
|
||||
await navigator.clipboard.writeText(cmd);
|
||||
toast(t("common.copied"), "good");
|
||||
} }, "⧉");
|
||||
mount(statusHost,
|
||||
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
|
||||
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
|
||||
h("span", { class: "muted" }, t("keys.h.install")),
|
||||
h("code", { class: "mono small cmd-chip" }, cmd),
|
||||
copyInstall) : null);
|
||||
}
|
||||
}
|
||||
api.harnesses().then((d) => {
|
||||
detected = d || {};
|
||||
@@ -187,6 +288,32 @@ console.log(resp.choices[0].message.content);`,
|
||||
paintStatus();
|
||||
}).catch(() => {});
|
||||
|
||||
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
|
||||
const osSeg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
|
||||
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
|
||||
);
|
||||
osSeg.querySelectorAll("button").forEach((b, i) =>
|
||||
b.addEventListener("click", () => {
|
||||
os = i === 0 ? "sh" : "ps";
|
||||
rebuildSnips();
|
||||
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
|
||||
pre.textContent = snips[cur];
|
||||
paintStatus();
|
||||
}));
|
||||
|
||||
// paste-your-key: substitutes the key into every snippet immediately
|
||||
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
|
||||
const applyKey = () => {
|
||||
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
rebuildSnips();
|
||||
pre.textContent = snips[cur];
|
||||
};
|
||||
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
|
||||
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
|
||||
|
||||
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||
copy.addEventListener("click", async () => {
|
||||
await navigator.clipboard.writeText(pre.textContent);
|
||||
@@ -198,7 +325,8 @@ console.log(resp.choices[0].message.content);`,
|
||||
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" },
|
||||
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), copy),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
|
||||
hintHost,
|
||||
pre,
|
||||
statusHost,
|
||||
protoWarn,
|
||||
@@ -273,7 +401,12 @@ console.log(resp.choices[0].message.content);`,
|
||||
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
|
||||
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
|
||||
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
|
||||
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
|
||||
h("td", { class: "right-align" },
|
||||
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
|
||||
k.monthly_budget_usd
|
||||
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
|
||||
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
|
||||
: null),
|
||||
h(
|
||||
"td",
|
||||
{ class: "right-align" },
|
||||
|
||||
Vendored
+51
-2
@@ -194,8 +194,30 @@ export async function renderSettings() {
|
||||
const modelsData = await api.listModels();
|
||||
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
|
||||
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
|
||||
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
|
||||
// Live proof-of-life for the cache card: entries/hits/saved — and an
|
||||
// honest warning when the cache is on but has no embedding model to work
|
||||
// with (it silently does nothing in that state).
|
||||
let cacheStats = null;
|
||||
try {
|
||||
cacheStats = (await api.stats("24h", "model")).cache || null;
|
||||
} catch { /* stats off */ }
|
||||
function cacheStatus() {
|
||||
const wrap = h("div", { style: "margin-bottom:10px" });
|
||||
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
|
||||
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
|
||||
return wrap;
|
||||
}
|
||||
if (cacheStats && s.cache?.enabled) {
|
||||
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
|
||||
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
|
||||
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
|
||||
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
|
||||
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
|
||||
}
|
||||
return wrap;
|
||||
}
|
||||
const cacheEmbedSel = h(
|
||||
"select",
|
||||
{},
|
||||
@@ -537,6 +559,31 @@ export async function renderSettings() {
|
||||
}
|
||||
});
|
||||
|
||||
const stopBtn = h("button", { class: "btn danger" }, "\u23fb " + t("settings.shutdown"));
|
||||
stopBtn.addEventListener("click", async () => {
|
||||
if (!(await confirmDialog(t("settings.shutdownConfirm")))) return;
|
||||
stopBtn.disabled = true;
|
||||
restartBtn.disabled = true;
|
||||
try {
|
||||
await api.shutdown();
|
||||
// the server is gone in ~300ms — replace the app with a static farewell
|
||||
document.body.innerHTML = "";
|
||||
document.body.appendChild(
|
||||
h(
|
||||
"div",
|
||||
{ class: "shutdown-screen" },
|
||||
h("div", { class: "shutdown-icon" }, "\u23fb"),
|
||||
h("h2", {}, t("settings.shutdownDone")),
|
||||
h("p", { class: "hint" }, t("settings.shutdownHint"))
|
||||
)
|
||||
);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
stopBtn.disabled = false;
|
||||
restartBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
mount(
|
||||
root,
|
||||
h(
|
||||
@@ -610,6 +657,7 @@ export async function renderSettings() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
|
||||
cacheStatus(),
|
||||
cacheEnabled.node,
|
||||
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
|
||||
field(t("settings.cache.embed"), cacheEmbedSel),
|
||||
@@ -659,7 +707,8 @@ export async function renderSettings() {
|
||||
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
|
||||
fileIn,
|
||||
h("div", { class: "grow" }),
|
||||
restartBtn
|
||||
restartBtn,
|
||||
stopBtn
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user