diff --git a/README.md b/README.md index 66881a3..1422adf 100644 --- a/README.md +++ b/README.md @@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY` and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)). +Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md). + --- ## 🖥️ Admin console diff --git a/README.ru.md b/README.ru.md index 3761ed5..03527a2 100644 --- a/README.ru.md +++ b/README.ru.md @@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте `CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)). +Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md). + --- ## 🖥️ Панель управления diff --git a/docs/DEPLOY.md b/docs/DEPLOY.md new file mode 100644 index 0000000..88de01e --- /dev/null +++ b/docs/DEPLOY.md @@ -0,0 +1,122 @@ +# Deploying cortiq-gateway for a team / office + +One gateway on a small server, API keys per department, local models for the +cheap tiers, cloud models for the hard ones. This page covers the ops side. + +## 1. Run as a service + +### Linux (systemd) + +```ini +# /etc/systemd/system/cortiq-gateway.service +[Unit] +Description=Cortiq Gateway +After=network-online.target + +[Service] +User=cortiq +Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway +ExecStart=/usr/local/bin/cortiq-gateway --no-port-search +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target +``` + +```bash +sudo systemctl enable --now cortiq-gateway +``` + +### macOS (launchd) + +```xml + + + + + Labelcom.cortiq.gateway + ProgramArguments + /usr/local/bin/cortiq-gateway + --no-port-search + + RunAtLoad + KeepAlive + +``` + +```bash +launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist +``` + +### Windows + +Task Scheduler → Create Task → "Run whether user is logged on or not", +trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use +[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`. + +### Docker + +```bash +docker run -d --restart unless-stopped -p 9000:9000 \ + -v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest +``` + +## 2. TLS and the outside world + +The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front +and bind the gateway to localhost: + +```toml +# gateway.toml +listen = "127.0.0.1:9000" +``` + +```nginx +server { + listen 443 ssl; + server_name ai.example.com; + ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem; + + location / { + proxy_pass http://127.0.0.1:9000; + proxy_http_version 1.1; + proxy_buffering off; # SSE streaming + proxy_read_timeout 3600s; # long generations + } +} +``` + +(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the +same with automatic certificates.) + +## 3. Security checklist + +- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it + the console warns that it is open to the network. +- **API keys**: create one per department/developer on the keys page. With at + least one key configured the model APIs require Bearer / x-api-key — + requests without a valid key get 401. +- **Rate limits**: per-key requests/min → 429 with Retry-After. +- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the + account's month-to-date spend crosses it. +- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw. + +## 4. Backups + +Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`): + +- `gateway.toml` — full config incl. keys (or use the console's TOML export); +- `secrets.toml` — provider API keys (chmod 600); +- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl. + cumulative per-key totals and month-to-date budget spend. + +Copy those three and you can rebuild the instance anywhere. + +## 5. Observability + +- Dashboard: KPIs, per-key spend, recent requests with complexity scores, + OpenRouter balance (red under $5). +- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus. +- Logs view in the console; `stats.max_file_mb` keeps the disk bounded. diff --git a/docs/DEPLOY.ru.md b/docs/DEPLOY.ru.md new file mode 100644 index 0000000..0008288 --- /dev/null +++ b/docs/DEPLOY.ru.md @@ -0,0 +1,123 @@ +# Развёртывание cortiq-gateway для команды / офиса + +Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для +дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию. + +## 1. Запуск как сервис + +### Linux (systemd) + +```ini +# /etc/systemd/system/cortiq-gateway.service +[Unit] +Description=Cortiq Gateway +After=network-online.target + +[Service] +User=cortiq +Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway +ExecStart=/usr/local/bin/cortiq-gateway --no-port-search +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target +``` + +```bash +sudo systemctl enable --now cortiq-gateway +``` + +### macOS (launchd) + +```xml + + + + + Labelcom.cortiq.gateway + ProgramArguments + /usr/local/bin/cortiq-gateway + --no-port-search + + RunAtLoad + KeepAlive + +``` + +```bash +launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist +``` + +### Windows + +Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа +пользователя», триггер «При запуске», действие `cortiq-gateway.exe +--no-port-search`. Либо [NSSM](https://nssm.cc): +`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`. + +### Docker + +```bash +docker run -d --restart unless-stopped -p 9000:9000 \ + -v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest +``` + +## 2. TLS и внешний мир + +Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз +привяжите к localhost: + +```toml +# gateway.toml +listen = "127.0.0.1:9000" +``` + +```nginx +server { + listen 443 ssl; + server_name ai.example.com; + ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem; + + location / { + proxy_pass http://127.0.0.1:9000; + proxy_http_version 1.1; + proxy_buffering off; # SSE-стриминг + proxy_read_timeout 3600s; # долгие генерации + } +} +``` + +(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же +с автоматическими сертификатами.) + +## 3. Чек-лист безопасности + +- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без + него консоль предупреждает, что открыта сети. +- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только + настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key — + запросы без валидного ключа получают 401. +- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After. +- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда + траты аккаунта за месяц превысят лимит. +- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой. + +## 4. Бэкапы + +Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`): + +- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли); +- `secrets.toml` — ключи провайдеров (chmod 600); +- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные + тоталы по ключам и месячные траты для бюджетов. + +Скопируйте эти три файла — и инстанс поднимается где угодно. + +## 5. Наблюдаемость + +- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами + сложности, баланс OpenRouter (красный ниже $5). +- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus. +- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск. diff --git a/src/admin/mod.rs b/src/admin/mod.rs index f4d8b22..96bcb2d 100644 --- a/src/admin/mod.rs +++ b/src/admin/mod.rs @@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router { get(get_protocols).put(put_protocols), ) .route("/admin/api/harnesses", get(get_harnesses)) + .route("/admin/api/balance", get(get_balance)) .route("/admin/api/settings", get(get_settings).put(put_settings)) .route("/admin/api/keys", get(list_keys).post(create_key)) .route("/admin/api/keys/:key", axum::routing::delete(delete_key)) @@ -1008,6 +1009,51 @@ async fn put_routing(State(state): State, Json(body): Json) -> ApiResult { + static CACHE: std::sync::OnceLock>> = + std::sync::OnceLock::new(); + let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None)); + if let Some((at, v)) = *cache.lock().unwrap() { + if at.elapsed() < std::time::Duration::from_secs(300) { + return ok(json!({ "openrouter_usd": v })); + } + } + let key = state + .secrets + .resolve("OPENROUTER_API_KEY") + .or_else(|| std::env::var("OPENROUTER_API_KEY").ok()); + let Some(key) = key.filter(|k| !k.is_empty()) else { + return ok(json!({ "openrouter_usd": serde_json::Value::Null })); + }; + let client = reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(8)) + .build() + .unwrap_or_default(); + let resp = client + .get("https://openrouter.ai/api/v1/credits") + .bearer_auth(&key) + .send() + .await; + let balance = match resp { + Ok(r) if r.status().is_success() => { + let v: serde_json::Value = r.json().await.unwrap_or_default(); + let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0); + let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0); + Some(total - used) + } + _ => None, + }; + let Some(balance) = balance else { + return ok(json!({ "openrouter_usd": serde_json::Value::Null })); + }; + *cache.lock().unwrap() = Some((std::time::Instant::now(), balance)); + ok(json!({ "openrouter_usd": balance })) +} + // ──────────────────────────── harness detection ──────────────────────────── // Which coding harnesses are present on the machine the gateway runs on, so the // API-keys view can mark tabs "installed" / "already configured for cortiq". @@ -1357,6 +1403,7 @@ async fn list_keys(State(state): State) -> ApiResult { "key_masked": mask_key(&k.key), "account": k.account, "rate_per_min": k.rate_per_min, + "monthly_budget_usd": k.monthly_budget_usd, "allow_models": k.allow_models, }) }) @@ -1372,6 +1419,8 @@ struct NewKey { #[serde(default)] rate_per_min: u32, #[serde(default)] + monthly_budget_usd: Option, + #[serde(default)] allow_models: Vec, } @@ -1390,6 +1439,7 @@ async fn create_key(State(state): State, Json(b): Json) -> account: b.account, rate_per_min: b.rate_per_min, allow_models: b.allow_models, + monthly_budget_usd: b.monthly_budget_usd, }); state.reload(cfg)?; // show the full key only once diff --git a/src/config.rs b/src/config.rs index 4151578..9aff363 100644 --- a/src/config.rs +++ b/src/config.rs @@ -620,6 +620,9 @@ pub struct ApiKeyCfg { pub account: String, #[serde(default)] pub rate_per_min: u32, + /// Monthly spend ceiling in USD for this key's account; 0/absent = none. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub monthly_budget_usd: Option, #[serde(default, skip_serializing_if = "Vec::is_empty")] pub allow_models: Vec, } diff --git a/src/error.rs b/src/error.rs index 5a11244..c954497 100644 --- a/src/error.rs +++ b/src/error.rs @@ -13,6 +13,8 @@ pub enum GatewayError { Unauthorized(String), #[error("rate limit exceeded")] RateLimited { retry_after_secs: u64 }, + #[error("monthly budget exhausted (limit ${budget_usd})")] + BudgetExhausted { budget_usd: f64 }, #[error("upstream unavailable: {0}")] UpstreamUnavailable(String), #[error("internal error: {0}")] @@ -37,6 +39,11 @@ impl GatewayError { "rate_limit_error", "rate_limited", ), + Self::BudgetExhausted { .. } => ( + StatusCode::TOO_MANY_REQUESTS, + "insufficient_quota", + "insufficient_quota", + ), Self::UpstreamUnavailable(_) => ( StatusCode::BAD_GATEWAY, "upstream_unavailable", diff --git a/src/main.rs b/src/main.rs index 9e52e0e..142b4aa 100644 --- a/src/main.rs +++ b/src/main.rs @@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> { .route("/health", get(|| async { "ok" })) .route("/readyz", get(|| async { "ready" })) .route("/metrics", get(admin::metrics)) - .merge(protocols::build_router()); + .merge(protocols::build_router(state.clone())); if admin_enabled { app = app @@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> { .allow_methods(tower_http::cors::Any) .allow_headers(tower_http::cors::Any); - let app = app.with_state(state).layer(cors); + // 32 MB request ceiling: enough for base64 keyframes and TOML imports, + // small enough that a hostile client can't balloon memory + let app = + app.with_state(state) + .layer(cors) + .layer(tower_http::limit::RequestBodyLimitLayer::new( + 32 * 1024 * 1024, + )); let listener = { let mut attempt = 0u32; @@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> { tracing::info!("open this URL to finish setup: {url}"); } } - axum::serve(listener, app).await?; + axum::serve(listener, app) + .with_graceful_shutdown(async { + let _ = tokio::signal::ctrl_c().await; + tracing::info!("shutdown signal received — draining connections"); + }) + .await?; Ok(()) } diff --git a/src/protocols/anthropic_messages.rs b/src/protocols/anthropic_messages.rs index 3f6e3d4..332cf60 100644 --- a/src/protocols/anthropic_messages.rs +++ b/src/protocols/anthropic_messages.rs @@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective { async fn handler( State(state): State, + acct: Option>, Json(req): Json, ) -> Result { if !state.live().cfg.protocols.anthropic_messages { @@ -108,6 +109,7 @@ async fn handler( stream: req.stream, meta: RequestMeta { protocol: "anthropic_messages".into(), + account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(), ..Default::default() }, }; diff --git a/src/protocols/mod.rs b/src/protocols/mod.rs index 4d7193c..55c7ea1 100644 --- a/src/protocols/mod.rs +++ b/src/protocols/mod.rs @@ -12,10 +12,59 @@ pub mod openai_models; use crate::state::SharedState; use axum::Router; +/// Account resolved by the API-key middleware, consumed by protocol handlers +/// for attribution (stats, budgets). Present only when keys are configured. +#[derive(Clone)] +pub struct AccountTag(pub String); + +/// API-key gate for the model-facing surface. Open mode (no keys configured) +/// passes everything through — the admin console already warns about it. +/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is +/// enforced with a fixed one-minute window (429 + Retry-After), and a +/// monthly budget, when set, turns into 429 insufficient_quota once the +/// account's month-to-date spend crosses it. +async fn api_auth( + axum::extract::State(state): axum::extract::State, + mut req: axum::extract::Request, + next: axum::middleware::Next, +) -> axum::response::Response { + use axum::response::IntoResponse; + let live = state.live(); + if live.cfg.api_keys.is_empty() { + return next.run(req).await; + } + let provided = req + .headers() + .get(axum::http::header::AUTHORIZATION) + .and_then(|v| v.to_str().ok()) + .and_then(|v| v.strip_prefix("Bearer ")) + .or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok())) + .map(str::trim) + .unwrap_or(""); + let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else { + return crate::error::GatewayError::Unauthorized("invalid or missing API key".into()) + .into_response(); + }; + if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) { + return crate::error::GatewayError::RateLimited { + retry_after_secs: state.rate.retry_after_secs(), + } + .into_response(); + } + if let Some(budget) = key.monthly_budget_usd { + if budget > 0.0 && state.stats.month_spend(&key.account) >= budget { + return crate::error::GatewayError::BudgetExhausted { budget_usd: budget } + .into_response(); + } + } + req.extensions_mut().insert(AccountTag(key.account.clone())); + next.run(req).await +} + /// Build the protocols router. Implemented adapters are **always** mounted; /// each handler checks the live `protocols.*` flag from config itself, so /// toggles in the admin panel take effect without a restart (hot switching). -pub fn build_router() -> Router { +pub fn build_router(state: SharedState) -> Router { Router::new() .merge(openai_chat::routes()) .merge(openai_completions::routes()) @@ -24,4 +73,5 @@ pub fn build_router() -> Router { .merge(anthropic_messages::routes()) .merge(mcp::routes()) .merge(native_passthrough::routes()) + .route_layer(axum::middleware::from_fn_with_state(state, api_auth)) } diff --git a/src/protocols/openai_chat.rs b/src/protocols/openai_chat.rs index d17592e..7f1c1a8 100644 --- a/src/protocols/openai_chat.rs +++ b/src/protocols/openai_chat.rs @@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective { async fn handler( State(state): State, + acct: Option>, Json(req): Json, ) -> Result { // hot protocol toggle: if the adapter is disabled in config — return 404 @@ -104,6 +105,7 @@ async fn handler( stream: req.stream, meta: RequestMeta { protocol: "openai_chat".into(), + account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(), ..Default::default() }, }; diff --git a/src/protocols/openai_completions.rs b/src/protocols/openai_completions.rs index df47d7d..a180609 100644 --- a/src/protocols/openai_completions.rs +++ b/src/protocols/openai_completions.rs @@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String { async fn handler( State(state): State, + acct: Option>, Json(req): Json, ) -> Result { if !state.live().cfg.protocols.openai_completions { @@ -89,6 +90,7 @@ async fn handler( stream: req.stream, meta: RequestMeta { protocol: "openai_completions".into(), + account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(), ..Default::default() }, }; diff --git a/src/state.rs b/src/state.rs index 8ebf558..5718ac0 100644 --- a/src/state.rs +++ b/src/state.rs @@ -66,6 +66,42 @@ impl Live { } } +/// Fixed-window per-key rate limiter (minute buckets). Small and honest: +/// at minute boundaries a burst can briefly see up to 2× the limit, which is +/// fine for office-grade fairness and costs one HashMap lookup per request. +#[derive(Default)] +pub struct RateWindows { + inner: std::sync::Mutex>, +} + +impl RateWindows { + pub fn allow(&self, key: &str, per_min: u32) -> bool { + let minute = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap_or_default() + .as_secs() + / 60; + let mut g = self.inner.lock().unwrap(); + let e = g.entry(key.to_string()).or_insert((minute, 0)); + if e.0 != minute { + *e = (minute, 0); + } + if e.1 >= per_min { + return false; + } + e.1 += 1; + true + } + /// Seconds until the current minute window rolls over. + pub fn retry_after_secs(&self) -> u64 { + 60 - (std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap_or_default() + .as_secs() + % 60) + } +} + pub struct AppState { pub live: ArcSwap, pub stats: Arc, @@ -85,6 +121,8 @@ pub struct AppState { /// Managed local CMF model server (install/update/spawn lifecycle + status). pub cmf: Arc, pub cache: Arc, + /// Per-API-key fixed-window rate limiter. + pub rate: RateWindows, pub secrets: SecretStore, pub config_path: String, pub pipeline: crate::pipeline::Pipeline, @@ -164,6 +202,7 @@ impl SharedState { admin_open_public: std::sync::atomic::AtomicBool::new(false), cmf, cache, + rate: RateWindows::default(), secrets, config_path, pipeline: crate::pipeline::Pipeline::new(), diff --git a/src/stats.rs b/src/stats.rs index fc14106..f6382af 100644 --- a/src/stats.rs +++ b/src/stats.rs @@ -137,6 +137,11 @@ struct Inner { /// = folded + by_account_alltime, with no double counting: a record is /// either in the file or folded, never both. folded: HashMap, + /// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this. + month_spend: HashMap, + /// Records with ts ≤ this are already reflected in the persisted + /// month_spend (watermark against double counting on replay). + month_upto_ts: u64, /// Appends since the last compaction size check. writes_since_check: u32, } @@ -193,6 +198,7 @@ impl Stats { .entry(rec.account.clone()) .or_default() .apply(&rec); + Self::apply_month(&mut inner, &rec); if rec.ts >= cutoff { Self::apply_aggregates(&mut inner, &rec, self.retention_secs); } @@ -200,6 +206,33 @@ impl Stats { } } + fn apply_month(inner: &mut Inner, rec: &RequestRecord) { + if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 { + return; + } + let m = month_of(rec.ts); + let e = inner + .month_spend + .entry(rec.account.clone()) + .or_insert((m.clone(), 0.0)); + if e.0 != m { + *e = (m, 0.0); + } + e.1 += rec.cost_usd; + } + + /// Month-to-date spend for an account (budget checks). + pub fn month_spend(&self, account: &str) -> f64 { + let inner = self.inner.lock().unwrap(); + let m = month_of(now_secs()); + inner + .month_spend + .get(account) + .filter(|(mm, _)| *mm == m) + .map(|(_, v)| *v) + .unwrap_or(0.0) + } + fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) { inner.recent.push_back(rec.clone()); while inner.recent.len() > ring_size { @@ -256,6 +289,7 @@ impl Stats { .entry(rec.account.clone()) .or_default() .apply(&rec); + Self::apply_month(&mut inner, &rec); inner.writes_since_check += 1; } if let Some(path) = &self.file { @@ -283,11 +317,42 @@ impl Stats { let Ok(content) = std::fs::read_to_string(&path) else { return; }; - if let Ok(m) = serde_json::from_str::>(&content) { + #[derive(serde::Deserialize)] + struct Sidecar { + #[serde(default)] + by_account: HashMap, + #[serde(default)] + month_spend: HashMap, + #[serde(default)] + month_upto_ts: u64, + } + // new format {by_account, month_spend, month_upto_ts}; the original + // file held the bare by_account map — accept both + if let Ok(sc) = serde_json::from_str::(&content) { + let mut inner = self.inner.lock().unwrap(); + inner.folded = sc.by_account; + inner.month_spend = sc.month_spend; + inner.month_upto_ts = sc.month_upto_ts; + } else if let Ok(m) = serde_json::from_str::>(&content) { self.inner.lock().unwrap().folded = m; } } + fn write_sidecar(&self, inner: &Inner) { + let Some(tpath) = self.totals_path() else { + return; + }; + let payload = serde_json::json!({ + "by_account": inner.folded, + "month_spend": inner.month_spend, + "month_upto_ts": now_secs(), + }); + let tmp = format!("{tpath}.tmp"); + if std::fs::write(&tmp, payload.to_string()).is_ok() { + let _ = std::fs::rename(&tmp, &tpath); + } + } + /// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records /// older than the retention window fold into the cumulative per-account /// sidecar and are dropped from the file (tmp + rename on both files, so @@ -305,6 +370,10 @@ impl Stats { return; } inner.writes_since_check = 0; + // piggyback: persist month-to-date spend so budgets survive + // restarts even between compactions + inner.month_upto_ts = now_secs(); + self.write_sidecar(&inner); } } let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0); @@ -359,14 +428,8 @@ impl Stats { } } // persist the folded totals FIRST, then swap the trimmed file in - if let Some(tpath) = self.totals_path() { - if let Ok(json) = serde_json::to_string(&inner.folded) { - let tmp = format!("{tpath}.tmp"); - if std::fs::write(&tmp, json).is_ok() { - let _ = std::fs::rename(&tmp, &tpath); - } - } - } + inner.month_upto_ts = now_secs(); + self.write_sidecar(&inner); let tmp = format!("{path}.tmp"); if std::fs::write(&tmp, kept).is_ok() { let _ = std::fs::rename(&tmp, path); @@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) { b.by_label.entry(r.task_label.clone()).or_default().apply(r); } +fn month_of(ts: u64) -> String { + // календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц) + let days = ts / 86_400; + let mut year = 1970u64; + let mut d = days; + loop { + let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0; + let len = if leap { 366 } else { 365 }; + if d < len { + break; + } + d -= len; + year += 1; + } + let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0; + let ml = [ + 31, + if leap { 29 } else { 28 }, + 31, + 30, + 31, + 30, + 31, + 31, + 30, + 31, + 30, + 31, + ]; + let mut month = 1; + for len in ml { + if d < len { + break; + } + d -= len; + month += 1; + } + format!("{year:04}-{month:02}") +} + fn avg(sum: u64, n: u64) -> u64 { sum.checked_div(n).unwrap_or(0) } diff --git a/web/js/api.js b/web/js/api.js index 276bfd2..32230ac 100644 --- a/web/js/api.js +++ b/web/js/api.js @@ -74,6 +74,7 @@ export const api = { // protocols getProtocols: () => j("GET", "/protocols"), harnesses: () => j("GET", "/harnesses"), + balance: () => j("GET", "/balance"), putProtocols: (b) => j("PUT", "/protocols", b), // settings diff --git a/web/js/i18n/de.js b/web/js/i18n/de.js index 3cf9684..16b28b7 100644 --- a/web/js/i18n/de.js +++ b/web/js/i18n/de.js @@ -639,4 +639,8 @@ export default { "settings.cache.entries": "Einträge", "settings.cache.hits": "Treffer", "settings.cache.saved": "gespart", + "keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)", + "keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.", + "keys.budgetOf": "Budget {b}/Mon.", + "dash.kpi.orBalance": "OpenRouter-Guthaben", }; diff --git a/web/js/i18n/en.js b/web/js/i18n/en.js index f7c292d..fad52b8 100644 --- a/web/js/i18n/en.js +++ b/web/js/i18n/en.js @@ -632,4 +632,8 @@ export default { "settings.cache.entries": "entries", "settings.cache.hits": "hits", "settings.cache.saved": "saved", + "keys.form.budget": "Monthly budget, $ (0 = unlimited)", + "keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.", + "keys.budgetOf": "budget {b}/mo", + "dash.kpi.orBalance": "OpenRouter balance", }; diff --git a/web/js/i18n/es.js b/web/js/i18n/es.js index 9fc8ee9..d3ca8e6 100644 --- a/web/js/i18n/es.js +++ b/web/js/i18n/es.js @@ -639,4 +639,8 @@ export default { "settings.cache.entries": "entradas", "settings.cache.hits": "aciertos", "settings.cache.saved": "ahorrado", + "keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)", + "keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.", + "keys.budgetOf": "presupuesto {b}/mes", + "dash.kpi.orBalance": "Saldo OpenRouter", }; diff --git a/web/js/i18n/fr.js b/web/js/i18n/fr.js index ba87394..a91fd5d 100644 --- a/web/js/i18n/fr.js +++ b/web/js/i18n/fr.js @@ -639,4 +639,8 @@ export default { "settings.cache.entries": "entrées", "settings.cache.hits": "hits", "settings.cache.saved": "économisé", + "keys.form.budget": "Budget mensuel, $ (0 = illimité)", + "keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.", + "keys.budgetOf": "budget {b}/mois", + "dash.kpi.orBalance": "Solde OpenRouter", }; diff --git a/web/js/i18n/ru.js b/web/js/i18n/ru.js index b0cd955..5bd19ad 100644 --- a/web/js/i18n/ru.js +++ b/web/js/i18n/ru.js @@ -631,4 +631,8 @@ export default { "settings.cache.entries": "записей", "settings.cache.hits": "попаданий", "settings.cache.saved": "сэкономлено", + "keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)", + "keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.", + "keys.budgetOf": "бюджет {b}/мес", + "dash.kpi.orBalance": "Баланс OpenRouter", }; diff --git a/web/js/i18n/tr.js b/web/js/i18n/tr.js index c92f902..b9e81ee 100644 --- a/web/js/i18n/tr.js +++ b/web/js/i18n/tr.js @@ -639,4 +639,8 @@ export default { "settings.cache.entries": "kayıt", "settings.cache.hits": "isabet", "settings.cache.saved": "tasarruf", + "keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)", + "keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.", + "keys.budgetOf": "bütçe {b}/ay", + "dash.kpi.orBalance": "OpenRouter bakiyesi", }; diff --git a/web/js/i18n/zh.js b/web/js/i18n/zh.js index 82b8e75..cd03cff 100644 --- a/web/js/i18n/zh.js +++ b/web/js/i18n/zh.js @@ -639,4 +639,8 @@ export default { "settings.cache.entries": "条目", "settings.cache.hits": "命中", "settings.cache.saved": "已节省", + "keys.form.budget": "每月预算，$（0 = 不限）", + "keys.form.budgetHint": "当账户当月支出超过上限时，请求将被拒绝并返回 429 insufficient_quota。", + "keys.budgetOf": "预算 {b}/月", + "dash.kpi.orBalance": "OpenRouter 余额", }; diff --git a/web/js/views/dashboard.js b/web/js/views/dashboard.js index c23a27d..bf59280 100644 --- a/web/js/views/dashboard.js +++ b/web/js/views/dashboard.js @@ -110,12 +110,13 @@ export async function renderDashboard() { async function load() { const rangeSecs = range; - let stats, recent, health; + let stats, recent, health, balance; try { - [stats, recent, health] = await Promise.all([ + [stats, recent, health, balance] = await Promise.all([ api.stats(rangeSecs, groupby), api.requests(40, 0), api.health().catch(() => null), + api.balance().catch(() => null), ]); } catch (e) { mount(root, h("div", { class: "callout" }, String(e.message || e))); @@ -162,6 +163,11 @@ export async function renderDashboard() { kpi(num(tot.requests), t("dash.kpi.requests")), kpi(num(tot.total_tokens), t("dash.kpi.tokens")), kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")), + balance && balance.openrouter_usd != null + ? kpi( + h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)), + t("dash.kpi.orBalance")) + : null, kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")), kpi(pct(tot.success_rate), t("dash.kpi.success")), kpi(num(tot.failovers), t("dash.kpi.failovers")) diff --git a/web/js/views/keys.js b/web/js/views/keys.js index 0c63f31..c3d4b27 100644 --- a/web/js/views/keys.js +++ b/web/js/views/keys.js @@ -10,12 +10,14 @@ export async function renderKeys() { function openCreate(reload) { const accountIn = h("input", { placeholder: "team-a" }); const rateIn = h("input", { type: "number", value: 600 }); + const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" }); const customIn = h("input", { placeholder: "sk-gw-…" }); const body = h( "div", {}, h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn), h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn), + h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))), h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint"))) ); modal(t("keys.form.title"), body, async () => { @@ -24,6 +26,7 @@ export async function renderKeys() { key: customIn.value.trim(), account: accountIn.value.trim() || "default", rate_per_min: parseInt(rateIn.value) || 0, + monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null, allow_models: [], }); reload(); @@ -273,7 +276,12 @@ console.log(resp.choices[0].message.content);`, h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")), h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"), h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"), - h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"), + h("td", { class: "right-align" }, + alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—", + k.monthly_budget_usd + ? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") }, + t("keys.budgetOf", { b: "$" + k.monthly_budget_usd })) + : null), h( "td", { class: "right-align" },