github/cortiq-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat!: API keys actually gate the gateway — auth, attribution, rate limits, budgets

Browse Source 
The office-readiness audit found the foundation missing: configured API keys
were decorative. Any bearer — or none — passed every model endpoint, per-key
rate limits existed only in the form, and all real traffic was recorded as
"anonymous", leaving the per-key spend table permanently blank.

A middleware on the model surface (/v1/*, /route, /mcp) now enforces keys:
- no keys configured → open mode as before (the console already warns);
- keys exist → Bearer or x-api-key must match (401 invalid_api_key);
- per-key rate_per_min → fixed one-minute window, 429 with Retry-After;
- NEW per-key monthly_budget_usd → 429 insufficient_quota once the account's
  month-to-date spend crosses the ceiling;
- the matched account rides the request into stats — the keys table's spend
  columns finally see real traffic.

Month-to-date spend is tracked per account in stats and persisted in the
totals sidecar with a watermark (no double counting across restarts). The
key dialog gains a budget field; the table shows "budget $N/mo" in red once
exhausted. All verified live: 401 wrong/missing key, 200 + attribution with
the right key, 429 + Retry-After over the limit, 429 insufficient_quota on
the second request past a $0.000001 budget.

Also: the dashboard shows the OpenRouter credit balance (GET
/api/v1/credits via the stored key, cached 5 min, hidden without a key, red
under $5 — live: $16.18), the server drains connections on Ctrl-C instead
of dropping them, request bodies are capped at 32 MB, and docs/DEPLOY(.ru)
cover office deployment: systemd/launchd/Windows service, nginx/caddy TLS,
security checklist, backups. 10 strings ×7 languages.
This commit is contained in:
infosave2007
2026-08-08 14:01:16 +03:00
parent dd864eed22
commit 057abaa95a
24 changed files with 578 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2
View File
@@ -87,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
---
## 🖥️ Admin console
README.ru.md
+2
View File
@@ -82,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
---
## 🖥️ Панель управления
docs/DEPLOY.md
+122
View File
@@ -0,0 +1,122 @@
# Deploying cortiq-gateway for a team / office
One gateway on a small server, API keys per department, local models for the
cheap tiers, cloud models for the hard ones. This page covers the ops side.
## 1. Run as a service
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Task Scheduler → Create Task → "Run whether user is logged on or not",
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS and the outside world
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
and bind the gateway to localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE streaming
proxy_read_timeout 3600s; # long generations
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
same with automatic certificates.)
## 3. Security checklist
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
the console warns that it is open to the network.
- **API keys**: create one per department/developer on the keys page. With at
least one key configured the model APIs require Bearer / x-api-key —
requests without a valid key get 401.
- **Rate limits**: per-key requests/min → 429 with Retry-After.
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
account's month-to-date spend crosses it.
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
## 4. Backups
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
- `secrets.toml` — provider API keys (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
cumulative per-key totals and month-to-date budget spend.
Copy those three and you can rebuild the instance anywhere.
## 5. Observability
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
OpenRouter balance (red under $5).
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
docs/DEPLOY.ru.md
+123
View File
@@ -0,0 +1,123 @@
# Развёртывание cortiq-gateway для команды / офиса
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
## 1. Запуск как сервис
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
--no-port-search`. Либо [NSSM](https://nssm.cc):
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS и внешний мир
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
привяжите к localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE-стриминг
proxy_read_timeout 3600s; # долгие генерации
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
с автоматическими сертификатами.)
## 3. Чек-лист безопасности
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
него консоль предупреждает, что открыта сети.
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
запросы без валидного ключа получают 401.
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
траты аккаунта за месяц превысят лимит.
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
## 4. Бэкапы
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
- `secrets.toml` — ключи провайдеров (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
тоталы по ключам и месячные траты для бюджетов.
Скопируйте эти три файла — и инстанс поднимается где угодно.
## 5. Наблюдаемость
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
сложности, баланс OpenRouter (красный ниже $5).
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
src/admin/mod.rs
+50
View File
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
get(get_protocols).put(put_protocols),
)
.route("/admin/api/harnesses", get(get_harnesses))
.route("/admin/api/balance", get(get_balance))
.route("/admin/api/settings", get(get_settings).put(put_settings))
.route("/admin/api/keys", get(list_keys).post(create_key))
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
@@ -1008,6 +1009,51 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
ok(json!({ "ok": true }))
}
// ─────────────────────────────── balance ───────────────────────────────────
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
// (null) when no OpenRouter key is configured — the card hides itself.
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
std::sync::OnceLock::new();
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
if let Some((at, v)) = *cache.lock().unwrap() {
if at.elapsed() < std::time::Duration::from_secs(300) {
return ok(json!({ "openrouter_usd": v }));
}
}
let key = state
.secrets
.resolve("OPENROUTER_API_KEY")
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
let Some(key) = key.filter(|k| !k.is_empty()) else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(8))
.build()
.unwrap_or_default();
let resp = client
.get("https://openrouter.ai/api/v1/credits")
.bearer_auth(&key)
.send()
.await;
let balance = match resp {
Ok(r) if r.status().is_success() => {
let v: serde_json::Value = r.json().await.unwrap_or_default();
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
Some(total - used)
}
_ => None,
};
let Some(balance) = balance else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
ok(json!({ "openrouter_usd": balance }))
}
// ──────────────────────────── harness detection ────────────────────────────
// Which coding harnesses are present on the machine the gateway runs on, so the
// API-keys view can mark tabs "installed" / "already configured for cortiq".
@@ -1357,6 +1403,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
"key_masked": mask_key(&k.key),
"account": k.account,
"rate_per_min": k.rate_per_min,
"monthly_budget_usd": k.monthly_budget_usd,
"allow_models": k.allow_models,
})
})
@@ -1372,6 +1419,8 @@ struct NewKey {
#[serde(default)]
rate_per_min: u32,
#[serde(default)]
monthly_budget_usd: Option<f64>,
#[serde(default)]
allow_models: Vec<String>,
}
@@ -1390,6 +1439,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
account: b.account,
rate_per_min: b.rate_per_min,
allow_models: b.allow_models,
monthly_budget_usd: b.monthly_budget_usd,
});
state.reload(cfg)?;
// show the full key only once
src/config.rs
+3
View File
@@ -620,6 +620,9 @@ pub struct ApiKeyCfg {
pub account: String,
#[serde(default)]
pub rate_per_min: u32,
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub monthly_budget_usd: Option<f64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub allow_models: Vec<String>,
}
src/error.rs
+7
View File
@@ -13,6 +13,8 @@ pub enum GatewayError {
Unauthorized(String),
#[error("rate limit exceeded")]
RateLimited { retry_after_secs: u64 },
#[error("monthly budget exhausted (limit ${budget_usd})")]
BudgetExhausted { budget_usd: f64 },
#[error("upstream unavailable: {0}")]
UpstreamUnavailable(String),
#[error("internal error: {0}")]
@@ -37,6 +39,11 @@ impl GatewayError {
"rate_limit_error",
"rate_limited",
),
Self::BudgetExhausted { .. } => (
StatusCode::TOO_MANY_REQUESTS,
"insufficient_quota",
"insufficient_quota",
),
Self::UpstreamUnavailable(_) => (
StatusCode::BAD_GATEWAY,
"upstream_unavailable",
src/main.rs
+15 -3
View File
@@ -258,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
.route("/health", get(|| async { "ok" }))
.route("/readyz", get(|| async { "ready" }))
.route("/metrics", get(admin::metrics))
.merge(protocols::build_router());
.merge(protocols::build_router(state.clone()));
if admin_enabled {
app = app
@@ -271,7 +271,14 @@ async fn main() -> anyhow::Result<()> {
.allow_methods(tower_http::cors::Any)
.allow_headers(tower_http::cors::Any);
let app = app.with_state(state).layer(cors);
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
// small enough that a hostile client can't balloon memory
let app =
app.with_state(state)
.layer(cors)
.layer(tower_http::limit::RequestBodyLimitLayer::new(
32 * 1024 * 1024,
));
let listener = {
let mut attempt = 0u32;
@@ -340,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("open this URL to finish setup: {url}");
}
}
axum::serve(listener, app).await?;
axum::serve(listener, app)
.with_graceful_shutdown(async {
let _ = tokio::signal::ctrl_c().await;
tracing::info!("shutdown signal received — draining connections");
})
.await?;
Ok(())
}
src/protocols/anthropic_messages.rs
+2
View File
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<MessagesRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.anthropic_messages {
@@ -108,6 +109,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "anthropic_messages".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
src/protocols/mod.rs
+51 -1
View File
@@ -12,10 +12,59 @@ pub mod openai_models;
use crate::state::SharedState;
use axum::Router;
/// Account resolved by the API-key middleware, consumed by protocol handlers
/// for attribution (stats, budgets). Present only when keys are configured.
#[derive(Clone)]
pub struct AccountTag(pub String);
/// API-key gate for the model-facing surface. Open mode (no keys configured)
/// passes everything through — the admin console already warns about it.
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
/// enforced with a fixed one-minute window (429 + Retry-After), and a
/// monthly budget, when set, turns into 429 insufficient_quota once the
/// account's month-to-date spend crosses it.
async fn api_auth(
axum::extract::State(state): axum::extract::State<SharedState>,
mut req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
use axum::response::IntoResponse;
let live = state.live();
if live.cfg.api_keys.is_empty() {
return next.run(req).await;
}
let provided = req
.headers()
.get(axum::http::header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|v| v.strip_prefix("Bearer "))
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
.map(str::trim)
.unwrap_or("");
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
.into_response();
};
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
return crate::error::GatewayError::RateLimited {
retry_after_secs: state.rate.retry_after_secs(),
}
.into_response();
}
if let Some(budget) = key.monthly_budget_usd {
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
.into_response();
}
}
req.extensions_mut().insert(AccountTag(key.account.clone()));
next.run(req).await
}
/// Build the protocols router. Implemented adapters are **always** mounted;
/// each handler checks the live `protocols.*` flag from config itself, so
/// toggles in the admin panel take effect without a restart (hot switching).
pub fn build_router() -> Router<SharedState> {
pub fn build_router(state: SharedState) -> Router<SharedState> {
Router::new()
.merge(openai_chat::routes())
.merge(openai_completions::routes())
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
.merge(anthropic_messages::routes())
.merge(mcp::routes())
.merge(native_passthrough::routes())
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
}
src/protocols/openai_chat.rs
+2
View File
@@ -75,6 +75,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<OpenAiChatRequest>,
) -> Result<Response> {
// hot protocol toggle: if the adapter is disabled in config — return 404
@@ -104,6 +105,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_chat".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
src/protocols/openai_completions.rs
+2
View File
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<CompletionsRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.openai_completions {
@@ -89,6 +90,7 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_completions".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
src/state.rs
+39
View File
@@ -66,6 +66,42 @@ impl Live {
}
}
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
/// fine for office-grade fairness and costs one HashMap lookup per request.
#[derive(Default)]
pub struct RateWindows {
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
}
impl RateWindows {
pub fn allow(&self, key: &str, per_min: u32) -> bool {
let minute = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
/ 60;
let mut g = self.inner.lock().unwrap();
let e = g.entry(key.to_string()).or_insert((minute, 0));
if e.0 != minute {
*e = (minute, 0);
}
if e.1 >= per_min {
return false;
}
e.1 += 1;
true
}
/// Seconds until the current minute window rolls over.
pub fn retry_after_secs(&self) -> u64 {
60 - (std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
% 60)
}
}
pub struct AppState {
pub live: ArcSwap<Live>,
pub stats: Arc<Stats>,
@@ -85,6 +121,8 @@ pub struct AppState {
/// Managed local CMF model server (install/update/spawn lifecycle + status).
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
pub cache: Arc<SemanticCache>,
/// Per-API-key fixed-window rate limiter.
pub rate: RateWindows,
pub secrets: SecretStore,
pub config_path: String,
pub pipeline: crate::pipeline::Pipeline,
@@ -164,6 +202,7 @@ impl SharedState {
admin_open_public: std::sync::atomic::AtomicBool::new(false),
cmf,
cache,
rate: RateWindows::default(),
secrets,
config_path,
pipeline: crate::pipeline::Pipeline::new(),
src/stats.rs
+112 -9
View File
@@ -137,6 +137,11 @@ struct Inner {
/// = folded + by_account_alltime, with no double counting: a record is
/// either in the file or folded, never both.
folded: HashMap<String, Totals>,
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
month_spend: HashMap<String, (String, f64)>,
/// Records with ts ≤ this are already reflected in the persisted
/// month_spend (watermark against double counting on replay).
month_upto_ts: u64,
/// Appends since the last compaction size check.
writes_since_check: u32,
}
@@ -193,6 +198,7 @@ impl Stats {
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
if rec.ts >= cutoff {
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
}
@@ -200,6 +206,33 @@ impl Stats {
}
}
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
return;
}
let m = month_of(rec.ts);
let e = inner
.month_spend
.entry(rec.account.clone())
.or_insert((m.clone(), 0.0));
if e.0 != m {
*e = (m, 0.0);
}
e.1 += rec.cost_usd;
}
/// Month-to-date spend for an account (budget checks).
pub fn month_spend(&self, account: &str) -> f64 {
let inner = self.inner.lock().unwrap();
let m = month_of(now_secs());
inner
.month_spend
.get(account)
.filter(|(mm, _)| *mm == m)
.map(|(_, v)| *v)
.unwrap_or(0.0)
}
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
inner.recent.push_back(rec.clone());
while inner.recent.len() > ring_size {
@@ -256,6 +289,7 @@ impl Stats {
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
inner.writes_since_check += 1;
}
if let Some(path) = &self.file {
@@ -283,11 +317,42 @@ impl Stats {
let Ok(content) = std::fs::read_to_string(&path) else {
return;
};
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
#[derive(serde::Deserialize)]
struct Sidecar {
#[serde(default)]
by_account: HashMap<String, Totals>,
#[serde(default)]
month_spend: HashMap<String, (String, f64)>,
#[serde(default)]
month_upto_ts: u64,
}
// new format {by_account, month_spend, month_upto_ts}; the original
// file held the bare by_account map — accept both
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
let mut inner = self.inner.lock().unwrap();
inner.folded = sc.by_account;
inner.month_spend = sc.month_spend;
inner.month_upto_ts = sc.month_upto_ts;
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
self.inner.lock().unwrap().folded = m;
}
}
fn write_sidecar(&self, inner: &Inner) {
let Some(tpath) = self.totals_path() else {
return;
};
let payload = serde_json::json!({
"by_account": inner.folded,
"month_spend": inner.month_spend,
"month_upto_ts": now_secs(),
});
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, payload.to_string()).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
/// older than the retention window fold into the cumulative per-account
/// sidecar and are dropped from the file (tmp + rename on both files, so
@@ -305,6 +370,10 @@ impl Stats {
return;
}
inner.writes_since_check = 0;
// piggyback: persist month-to-date spend so budgets survive
// restarts even between compactions
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
}
}
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
@@ -359,14 +428,8 @@ impl Stats {
}
}
// persist the folded totals FIRST, then swap the trimmed file in
if let Some(tpath) = self.totals_path() {
if let Ok(json) = serde_json::to_string(&inner.folded) {
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
}
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
let tmp = format!("{path}.tmp");
if std::fs::write(&tmp, kept).is_ok() {
let _ = std::fs::rename(&tmp, path);
@@ -569,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
}
fn month_of(ts: u64) -> String {
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
let days = ts / 86_400;
let mut year = 1970u64;
let mut d = days;
loop {
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let len = if leap { 366 } else { 365 };
if d < len {
break;
}
d -= len;
year += 1;
}
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let ml = [
31,
if leap { 29 } else { 28 },
31,
30,
31,
30,
31,
31,
30,
31,
30,
31,
];
let mut month = 1;
for len in ml {
if d < len {
break;
}
d -= len;
month += 1;
}
format!("{year:04}-{month:02}")
}
fn avg(sum: u64, n: u64) -> u64 {
sum.checked_div(n).unwrap_or(0)
}
web/js/api.js
Vendored
+1
View File
@@ -74,6 +74,7 @@ export const api = {
// protocols
getProtocols: () => j("GET", "/protocols"),
harnesses: () => j("GET", "/harnesses"),
balance: () => j("GET", "/balance"),
putProtocols: (b) => j("PUT", "/protocols", b),
// settings
web/js/i18n/de.js
Vendored
+4
View File
@@ -639,4 +639,8 @@ export default {
"settings.cache.entries": "Einträge",
"settings.cache.hits": "Treffer",
"settings.cache.saved": "gespart",
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
"keys.budgetOf": "Budget {b}/Mon.",
"dash.kpi.orBalance": "OpenRouter-Guthaben",
};
web/js/i18n/en.js
Vendored
+4
View File
@@ -632,4 +632,8 @@ export default {
"settings.cache.entries": "entries",
"settings.cache.hits": "hits",
"settings.cache.saved": "saved",
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
"keys.budgetOf": "budget {b}/mo",
"dash.kpi.orBalance": "OpenRouter balance",
};
web/js/i18n/es.js
Vendored
+4
View File
@@ -639,4 +639,8 @@ export default {
"settings.cache.entries": "entradas",
"settings.cache.hits": "aciertos",
"settings.cache.saved": "ahorrado",
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
"keys.budgetOf": "presupuesto {b}/mes",
"dash.kpi.orBalance": "Saldo OpenRouter",
};
web/js/i18n/fr.js
Vendored
+4
View File
@@ -639,4 +639,8 @@ export default {
"settings.cache.entries": "entrées",
"settings.cache.hits": "hits",
"settings.cache.saved": "économisé",
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
"keys.budgetOf": "budget {b}/mois",
"dash.kpi.orBalance": "Solde OpenRouter",
};
web/js/i18n/ru.js
Vendored
+4
View File
@@ -631,4 +631,8 @@ export default {
"settings.cache.entries": "записей",
"settings.cache.hits": "попаданий",
"settings.cache.saved": "сэкономлено",
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
"keys.budgetOf": "бюджет {b}/мес",
"dash.kpi.orBalance": "Баланс OpenRouter",
};
web/js/i18n/tr.js
Vendored
+4
View File
@@ -639,4 +639,8 @@ export default {
"settings.cache.entries": "kayıt",
"settings.cache.hits": "isabet",
"settings.cache.saved": "tasarruf",
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
"keys.budgetOf": "bütçe {b}/ay",
"dash.kpi.orBalance": "OpenRouter bakiyesi",
};
web/js/i18n/zh.js
Vendored
+4
View File
@@ -639,4 +639,8 @@ export default {
"settings.cache.entries": "条目",
"settings.cache.hits": "命中",
"settings.cache.saved": "已节省",
"keys.form.budget": "每月预算，$（0 = 不限）",
"keys.form.budgetHint": "当账户当月支出超过上限时，请求将被拒绝并返回 429 insufficient_quota。",
"keys.budgetOf": "预算 {b}/月",
"dash.kpi.orBalance": "OpenRouter 余额",
};
web/js/views/dashboard.js
Vendored
+8 -2
View File
@@ -110,12 +110,13 @@ export async function renderDashboard() {
async function load() {
const rangeSecs = range;
let stats, recent, health;
let stats, recent, health, balance;
try {
[stats, recent, health] = await Promise.all([
[stats, recent, health, balance] = await Promise.all([
api.stats(rangeSecs, groupby),
api.requests(40, 0),
api.health().catch(() => null),
api.balance().catch(() => null),
]);
} catch (e) {
mount(root, h("div", { class: "callout" }, String(e.message || e)));
@@ -162,6 +163,11 @@ export async function renderDashboard() {
kpi(num(tot.requests), t("dash.kpi.requests")),
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
balance && balance.openrouter_usd != null
? kpi(
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
t("dash.kpi.orBalance"))
: null,
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
kpi(pct(tot.success_rate), t("dash.kpi.success")),
kpi(num(tot.failovers), t("dash.kpi.failovers"))
web/js/views/keys.js
Vendored
+9 -1
View File
@@ -10,12 +10,14 @@ export async function renderKeys() {
function openCreate(reload) {
const accountIn = h("input", { placeholder: "team-a" });
const rateIn = h("input", { type: "number", value: 600 });
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
const customIn = h("input", { placeholder: "sk-gw-…" });
const body = h(
"div",
{},
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
);
modal(t("keys.form.title"), body, async () => {
@@ -24,6 +26,7 @@ export async function renderKeys() {
key: customIn.value.trim(),
account: accountIn.value.trim() || "default",
rate_per_min: parseInt(rateIn.value) || 0,
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
allow_models: [],
});
reload();
@@ -273,7 +276,12 @@ console.log(resp.choices[0].message.content);`,
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
h("td", { class: "right-align" },
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
k.monthly_budget_usd
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
: null),
h(
"td",
{ class: "right-align" },