feat: ops & UX batch — jobs survive restarts, logs/system in the console, media queue, gallery, chat, exact stream tokens
Fifteen changes aimed at the first day of real use: - Import and media jobs persist to <data>/jobs-*.json and reload on start (running ones settle as errors); the Register button no longer vanishes because the gateway restarted. - Docker ships the cortiq binary (built in the builder stage) — local .cmf chat models and image/video generation now work in containers, where auto-install never could (no cargo in the runtime image). libvulkan1 added for optional GPU offload. - Open admin console on 0.0.0.0 without a token now warns in the log and shows a dismissible banner — the VPS footgun is at least labeled. - "Restart gateway" button in Settings: stops managed model servers, spawns a fresh copy (bind retried while the port frees), Docker relies on the container restart policy instead. - Dashboard "System" panel: CPU sparkline + load, RAM (with the gateway's own RSS), free disk at the models dir — sampled by sysinfo every 2 s. - Featured model cards warn "not enough disk" / "may not fit in RAM" before the download, from the same live snapshot. - Log viewer in the sidebar: a 500-line in-memory ring fed by a second ANSI-free tracing layer; filter, level, pause, copy. - Update check: crates.io polled twice a day, the console shows an "update available" notice. - Media generations queue up (with position shown) instead of being rejected; progress gained an honest ETA from measured throughput. - Generation gallery under the image/video tabs: lazy PPM thumbnails, click to reopen with seed/steps, delete. - Playground chat is a real dialogue now: history is resent, bubbles, Clear, Cmd/Ctrl+Enter. - "Connect your app" snippets (curl / Python / JS) on the keys page, with the real key inlined right after creation. - One-click benchmark on the Models page via `cortiq bench --json`. - Config export now downloads the actual gateway.toml (comments intact); import accepts TOML (validated, hot-reloaded) and the legacy JSON. - Streaming token counts are exact when the model server provides usage (cortiq-server patch, shipped separately); the gateway fallback now counts content chunks (1/token on cortiq) instead of chars/4. Wizard's Finish lands on the Playground. 37 new strings in all 7 languages.
This commit is contained in:
+12
-2
@@ -738,6 +738,7 @@ fn tap_stream(
|
||||
let mut completion_tokens = 0u32;
|
||||
let mut got_usage = false;
|
||||
let mut delta_chars = 0usize;
|
||||
let mut delta_chunks = 0u32;
|
||||
let mut errored: Option<String> = None;
|
||||
|
||||
while let Some(item) = inner.next().await {
|
||||
@@ -774,6 +775,7 @@ fn tap_stream(
|
||||
v["choices"][0]["delta"]["content"].as_str()
|
||||
{
|
||||
delta_chars += c.chars().count();
|
||||
delta_chunks += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -789,7 +791,11 @@ fn tap_stream(
|
||||
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
|
||||
// mistakes it for the tokenizer's truth.
|
||||
if done_marker && !got_usage {
|
||||
let est_completion = (delta_chars / 4) as u32;
|
||||
let est_completion = if delta_chunks > 0 {
|
||||
delta_chunks
|
||||
} else {
|
||||
(delta_chars / 4) as u32
|
||||
};
|
||||
let est_prompt = (prompt_chars / 4) as u32;
|
||||
let evt = serde_json::json!({
|
||||
"object": "chat.completion.chunk",
|
||||
@@ -812,7 +818,11 @@ fn tap_stream(
|
||||
|
||||
if !got_usage && completion_tokens == 0 {
|
||||
// stream ended without a [DONE] marker — estimate for the stats
|
||||
completion_tokens = (delta_chars / 4) as u32;
|
||||
completion_tokens = if delta_chunks > 0 {
|
||||
delta_chunks
|
||||
} else {
|
||||
(delta_chars / 4) as u32
|
||||
};
|
||||
prompt_tokens = (prompt_chars / 4) as u32;
|
||||
}
|
||||
let cost = provider.price(prompt_tokens, completion_tokens);
|
||||
|
||||
Reference in New Issue
Block a user