feat(playground): image & video generation tabs + real chat metrics

The infosave collection now ships media models (Lumina-Image-2.0, MiniMax-H3
Turbo) that cortiq runs via `imagine`/`animate` CLI, not `cortiq serve`. The
gateway grows a media API: local .cmf files are classified by `cortiq info`
(chat/image/video, cached), generations run as tracked jobs with step progress
parsed from stdout, one at a time, cancellable, outputs under <data>/media.

The playground becomes three tabs. Image: width/height/steps/CFG/seed, live
progress, result rendered from P6 PPM to a canvas in the browser (PNG export
via toBlob). Video: frames/steps/quality/seed plus optional first/last
keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded in JS
and played on a canvas synced to the wav — no ffmpeg on either side (mp4 remux
happens opportunistically when ffmpeg exists). A metrics card shows resolution,
steps, seed, total time, time per step/frame and file size.

Chat metrics: the local cortiq server sends no usage chunk in streams, so
token counts showed 0/0. tap_stream now injects one OpenAI-shaped usage event
ahead of [DONE] with the gateway's ~4-chars/token estimate, flagged
`cortiq_estimated`; the UI renders it as "≈ N / M" and adds tok/s and
time-to-first-token. Registration skips media packs (a `cortiq serve` entry
for a diffusion model would only fail) in both Register and the wizard's
Finish.

36 new UI strings translated across all 7 languages.
This commit is contained in:
infosave2007
2026-08-07 15:55:06 +03:00
parent 32ce2f7ed7
commit a77beeabab
16 changed files with 1814 additions and 30 deletions
+35 -1
View File
@@ -542,6 +542,11 @@ impl Pipeline {
cost_usd: 0.0,
failover: attempt > 0,
};
// for the client-facing usage estimate when the provider
// sends none: the prompt size is known here, the reply is
// measured inside the tap
let prompt_chars: usize =
req.messages.iter().map(|m| m.content.chars().count()).sum();
let tapped = tap_stream(
stream,
state.stats.clone(),
@@ -551,6 +556,7 @@ impl Pipeline {
protocol,
directive,
started,
prompt_chars,
);
return Ok((info, Box::pin(tapped)));
}
@@ -722,6 +728,7 @@ fn tap_stream(
protocol: String,
directive: String,
started: Instant,
prompt_chars: usize,
) -> impl futures::Stream<Item = Result<bytes::Bytes>> {
use futures::StreamExt;
async_stream::stream! {
@@ -734,9 +741,11 @@ fn tap_stream(
let mut errored: Option<String> = None;
while let Some(item) = inner.next().await {
let mut done_marker = false;
match &item {
Ok(bytes) => {
if let Ok(s) = std::str::from_utf8(bytes) {
done_marker = s.contains("data: [DONE]");
buf.push_str(s);
while let Some(idx) = buf.find("\n\n") {
let evt: String = buf.drain(..idx + 2).collect();
@@ -774,12 +783,37 @@ fn tap_stream(
}
Err(e) => errored = Some(e.to_string()),
}
// The local cortiq server (and some providers) never send a usage
// chunk. Clients still deserve token counts, so ahead of [DONE] we
// inject one final OpenAI-shaped usage event with the gateway's
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
// mistakes it for the tokenizer's truth.
if done_marker && !got_usage {
let est_completion = (delta_chars / 4) as u32;
let est_prompt = (prompt_chars / 4) as u32;
let evt = serde_json::json!({
"object": "chat.completion.chunk",
"model": info.selected_model,
"choices": [],
"usage": {
"prompt_tokens": est_prompt,
"completion_tokens": est_completion,
"total_tokens": est_prompt + est_completion,
},
"cortiq_estimated": true,
});
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
prompt_tokens = est_prompt;
completion_tokens = est_completion;
got_usage = true;
}
yield item;
}
if !got_usage && completion_tokens == 0 {
// estimate when the server didn't return usage (~4 chars per token)
// stream ended without a [DONE] marker — estimate for the stats
completion_tokens = (delta_chars / 4) as u32;
prompt_tokens = (prompt_chars / 4) as u32;
}
let cost = provider.price(prompt_tokens, completion_tokens);
stats.record(RequestRecord {