Compare commits

..
4 Commits
Author SHA1 Message Date
infosave2007 914ff42fc5 release: v0.2.38 — media generation, task-based routing, ops batch
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-07 18:34:00 +03:00
infosave2007 2ede0ee452 feat(routing): per-task rules on top of complexity tiers; hosted router is the default
The router has always returned two signals — task label and complexity tier —
but only the tier picked models. [routing.tasks] adds `label → ordered models`:
a matching rule's models lead the candidate chain and the tier chain follows as
fallback, so pinning translations to a local model never weakens failover.
Requests whose chain was headed by a task rule report
route_source "router+task-rule".

The Routing page grows a per-task editor built from the account's LIVE
taxonomy (GET /v1/taxonomies via the new /admin/api/routing/labels proxy;
built-in label set when the router is unreachable), with localized task names,
each task's share of 7-day traffic, "by complexity (no rule)" as the explicit
empty state, and custom labels for other taxonomies. Deleting a model scrubs
it from task rules like it always did from tiers.

Also fixes the default config pointing at a placeholder localhost:8080 router
with a nonexistent taxonomy — a fresh install now targets the hosted allaigate
router (IP endpoint, verify_tls off, data-assistant, 12 s oracle timeout) and
degrades gracefully until a key is set. Verified end-to-end against the live
router: translation → task-rule model, math → tier path.

16 new strings in all 7 languages.
2026-08-07 17:39:38 +03:00
infosave2007 15de36be3b feat: ops & UX batch — jobs survive restarts, logs/system in the console, media queue, gallery, chat, exact stream tokens
Fifteen changes aimed at the first day of real use:

- Import and media jobs persist to <data>/jobs-*.json and reload on start
  (running ones settle as errors); the Register button no longer vanishes
  because the gateway restarted.
- Docker ships the cortiq binary (built in the builder stage) — local .cmf
  chat models and image/video generation now work in containers, where
  auto-install never could (no cargo in the runtime image). libvulkan1 added
  for optional GPU offload.
- Open admin console on 0.0.0.0 without a token now warns in the log and
  shows a dismissible banner — the VPS footgun is at least labeled.
- "Restart gateway" button in Settings: stops managed model servers, spawns
  a fresh copy (bind retried while the port frees), Docker relies on the
  container restart policy instead.
- Dashboard "System" panel: CPU sparkline + load, RAM (with the gateway's
  own RSS), free disk at the models dir — sampled by sysinfo every 2 s.
- Featured model cards warn "not enough disk" / "may not fit in RAM" before
  the download, from the same live snapshot.
- Log viewer in the sidebar: a 500-line in-memory ring fed by a second
  ANSI-free tracing layer; filter, level, pause, copy.
- Update check: crates.io polled twice a day, the console shows an
  "update available" notice.
- Media generations queue up (with position shown) instead of being
  rejected; progress gained an honest ETA from measured throughput.
- Generation gallery under the image/video tabs: lazy PPM thumbnails,
  click to reopen with seed/steps, delete.
- Playground chat is a real dialogue now: history is resent, bubbles,
  Clear, Cmd/Ctrl+Enter.
- "Connect your app" snippets (curl / Python / JS) on the keys page, with
  the real key inlined right after creation.
- One-click benchmark on the Models page via `cortiq bench --json`.
- Config export now downloads the actual gateway.toml (comments intact);
  import accepts TOML (validated, hot-reloaded) and the legacy JSON.
- Streaming token counts are exact when the model server provides usage
  (cortiq-server patch, shipped separately); the gateway fallback now
  counts content chunks (1/token on cortiq) instead of chars/4.

Wizard's Finish lands on the Playground. 37 new strings in all 7 languages.
2026-08-07 17:07:48 +03:00
infosave2007 a77beeabab feat(playground): image & video generation tabs + real chat metrics
The infosave collection now ships media models (Lumina-Image-2.0, MiniMax-H3
Turbo) that cortiq runs via `imagine`/`animate` CLI, not `cortiq serve`. The
gateway grows a media API: local .cmf files are classified by `cortiq info`
(chat/image/video, cached), generations run as tracked jobs with step progress
parsed from stdout, one at a time, cancellable, outputs under <data>/media.

The playground becomes three tabs. Image: width/height/steps/CFG/seed, live
progress, result rendered from P6 PPM to a canvas in the browser (PNG export
via toBlob). Video: frames/steps/quality/seed plus optional first/last
keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded in JS
and played on a canvas synced to the wav — no ffmpeg on either side (mp4 remux
happens opportunistically when ffmpeg exists). A metrics card shows resolution,
steps, seed, total time, time per step/frame and file size.

Chat metrics: the local cortiq server sends no usage chunk in streams, so
token counts showed 0/0. tap_stream now injects one OpenAI-shaped usage event
ahead of [DONE] with the gateway's ~4-chars/token estimate, flagged
`cortiq_estimated`; the UI renders it as "≈ N / M" and adds tok/s and
time-to-first-token. Registration skips media packs (a `cortiq serve` entry
for a diffusion model would only fail) in both Register and the wizard's
Finish.

36 new UI strings translated across all 7 languages.
2026-08-07 15:55:06 +03:00
43 changed files with 4145 additions and 136 deletions
+41
View File
@@ -3,6 +3,47 @@
All notable changes to this project are documented here. All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/). The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.38] - 2026-08-07
### Added
- **Image & video generation in the Playground**: three tabs now — Chat, Image, Video. Image runs
`cortiq imagine` over a local text-to-image `.cmf` (Lumina) with width/height/steps/CFG/seed,
live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser
with one-click PNG export. Video runs `cortiq animate` (MiniMax-H3) with frames/steps/quality
and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded
in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens
opportunistically when ffmpeg exists). Local `.cmf` files are classified chat/image/video via
`cortiq info`; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56.
- **Task-based routing**: `[routing.tasks]` maps a router task label to an ordered model list.
A matching rule's models lead the candidate chain and the complexity chain follows as fallback,
so a task pin never weakens failover. The Routing page builds the editor from the account's
live taxonomy (`GET /v1/taxonomies`), shows each task's share of 7-day traffic, and supports
custom labels. Responses whose chain was headed by a rule report `route_source "router+task-rule"`.
- **Generation gallery** under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete),
and generations **queue up** (with position shown) instead of being rejected.
- **Chat is a real dialogue**: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live
metrics: token counts, tok/s and time-to-first-token.
- **Exact stream token counts** when the model server provides usage; the gateway fallback counts
content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate.
- **Jobs survive restarts**: import and media job lists persist to `<data>/jobs-*.json` and are
reconciled on start — the Register button no longer vanishes because the gateway restarted.
- **Admin console, ops batch**: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy);
a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at
the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download;
a Restart-gateway button; an update-available notice (crates.io checked twice a day); config
export as the real `gateway.toml` and TOML import (validated, hot-reloaded); "Connect your app"
snippets (curl/Python/JS) on the keys page; one-click `cortiq bench --json` on the Models page.
- **Docker ships the cortiq binary** (built in the builder stage, plus `libvulkan1`): local `.cmf`
chat models and media generation now work in containers, where auto-install never could.
- **Security**: an open admin console listening on 0.0.0.0 outside Docker warns in the log and
shows a dismissible banner.
### Fixed
- The default config pointed at a placeholder `localhost:8080` router with a nonexistent taxonomy,
leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now
targets the hosted allaigate router and degrades gracefully until a key is set.
- The wizard's Finish lands on the Playground instead of the Models list.
## [0.2.37] - 2026-08-06 ## [0.2.37] - 2026-08-06
### Added ### Added
Generated
+219 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]] [[package]]
name = "cortiq-gateway" name = "cortiq-gateway"
version = "0.2.37" version = "0.2.38"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"arc-swap", "arc-swap",
@@ -302,6 +302,7 @@ dependencies = [
"rust-embed", "rust-embed",
"serde", "serde",
"serde_json", "serde_json",
"sysinfo",
"thiserror 1.0.69", "thiserror 1.0.69",
"tokio", "tokio",
"tokio-stream", "tokio-stream",
@@ -341,6 +342,16 @@ dependencies = [
"crypto-common", "crypto-common",
] ]
[[package]]
name = "dispatch2"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
dependencies = [
"bitflags",
"objc2",
]
[[package]] [[package]]
name = "displaydoc" name = "displaydoc"
version = "0.2.6" version = "0.2.6"
@@ -873,6 +884,15 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "ntapi"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae"
dependencies = [
"winapi",
]
[[package]] [[package]]
name = "nu-ansi-term" name = "nu-ansi-term"
version = "0.50.3" version = "0.50.3"
@@ -882,6 +902,63 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "objc2"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a12a8ed07aefc768292f076dc3ac8c48f3781c8f2d5851dd3d98950e8c5a89f"
dependencies = [
"objc2-encode",
]
[[package]]
name = "objc2-core-foundation"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
dependencies = [
"bitflags",
"dispatch2",
"objc2",
]
[[package]]
name = "objc2-encode"
version = "4.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33"
[[package]]
name = "objc2-foundation"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
dependencies = [
"bitflags",
"objc2",
]
[[package]]
name = "objc2-io-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15"
dependencies = [
"libc",
"objc2-core-foundation",
]
[[package]]
name = "objc2-open-directory"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb82bed227edf5201dfedf072bba4015a33d3d4a98519837295a90f0a23f676d"
dependencies = [
"objc2",
"objc2-core-foundation",
"objc2-foundation",
]
[[package]] [[package]]
name = "once_cell" name = "once_cell"
version = "1.21.4" version = "1.21.4"
@@ -1430,6 +1507,21 @@ dependencies = [
"syn", "syn",
] ]
[[package]]
name = "sysinfo"
version = "0.39.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2071df9448915b71c4fe6d25deaf1c22f12bd234f01540b77312bb8e41361e6"
dependencies = [
"libc",
"memchr",
"ntapi",
"objc2-core-foundation",
"objc2-io-kit",
"objc2-open-directory",
"windows",
]
[[package]] [[package]]
name = "thiserror" name = "thiserror"
version = "1.0.69" version = "1.0.69"
@@ -1914,6 +2006,22 @@ dependencies = [
"rustls-pki-types", "rustls-pki-types",
] ]
[[package]]
name = "winapi"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]] [[package]]
name = "winapi-util" name = "winapi-util"
version = "0.1.11" version = "0.1.11"
@@ -1923,12 +2031,113 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580"
dependencies = [
"windows-collections",
"windows-core",
"windows-future",
"windows-numerics",
]
[[package]]
name = "windows-collections"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610"
dependencies = [
"windows-core",
]
[[package]]
name = "windows-core"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
dependencies = [
"windows-implement",
"windows-interface",
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-future"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb"
dependencies = [
"windows-core",
"windows-link",
"windows-threading",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-interface"
version = "0.59.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]] [[package]]
name = "windows-link" name = "windows-link"
version = "0.2.1" version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-numerics"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26"
dependencies = [
"windows-core",
"windows-link",
]
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-strings"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
dependencies = [
"windows-link",
]
[[package]] [[package]]
name = "windows-sys" name = "windows-sys"
version = "0.52.0" version = "0.52.0"
@@ -1989,6 +2198,15 @@ dependencies = [
"windows_x86_64_msvc 0.53.1", "windows_x86_64_msvc 0.53.1",
] ]
[[package]]
name = "windows-threading"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37"
dependencies = [
"windows-link",
]
[[package]] [[package]]
name = "windows_aarch64_gnullvm" name = "windows_aarch64_gnullvm"
version = "0.52.6" version = "0.52.6"
+4 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "cortiq-gateway" name = "cortiq-gateway"
version = "0.2.37" version = "0.2.38"
edition = "2021" edition = "2021"
rust-version = "1.75" rust-version = "1.75"
license = "Apache-2.0" license = "Apache-2.0"
@@ -61,6 +61,9 @@ clap = { version = "4", features = ["derive"] }
# open browser on first run (optional) # open browser on first run (optional)
open = { version = "5", optional = true } open = { version = "5", optional = true }
# host metrics for the admin dashboard (CPU / RAM / disk)
sysinfo = "0.39"
# errors / logging / metrics # errors / logging / metrics
thiserror = "1" thiserror = "1"
anyhow = "1" anyhow = "1"
+8 -2
View File
@@ -3,6 +3,10 @@
# drifts to a newer glibc than the runtime image and the binary refuses to start # drifts to a newer glibc than the runtime image and the binary refuses to start
FROM rust:1-slim-trixie AS builder FROM rust:1-slim-trixie AS builder
WORKDIR /app WORKDIR /app
# the cortiq runtime CANNOT be auto-installed inside the container (no cargo in
# the runtime image), so it is compiled here and shipped alongside the gateway —
# local .cmf chat models and image/video generation work in Docker out of the box
RUN cargo install cortiq-cli --locked
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present # the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
COPY Cargo.toml Cargo.lock ./ COPY Cargo.toml Cargo.lock ./
COPY src ./src COPY src ./src
@@ -11,13 +15,15 @@ RUN cargo build --release
# ---- runtime stage ---------------------------------------------------------- # ---- runtime stage ----------------------------------------------------------
FROM debian:trixie-slim FROM debian:trixie-slim
# curl is required at runtime — model imports download ready .cmf files with it # curl — model imports download ready .cmf files with it;
# libvulkan1 — optional GPU offload for cortiq (falls back to CPU without a GPU)
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \ && apt-get install -y --no-install-recommends ca-certificates curl libvulkan1 \
&& rm -rf /var/lib/apt/lists/* \ && rm -rf /var/lib/apt/lists/* \
&& useradd -r -u 10001 cortiq \ && useradd -r -u 10001 cortiq \
&& mkdir -p /app/config /app/data/models && mkdir -p /app/config /app/data/models
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
COPY --from=builder /usr/local/cargo/bin/cortiq /usr/local/bin/cortiq
# ship the example as the default config so `docker run` works out of the box # ship the example as the default config so `docker run` works out of the box
COPY config/gateway.example.toml /app/config/gateway.toml COPY config/gateway.example.toml /app/config/gateway.toml
RUN chown -R 10001:10001 /app RUN chown -R 10001:10001 /app
+3 -2
View File
@@ -40,7 +40,8 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)). - **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE). - **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators. - **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. - **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. Plus **per-task rules** (`[routing.tasks]`): pin translations to a local model and code to your strongest coder — the complexity chain stays as fallback ([docs](docs/ROUTING.md)).
- **Image & video generation.** The Playground drives local text-to-image (Lumina) and text-to-video (MiniMax-H3) `.cmf` models via `cortiq imagine`/`animate`: live progress with ETA, a generation gallery, PNG export, in-browser AVI playback — no ffmpeg anywhere.
- **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard). - **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard).
- **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes. - **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes.
- **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint. - **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint.
@@ -137,7 +138,7 @@ The wizard takes you from nothing to a working local model:
| **Language** | 7 languages; changeable later from the header | | **Language** | 7 languages; changeable later from the header |
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress | | **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` | | **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, and `routing.default` points at it | | **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, `routing.default` points at it, and you land in the Playground |
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config — the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
+3 -2
View File
@@ -39,7 +39,8 @@
- **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели. - **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели.
- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)). - **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE). - **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. - **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. Плюс **правила по задачам** (`[routing.tasks]`): переводы — в локальную модель, код — сильнейшему кодеру; цепочка по сложности остаётся резервом ([доки](docs/ROUTING.ru.md)).
- **Генерация картинок и видео.** Песочница запускает локальные text-to-image (Lumina) и text-to-video (MiniMax-H3) `.cmf` модели через `cortiq imagine`/`animate`: живой прогресс с ETA, галерея генераций, экспорт в PNG, проигрывание AVI прямо в браузере — без ffmpeg вообще.
- **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема. - **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
- **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`. - **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
- **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга. - **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга.
@@ -129,7 +130,7 @@ cortiq-gateway
| **Язык** | 7 языков; потом меняется в шапке | | **Язык** | 7 языков; потом меняется в шапке |
| **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки | | **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки |
| **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` | | **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` |
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё | | **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё — и вы попадаете в Песочницу |
Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели. Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели.
Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг
+4 -4
View File
@@ -14,11 +14,11 @@ listen = "0.0.0.0:9000"
# NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the # NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the
# hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default. # hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default.
[router] [router]
url = "http://localhost:8080" url = "https://138.226.222.209" # hosted allaigate router (or your own cortiq-router URL)
api_key_env = "CORTIQ_ROUTER_KEY" # name of the env var holding the router key api_key_env = "CORTIQ_ROUTER_KEY" # name of the env var holding the router key
timeout_ms = 800 # timeout for the /v1/route call (12000+ for oracle) timeout_ms = 12000 # hard prompts escalate to an oracle (~10 s)
taxonomy_id = "general.task-type" # for allaigate: "data-assistant" taxonomy_id = "data-assistant" # the hosted router's task taxonomy
verify_tls = true # false if the router uses an IP / self-signed cert verify_tls = false # the hosted endpoint is IP-bound; true for your own domain
# --- how to pick the text to classify from the conversation ----------------- # --- how to pick the text to classify from the conversation -----------------
[route] [route]
+20
View File
@@ -162,3 +162,23 @@ fine-tune the task mask. Over time this makes routing fit your specific pool mor
enabled = false enabled = false
# the ground-truth source is set by the integrator (e.g. the X-Cortiq-Correct-Label header) # the ground-truth source is set by the integrator (e.g. the X-Cortiq-Correct-Label header)
``` ```
## Task-based routing
The router classifies every request twice: a **task label** (what kind of work it
is) and a **complexity tier** (how hard it is). Tiers pick *how much model* you
need; task rules pick *which* model is best at the job:
```toml
[routing.tasks]
translation = ["local-qwen"] # translations are cheap — keep them local
code = ["claude-sonnet", "gpt"] # code goes to the strongest coder first
```
A matching rule's models go **first** in the candidate chain; the complexity
chain follows as fallback, so a task pin never weakens failover. Tasks without
a rule route by complexity alone. The available labels come from the router
(`GET /v1/taxonomies`) — the admin console's Routing page builds the editor
from your account's live taxonomy and shows each task's share of recent
traffic. When a rule supplied the head of the chain the response reports
`route_source: "router+task-rule"`.
+19
View File
@@ -160,3 +160,22 @@ cooldown = "30s"
enabled = false enabled = false
# источник истины задаёт интегратор (например, заголовок X-Cortiq-Correct-Label) # источник истины задаёт интегратор (например, заголовок X-Cortiq-Correct-Label)
``` ```
## Распределение по задачам
Роутер классифицирует каждый запрос дважды: **тип задачи** (что это за работа)
и **уровень сложности** (насколько она трудна). Уровни выбирают, *сколько
модели* нужно; правила по задачам — *какая* модель лучше в этом деле:
```toml
[routing.tasks]
translation = ["local-qwen"] # переводы дёшевы — оставляем локально
code = ["claude-sonnet", "gpt"] # код — сильнейшему кодеру
```
Модели подходящего правила идут **первыми** в цепочке кандидатов; цепочка по
сложности следует за ними как резерв — правило по задаче не ослабляет failover.
Задачи без правила маршрутизируются только по сложности. Список меток приходит
с роутера (`GET /v1/taxonomies`) — вкладка Роутинг строит редактор по живой
таксономии аккаунта и показывает долю каждой задачи в трафике. Когда цепочку
возглавило правило задачи, ответ сообщает `route_source: "router+task-rule"`.
+277 -2
View File
@@ -155,6 +155,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
) )
.route("/admin/api/models/:id/probe", post(probe_model)) .route("/admin/api/models/:id/probe", post(probe_model))
.route("/admin/api/routing", get(get_routing).put(put_routing)) .route("/admin/api/routing", get(get_routing).put(put_routing))
.route("/admin/api/routing/labels", get(get_routing_labels))
.route( .route(
"/admin/api/protocols", "/admin/api/protocols",
get(get_protocols).put(put_protocols), get(get_protocols).put(put_protocols),
@@ -184,6 +185,20 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
.route("/admin/api/requests", get(get_requests)) .route("/admin/api/requests", get(get_requests))
.route("/admin/api/test", post(run_test)) .route("/admin/api/test", post(run_test))
.route("/admin/api/test/stream", post(run_test_stream)) .route("/admin/api/test/stream", post(run_test_stream))
.route("/admin/api/logs", get(get_logs))
.route("/admin/api/system", get(get_system))
.route("/admin/api/restart", post(restart_gateway))
.route("/admin/api/bench", get(bench_list).post(bench_start))
.route("/admin/api/config/export", get(config_export))
.route("/admin/api/config/import", post(config_import))
.route("/admin/api/media/models", get(media_models))
.route("/admin/api/media", get(media_jobs).post(media_generate))
.route(
"/admin/api/media/:job",
get(media_job_status).delete(media_delete),
)
.route("/admin/api/media/:job/cancel", post(media_cancel))
.route("/admin/api/media/:job/file/:kind", get(media_file))
.route("/admin/api/setup/status", get(setup_status)) .route("/admin/api/setup/status", get(setup_status))
.route("/admin/api/setup/complete", post(setup_complete)) .route("/admin/api/setup/complete", post(setup_complete))
.route("/admin/api/setup/featured", get(setup_featured)) .route("/admin/api/setup/featured", get(setup_featured))
@@ -583,9 +598,13 @@ async fn provider_models(
ok(resp) ok(resp)
} }
async fn meta() -> ApiResult { async fn meta(State(state): State<SharedState>) -> ApiResult {
ok(json!({ ok(json!({
"version": env!("CARGO_PKG_VERSION"), "version": env!("CARGO_PKG_VERSION"),
"update_available": state.gateway_update.lock().unwrap().clone(),
"admin_open_public": state
.admin_open_public
.load(std::sync::atomic::Ordering::Relaxed),
"providers": ["openai", "anthropic", "openrouter", "lmstudio", "ollama", "http"], "providers": ["openai", "anthropic", "openrouter", "lmstudio", "ollama", "http"],
"provider_impl": { "openai": true, "anthropic": true, "openrouter": true, "lmstudio": true, "ollama": true, "http": true }, "provider_impl": { "openai": true, "anthropic": true, "openrouter": true, "lmstudio": true, "ollama": true, "http": true },
"cost_tiers": ["cheap", "mid", "expensive", "local"], "cost_tiers": ["cheap", "mid", "expensive", "local"],
@@ -726,7 +745,7 @@ fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
if routing.default == id { if routing.default == id {
routing.default = String::new(); routing.default = String::new();
} }
for tt in routing.tiers.values_mut() { for tt in routing.tiers.values_mut().chain(routing.tasks.values_mut()) {
match tt { match tt {
crate::config::TierTargets::List(v) => v.retain(|x| x != id), crate::config::TierTargets::List(v) => v.retain(|x| x != id),
crate::config::TierTargets::One(s) if s == id => { crate::config::TierTargets::One(s) if s == id => {
@@ -735,6 +754,11 @@ fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
crate::config::TierTargets::One(_) => {} crate::config::TierTargets::One(_) => {}
} }
} }
// an emptied task rule means "no rule", not "no candidates"
routing.tasks.retain(|_, tt| match tt {
crate::config::TierTargets::List(v) => !v.is_empty(),
crate::config::TierTargets::One(s) => !s.is_empty(),
});
} }
/// Whether any managed server or static model in `cfg` still uses `path` as its /// Whether any managed server or static model in `cfg` still uses `path` as its
@@ -875,17 +899,70 @@ async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -
async fn get_routing(State(state): State<SharedState>) -> ApiResult { async fn get_routing(State(state): State<SharedState>) -> ApiResult {
let live = state.live(); let live = state.live();
let tasks: HashMap<String, Vec<String>> = live
.cfg
.routing
.tasks
.iter()
.map(|(k, v)| {
(
k.clone(),
match v {
TierTargets::List(l) => l.clone(),
TierTargets::One(s) => vec![s.clone()],
},
)
})
.collect();
ok(json!({ ok(json!({
"tiers": tiers_to_map(&live.cfg.routing), "tiers": tiers_to_map(&live.cfg.routing),
"tasks": tasks,
"default": live.cfg.routing.default, "default": live.cfg.routing.default,
"policy": live.cfg.routing.policy, "policy": live.cfg.routing.policy,
})) }))
} }
/// Task labels of the active taxonomy, straight from the router (`GET
/// /v1/taxonomies`) — the UI builds the per-task rule editor from this, so a
/// custom taxonomy shows its own labels without a gateway update.
async fn get_routing_labels(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
match live.router.taxonomies().await {
Ok(Some(v)) => {
let want = live.cfg.router.taxonomy_id.clone().unwrap_or_default();
let mut labels: Vec<String> = Vec::new();
if let Some(arr) = v["taxonomies"].as_array() {
for tx in arr {
let id = tx["taxonomy_id"].as_str().unwrap_or("");
if want.is_empty() || id == want || labels.is_empty() {
if let Some(ls) = tx["labels"].as_array() {
let this: Vec<String> = ls
.iter()
.filter_map(|l| l.as_str().map(String::from))
.collect();
if id == want || labels.is_empty() {
labels = this;
}
}
}
}
}
if labels.is_empty() {
ok(json!({ "labels": Value::Null, "source": "router-empty" }))
} else {
ok(json!({ "labels": labels, "source": "router" }))
}
}
_ => ok(json!({ "labels": Value::Null, "source": "unavailable" })),
}
}
#[derive(Deserialize)] #[derive(Deserialize)]
struct RoutingBody { struct RoutingBody {
#[serde(default)] #[serde(default)]
tiers: HashMap<String, Vec<String>>, tiers: HashMap<String, Vec<String>>,
#[serde(default)]
tasks: HashMap<String, Vec<String>>,
default: String, default: String,
#[serde(default)] #[serde(default)]
policy: RoutingPolicy, policy: RoutingPolicy,
@@ -899,6 +976,12 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
.into_iter() .into_iter()
.map(|(k, v)| (k, TierTargets::List(v))) .map(|(k, v)| (k, TierTargets::List(v)))
.collect(), .collect(),
tasks: body
.tasks
.into_iter()
.filter(|(_, v)| !v.is_empty()) // empty rule = "по сложности", not a rule
.map(|(k, v)| (k, TierTargets::List(v)))
.collect(),
default: body.default, default: body.default,
policy: body.policy, policy: body.policy,
}; };
@@ -1420,8 +1503,16 @@ async fn setup_complete(
.map(|j| j.output) .map(|j| j.output)
.collect(); .collect();
if !finished.is_empty() { if !finished.is_empty() {
let cortiq_bin = state.live().cfg.cmf.cortiq_bin.clone();
let mut cfg = current_cfg(&state); let mut cfg = current_cfg(&state);
for output in &finished { for output in &finished {
// media packs (image/video) are used by the media API directly,
// not served — connecting them as chat servers would only fail
let modality =
crate::media::classify_file(std::path::Path::new(output), &cortiq_bin).await;
if modality != crate::media::Modality::Chat {
continue;
}
if let Some(id) = register_cmf_output(&mut cfg, output) { if let Some(id) = register_cmf_output(&mut cfg, output) {
registered.push(id); registered.push(id);
} }
@@ -1972,6 +2063,15 @@ async fn register_import(State(state): State<SharedState>, Path(job): Path<Strin
if j.state != "done" { if j.state != "done" {
return Err(ApiError::bad("conversion not finished")); return Err(ApiError::bad("conversion not finished"));
} }
let live = state.live();
// Image/video packs are not served by `cortiq serve` — they are used
// directly by the media API. Nothing to register; tell the UI what it is.
let modality =
crate::media::classify_file(std::path::Path::new(&j.output), &live.cfg.cmf.cortiq_bin)
.await;
if modality != crate::media::Modality::Chat {
return ok(json!({ "ok": true, "media_kind": modality, "model_id": null }));
}
let mut cfg = current_cfg(&state); let mut cfg = current_cfg(&state);
let id = register_cmf_output(&mut cfg, &j.output).ok_or_else(|| { let id = register_cmf_output(&mut cfg, &j.output).ok_or_else(|| {
ApiError::bad(format!( ApiError::bad(format!(
@@ -2004,6 +2104,181 @@ async fn get_requests(State(state): State<SharedState>, Query(q): Query<RecentQu
ok(json!({ "requests": items })) ok(json!({ "requests": items }))
} }
// ───────────────────────── system / logs / lifecycle ─────────────────────────
/// Recent gateway log lines (in-memory ring) — for users who never see the
/// terminal: Docker, autostart, a double-clicked exe.
async fn get_logs(Query(q): Query<RecentQuery>) -> ApiResult {
let limit = q.limit.unwrap_or(300).min(500);
ok(json!({ "lines": crate::logbuf::snapshot(limit) }))
}
/// Host metrics snapshot: CPU, RAM, free disk at the models dir, gateway RSS.
async fn get_system(State(state): State<SharedState>) -> ApiResult {
let snap = state.sysmon.snapshot();
ok(serde_json::to_value(snap).unwrap_or_else(|_| json!({})))
}
/// Restart the gateway in place: stop managed model servers, spawn a fresh
/// copy of ourselves (it retries the bind while our port frees), then exit.
/// In Docker we only exit — we are PID 1, a spawned child would die with the
/// container anyway; the container's restart policy brings the gateway back.
async fn restart_gateway(State(state): State<SharedState>) -> ApiResult {
state.cmf.stop().await;
let in_docker = std::path::Path::new("/.dockerenv").exists();
let exe = std::env::current_exe().map_err(|e| ApiError::bad(e.to_string()))?;
let args: Vec<String> = std::env::args().skip(1).collect();
tokio::spawn(async move {
// let the 200 reach the browser first
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
if !in_docker {
let _ = std::process::Command::new(exe)
.args(args)
.env("CORTIQ_BIND_RETRY", "1")
.spawn();
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
}
std::process::exit(0);
});
ok(json!({ "ok": true }))
}
#[derive(Deserialize)]
struct BenchBody {
model: String,
}
/// Benchmark a local .cmf with `cortiq bench --json` — real tok/s on this box.
async fn bench_start(State(state): State<SharedState>, Json(b): Json<BenchBody>) -> ApiResult {
let live = state.live();
crate::bench::start(state.bench.clone(), &live.cfg.cmf, b.model).map_err(ApiError::bad)?;
ok(json!({ "ok": true }))
}
async fn bench_list(State(state): State<SharedState>) -> ApiResult {
ok(json!({ "runs": state.bench.list() }))
}
/// The config file as-is (comments included) for download. Secrets live in
/// secrets.toml and are deliberately NOT part of the export.
async fn config_export(State(state): State<SharedState>) -> Response {
match tokio::fs::read_to_string(&state.config_path).await {
Ok(text) => (
[
(header::CONTENT_TYPE, "application/toml".to_string()),
(
header::CONTENT_DISPOSITION,
"attachment; filename=\"gateway.toml\"".to_string(),
),
],
text,
)
.into_response(),
Err(e) => ApiError::bad(format!("cannot read config: {e}")).into_response(),
}
}
/// Replace the whole config from an uploaded TOML. Validated and hot-reloaded
/// exactly like a panel edit; on any error the current config stays untouched.
async fn config_import(State(state): State<SharedState>, body: String) -> ApiResult {
let cfg: crate::config::Config =
toml::from_str(&body).map_err(|e| ApiError::bad(format!("invalid TOML: {e}")))?;
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── media generation ─────────────────────────────
/// Local `.cmf` files with their modality (chat / image / video) — the
/// playground's media tabs read this to know what can generate what.
async fn media_models(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let models =
crate::media::list_media_models(&live.cfg.cmf.models_dir, &live.cfg.cmf.cortiq_bin).await;
ok(json!({ "models": models }))
}
async fn media_jobs(State(state): State<SharedState>) -> ApiResult {
ok(json!({ "jobs": state.media.list() }))
}
async fn media_generate(
State(state): State<SharedState>,
Json(p): Json<crate::media::MediaParams>,
) -> ApiResult {
let live = state.live();
let id = crate::media::start_generation(state.media.clone(), &live.cfg.cmf, p)
.map_err(ApiError::bad)?;
ok(json!({ "job": id }))
}
async fn media_job_status(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
match state.media.get(&job) {
Some(j) => {
let mut v = serde_json::to_value(&j).unwrap_or_else(|_| json!({}));
if j.state == "queued" {
v["queue_position"] = json!(state.media.queue_position(&job));
}
ok(v)
}
None => Err(ApiError::not_found(format!("media job '{job}' not found"))),
}
}
async fn media_cancel(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
if state.media.get(&job).is_none() {
return Err(ApiError::not_found(format!("media job '{job}' not found")));
}
ok(json!({ "ok": state.media.cancel(&job, &crate::media::media_dir()) }))
}
async fn media_delete(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
if state.media.get(&job).is_none() {
return Err(ApiError::not_found(format!("media job '{job}' not found")));
}
let removed = state
.media
.delete(&job, &crate::media::media_dir())
.map_err(ApiError::bad)?;
ok(json!({ "ok": removed }))
}
/// Serve a finished job's output bytes (ppm / avi / wav / mp4). The job id is
/// server-generated hex and the file name is fixed per kind, so no path from the
/// client ever reaches the filesystem.
async fn media_file(
State(state): State<SharedState>,
Path((job, kind)): Path<(String, String)>,
) -> Response {
let Some(j) = state.media.get(&job) else {
return ApiError::not_found(format!("media job '{job}' not found")).into_response();
};
if j.state != "done" {
return ApiError::bad("generation not finished").into_response();
}
let Some((file, mime)) = crate::media::output_file(&kind) else {
return ApiError::bad(format!("unknown output kind '{kind}'")).into_response();
};
let path = crate::media::media_dir().join(&job).join(file);
match tokio::fs::read(&path).await {
Ok(bytes) => (
[
(header::CONTENT_TYPE, mime.to_string()),
(
header::CONTENT_DISPOSITION,
format!(
"inline; filename=\"cortiq-{job}.{}\"",
file.rsplit('.').next().unwrap_or("bin")
),
),
],
bytes,
)
.into_response(),
Err(_) => ApiError::not_found("output file missing on disk").into_response(),
}
}
// ───────────────────────────── playground (test) ─────────────────────────── // ───────────────────────────── playground (test) ───────────────────────────
#[derive(Deserialize)] #[derive(Deserialize)]
+166
View File
@@ -0,0 +1,166 @@
//! One-click local model benchmark: run `cortiq bench <model> --json` and keep
//! the parsed result per model, so the Models page can show real tok/s numbers
//! measured on this machine.
use serde::Serialize;
use std::collections::HashMap;
use std::process::Stdio;
use std::sync::{Arc, Mutex};
#[derive(Clone, Serialize)]
pub struct BenchRun {
pub model: String, // file stem under models_dir
pub state: String, // running | done | error
pub log: Vec<String>,
pub tok_s: Option<f64>,
/// The full JSON object the benchmark printed, passed through untouched.
pub raw: Option<serde_json::Value>,
pub started: u64,
pub finished: Option<u64>,
}
#[derive(Default)]
pub struct BenchStore {
runs: Mutex<HashMap<String, BenchRun>>,
busy: Mutex<bool>,
}
fn now() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// tok/s out of the benchmark's JSON: first numeric field whose key mentions
/// tokens-per-second (the exact name has varied across cortiq releases).
fn extract_tok_s(v: &serde_json::Value) -> Option<f64> {
let obj = v.as_object()?;
for want in ["tok_s", "tok_per_s", "tokens_per_sec", "decode_tok_s"] {
if let Some(n) = obj.get(want).and_then(|x| x.as_f64()) {
return Some(n);
}
}
obj.iter()
.find(|(k, val)| {
let k = k.to_ascii_lowercase();
(k.contains("tok") && (k.contains("/s") || k.contains("_s") || k.contains("per")))
&& val.as_f64().is_some()
})
.and_then(|(_, val)| val.as_f64())
}
impl BenchStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
pub fn get(&self, model: &str) -> Option<BenchRun> {
self.runs.lock().unwrap().get(model).cloned()
}
pub fn list(&self) -> Vec<BenchRun> {
self.runs.lock().unwrap().values().cloned().collect()
}
}
/// Kick off a benchmark for `<models_dir>/<model>.cmf`. One at a time — a
/// benchmark loads the whole model and owns the machine while it runs.
pub fn start(
store: Arc<BenchStore>,
cmf: &crate::config::CmfCfg,
model: String,
) -> Result<(), String> {
let path = std::path::Path::new(&cmf.models_dir).join(format!("{model}.cmf"));
if !path.exists() {
return Err(format!("model file not found: {}", path.display()));
}
{
let mut busy = store.busy.lock().unwrap();
if *busy {
return Err("a benchmark is already running — wait for it to finish".into());
}
*busy = true;
}
store.runs.lock().unwrap().insert(
model.clone(),
BenchRun {
model: model.clone(),
state: "running".into(),
log: Vec::new(),
tok_s: None,
raw: None,
started: now(),
finished: None,
},
);
let bin = cmf.cortiq_bin.clone();
let gpu = cmf.gpu;
tokio::spawn(async move {
let out = tokio::process::Command::new(&bin)
.arg("bench")
.arg(&path)
.arg("--json")
.env("CMF_GPU", if gpu { "1" } else { "0" })
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.await;
let mut g = store.runs.lock().unwrap();
let run = g.get_mut(&model).unwrap();
run.finished = Some(now());
match out {
Ok(o) => {
let text = format!(
"{}\n{}",
String::from_utf8_lossy(&o.stdout),
String::from_utf8_lossy(&o.stderr)
);
// keep a readable tail; the JSON line is parsed separately
run.log = text
.lines()
.map(|l| l.trim())
.filter(|l| !l.is_empty() && !l.contains("Metal GPU path"))
.map(String::from)
.collect::<Vec<_>>();
let n = run.log.len();
if n > 30 {
run.log.drain(0..n - 30);
}
// the --json contract: one JSON object on its own line
let parsed = text
.lines()
.rev()
.map(str::trim)
.find(|l| l.starts_with('{') && l.ends_with('}'))
.and_then(|l| serde_json::from_str::<serde_json::Value>(l).ok());
match parsed {
Some(v) if o.status.success() => {
run.tok_s = extract_tok_s(&v);
run.raw = Some(v);
run.state = "done".into();
}
_ => run.state = "error".into(),
}
}
Err(e) => {
run.log.push(format!("✗ failed to start cortiq bench: {e}"));
run.state = "error".into();
}
}
drop(g);
*store.busy.lock().unwrap() = false;
});
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tok_s_from_common_keys() {
let v = serde_json::json!({"tok_s": 42.5, "allocs_per_token": 0});
assert_eq!(extract_tok_s(&v), Some(42.5));
let v = serde_json::json!({"decode_tokens_per_sec": 31.0});
assert_eq!(extract_tok_s(&v), Some(31.0));
}
}
+13 -3
View File
@@ -197,14 +197,14 @@ impl Default for CmfRuntime {
} }
} }
/// crates.io `newest_version` for `cortiq-cli`. /// crates.io `newest_version` for any crate.
async fn latest_crates_version() -> Option<String> { async fn latest_version_of(krate: &str) -> Option<String> {
let client = reqwest::Client::builder() let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10)) .timeout(Duration::from_secs(10))
.build() .build()
.ok()?; .ok()?;
let body = client let body = client
.get("https://crates.io/api/v1/crates/cortiq-cli") .get(format!("https://crates.io/api/v1/crates/{krate}"))
.header("User-Agent", "cortiq-gateway") .header("User-Agent", "cortiq-gateway")
.send() .send()
.await .await
@@ -216,6 +216,16 @@ async fn latest_crates_version() -> Option<String> {
v["crate"]["newest_version"].as_str().map(|s| s.to_string()) v["crate"]["newest_version"].as_str().map(|s| s.to_string())
} }
/// crates.io `newest_version` for `cortiq-cli`.
async fn latest_crates_version() -> Option<String> {
latest_version_of("cortiq-cli").await
}
/// Newest published gateway version — the "update available" footer check.
pub(crate) async fn latest_gateway_version() -> Option<String> {
latest_version_of("cortiq-gateway").await
}
/// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2"). /// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2").
pub(crate) fn installed_version(bin: &str) -> Option<String> { pub(crate) fn installed_version(bin: &str) -> Option<String> {
let out = std::process::Command::new(bin) let out = std::process::Command::new(bin)
+29 -4
View File
@@ -396,6 +396,12 @@ pub struct RoutingCfg {
pub default: String, pub default: String,
#[serde(default)] #[serde(default)]
pub policy: RoutingPolicy, pub policy: RoutingPolicy,
/// Per-task rules: `task_label → ordered model list`. A matching rule's
/// models go FIRST; the complexity-tier chain follows as fallback, so a
/// task pin never weakens failover. Named field, so a tier cannot be
/// called "tasks" — serde extracts it before the flattened tier map.
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
pub tasks: HashMap<String, TierTargets>,
} }
/// A routing tier is either a list of model_ids or the string `default`. /// A routing tier is either a list of model_ids or the string `default`.
@@ -724,13 +730,16 @@ impl Default for Config {
fn default() -> Self { fn default() -> Self {
Self { Self {
listen: "0.0.0.0:9000".to_string(), listen: "0.0.0.0:9000".to_string(),
// The hosted allaigate router — the thing this gateway is built
// around. Without a key the pipeline degrades gracefully to local
// models and the console asks for one; nothing breaks.
router: RouterCfg { router: RouterCfg {
enabled: true, enabled: true,
url: "http://localhost:8080".to_string(), url: "https://138.226.222.209".to_string(),
timeout_ms: default_router_timeout(), timeout_ms: 12_000, // hard prompts escalate to an oracle (~10 s)
verify_tls: true, verify_tls: false, // the hosted endpoint is IP-bound
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()), api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
taxonomy_id: Some("general.task-type".to_string()), taxonomy_id: Some("data-assistant".to_string()),
}, },
route: RouteCfg::default(), route: RouteCfg::default(),
models: Vec::new(), models: Vec::new(),
@@ -738,6 +747,7 @@ impl Default for Config {
tiers: HashMap::new(), tiers: HashMap::new(),
default: String::new(), default: String::new(),
policy: RoutingPolicy::default(), policy: RoutingPolicy::default(),
tasks: HashMap::new(),
}, },
breaker: BreakerCfg::default(), breaker: BreakerCfg::default(),
protocols: ProtocolsCfg { protocols: ProtocolsCfg {
@@ -862,6 +872,21 @@ impl Config {
} }
} }
} }
for (task, targets) in &self.routing.tasks {
let list = match targets {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
for id in &list {
if !ids.contains(id.as_str()) {
anyhow::bail!(
"config: routing.tasks.{} references unknown model id '{}'",
task,
id
);
}
}
}
Ok(()) Ok(())
} }
+51 -1
View File
@@ -132,7 +132,7 @@ fn default_quant() -> String {
} }
/// Live state of one conversion job. /// Live state of one conversion job.
#[derive(Clone, Debug, Serialize)] #[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Job { pub struct Job {
pub id: String, pub id: String,
pub repo: String, pub repo: String,
@@ -162,14 +162,60 @@ pub struct JobStore {
/// Cancel handles, keyed by job id (not serialized). Notifying aborts the /// Cancel handles, keyed by job id (not serialized). Notifying aborts the
/// converter process and cleans up its partial output. /// converter process and cleans up its partial output.
cancels: Mutex<HashMap<String, Arc<Notify>>>, cancels: Mutex<HashMap<String, Arc<Notify>>>,
/// When set, every state change lands in this JSON file, so the job list —
/// and its Register buttons — survives a gateway restart.
persist: Mutex<Option<std::path::PathBuf>>,
last_save: Mutex<Option<std::time::Instant>>,
} }
impl JobStore { impl JobStore {
pub fn new() -> Arc<Self> { pub fn new() -> Arc<Self> {
Arc::new(Self::default()) Arc::new(Self::default())
} }
/// Load previously saved jobs and start persisting to `path`. A job that
/// was `running` when the gateway died is settled as an error — its process
/// is gone; the file it produced (if complete) is still on disk.
pub fn attach_persistence(&self, path: std::path::PathBuf) {
if let Ok(text) = std::fs::read_to_string(&path) {
if let Ok(mut jobs) = serde_json::from_str::<Vec<Job>>(&text) {
let mut g = self.jobs.lock().unwrap();
for j in jobs.drain(..) {
let mut j = j;
if j.state == "running" {
j.state = "error".into();
j.log.push("✗ interrupted by a gateway restart".into());
j.finished = Some(now());
}
g.insert(j.id.clone(), j);
}
}
}
*self.persist.lock().unwrap() = Some(path);
}
fn save(&self, force: bool) {
let Some(path) = self.persist.lock().unwrap().clone() else {
return;
};
if !force {
let mut last = self.last_save.lock().unwrap();
if let Some(t) = *last {
if t.elapsed() < std::time::Duration::from_secs(2) {
return;
}
}
*last = Some(std::time::Instant::now());
}
let jobs: Vec<Job> = self.jobs.lock().unwrap().values().cloned().collect();
if let Ok(json) = serde_json::to_vec(&jobs) {
let tmp = path.with_extension("json.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &path);
}
}
}
fn insert(&self, job: Job) { fn insert(&self, job: Job) {
self.jobs.lock().unwrap().insert(job.id.clone(), job); self.jobs.lock().unwrap().insert(job.id.clone(), job);
self.save(true);
} }
fn set_cancel(&self, id: &str, n: Arc<Notify>) { fn set_cancel(&self, id: &str, n: Arc<Notify>) {
self.cancels.lock().unwrap().insert(id.to_string(), n); self.cancels.lock().unwrap().insert(id.to_string(), n);
@@ -215,6 +261,7 @@ impl JobStore {
j.phase = Some(phase); j.phase = Some(phase);
} }
} }
self.save(false);
} }
/// Expected total download size, once the HF tree is known. /// Expected total download size, once the HF tree is known.
fn set_total(&self, id: &str, total: u64) { fn set_total(&self, id: &str, total: u64) {
@@ -263,6 +310,7 @@ impl JobStore {
j.state = state.into(); j.state = state.into();
j.finished = Some(now()); j.finished = Some(now());
} }
self.save(true);
} }
fn finish(&self, id: &str, ok: bool) { fn finish(&self, id: &str, ok: bool) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) { if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
@@ -280,6 +328,7 @@ impl JobStore {
j.total_bytes = Some(j.total_bytes.unwrap_or(on_disk).max(on_disk)); j.total_bytes = Some(j.total_bytes.unwrap_or(on_disk).max(on_disk));
} }
} }
self.save(true);
} }
pub fn get(&self, id: &str) -> Option<Job> { pub fn get(&self, id: &str) -> Option<Job> {
self.jobs.lock().unwrap().get(id).cloned() self.jobs.lock().unwrap().get(id).cloned()
@@ -295,6 +344,7 @@ impl JobStore {
cleanup_output(&job.output); cleanup_output(&job.output);
self.cancels.lock().unwrap().remove(id); self.cancels.lock().unwrap().remove(id);
self.jobs.lock().unwrap().remove(id); self.jobs.lock().unwrap().remove(id);
self.save(true);
Ok(true) Ok(true)
} }
pub fn list(&self) -> Vec<Job> { pub fn list(&self) -> Vec<Job> {
+73
View File
@@ -0,0 +1,73 @@
//! In-memory ring of recent log lines, so the admin console can show logs to
//! users who never see the terminal (Docker, autostart, double-clicked exe).
//! Fed by a second `tracing` layer writing plain text (no ANSI) into the ring
//! while the normal stdout layer stays colored.
use std::collections::VecDeque;
use std::io;
use std::sync::{Arc, Mutex, OnceLock};
const CAP: usize = 500;
fn ring() -> &'static Arc<Mutex<VecDeque<String>>> {
static RING: OnceLock<Arc<Mutex<VecDeque<String>>>> = OnceLock::new();
RING.get_or_init(|| Arc::new(Mutex::new(VecDeque::with_capacity(CAP))))
}
/// Last `limit` lines, oldest first.
pub fn snapshot(limit: usize) -> Vec<String> {
let g = ring().lock().unwrap();
g.iter()
.skip(g.len().saturating_sub(limit))
.cloned()
.collect()
}
pub fn push_line(line: &str) {
let line = line.trim_end();
if line.is_empty() {
return;
}
let mut g = ring().lock().unwrap();
if g.len() >= CAP {
g.pop_front();
}
g.push_back(line.to_string());
}
/// `MakeWriter` for the ring layer: buffers partial writes until a newline.
#[derive(Clone, Default)]
pub struct RingMakeWriter;
pub struct RingWriter {
buf: String,
}
impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for RingMakeWriter {
type Writer = RingWriter;
fn make_writer(&'a self) -> Self::Writer {
RingWriter { buf: String::new() }
}
}
impl io::Write for RingWriter {
fn write(&mut self, data: &[u8]) -> io::Result<usize> {
self.buf.push_str(&String::from_utf8_lossy(data));
while let Some(idx) = self.buf.find('\n') {
let line: String = self.buf.drain(..=idx).collect();
push_line(&line);
}
Ok(data.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl Drop for RingWriter {
fn drop(&mut self) {
if !self.buf.is_empty() {
push_line(&self.buf.clone());
}
}
}
+52 -7
View File
@@ -9,11 +9,14 @@
#![allow(dead_code)] #![allow(dead_code)]
mod admin; mod admin;
mod bench;
mod cache; mod cache;
mod cmf_runtime; mod cmf_runtime;
mod config; mod config;
mod error; mod error;
mod import; mod import;
mod logbuf;
mod media;
mod model; mod model;
mod pipeline; mod pipeline;
mod promotion; mod promotion;
@@ -25,6 +28,7 @@ mod routing;
mod secrets; mod secrets;
mod state; mod state;
mod stats; mod stats;
mod sysmon;
use axum::{routing::get, Router}; use axum::{routing::get, Router};
use clap::Parser; use clap::Parser;
@@ -203,12 +207,23 @@ async fn main() -> anyhow::Result<()> {
if listen != original_listen { if listen != original_listen {
let _ = cfg.save(&config_path); let _ = cfg.save(&config_path);
} }
tracing_subscriber::fmt() {
.with_env_filter( use tracing_subscriber::layer::SubscriberExt;
tracing_subscriber::EnvFilter::try_from_default_env() use tracing_subscriber::util::SubscriberInitExt;
.unwrap_or_else(|_| cfg.log.level.clone().into()), let filter = tracing_subscriber::EnvFilter::try_from_default_env()
) .unwrap_or_else(|_| cfg.log.level.clone().into());
.init(); tracing_subscriber::registry()
.with(filter)
.with(tracing_subscriber::fmt::layer())
// second copy of every line, without ANSI, into the in-memory ring
// that backs the admin console's Логи view
.with(
tracing_subscriber::fmt::layer()
.with_ansi(false)
.with_writer(logbuf::RingMakeWriter),
)
.init();
}
let admin_enabled = cfg.admin.enabled; let admin_enabled = cfg.admin.enabled;
let admin_token = resolve_admin_token(&cfg, &args); let admin_token = resolve_admin_token(&cfg, &args);
@@ -216,6 +231,18 @@ async fn main() -> anyhow::Result<()> {
let state = SharedState::build(cfg, config_path.clone())?; let state = SharedState::build(cfg, config_path.clone())?;
// Open admin console reachable from other machines: fine on a laptop,
// a footgun on a VPS. Flag it for the UI banner and say it in the log.
let host_public = listen.starts_with("0.0.0.0") || listen.starts_with("[::]");
if admin_enabled && admin_token.is_empty() && host_public && !is_docker() {
state
.admin_open_public
.store(true, std::sync::atomic::Ordering::Relaxed);
tracing::warn!(
"admin console is OPEN (no token) and listening on {listen} — anyone on your network can manage this gateway; set a token via --admin-token or [admin].token if this machine is reachable from outside"
);
}
// Close the loop with the CMF format (github.com/infosave2007/cmf): install // Close the loop with the CMF format (github.com/infosave2007/cmf): install
// / update cortiq-cli and run a local `cortiq serve` if configured. Runs in // / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
// the background so startup is never blocked by a cargo install or a model // the background so startup is never blocked by a cargo install or a model
@@ -246,7 +273,25 @@ async fn main() -> anyhow::Result<()> {
let app = app.with_state(state).layer(cors); let app = app.with_state(state).layer(cors);
let listener = tokio::net::TcpListener::bind(&listen).await?; let listener = {
let mut attempt = 0u32;
loop {
match tokio::net::TcpListener::bind(&listen).await {
Ok(l) => break l,
Err(e) => {
// during self-restart the old process is still letting go of
// the port — retry for a few seconds before giving up
let retrying = std::env::var("CORTIQ_BIND_RETRY").is_ok();
if retrying && attempt < 50 {
attempt += 1;
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
continue;
}
return Err(e.into());
}
}
}
};
tracing::info!("cortiq-gateway listening on {listen}"); tracing::info!("cortiq-gateway listening on {listen}");
if admin_enabled { if admin_enabled {
let host_for_url = if listen.starts_with("0.0.0.0:") { let host_for_url = if listen.starts_with("0.0.0.0:") {
+861
View File
@@ -0,0 +1,861 @@
//! Media generation: run `cortiq imagine` (text→image, Lumina) and
//! `cortiq animate` (text→video+audio, MiniMax-H3) over local `.cmf` files as
//! tracked background jobs, the same shape as import jobs so the admin UI can
//! poll progress and fetch the result.
//!
//! These models are NOT served by `cortiq serve` — each generation is one CLI
//! run. The gateway owns the process, parses its progress, and keeps the output
//! under `<data>/media/<job>/`.
use serde::Serialize;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::process::Stdio;
use std::sync::{Arc, Mutex};
use std::time::{SystemTime, UNIX_EPOCH};
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::sync::Notify;
fn now() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// Random job id — media jobs are never deduplicated (every run is a new work).
fn gen_media_id() -> String {
let mut b = [0u8; 8];
let _ = getrandom::getrandom(&mut b);
b.iter().map(|x| format!("{x:02x}")).collect()
}
/// What a `.cmf` under models_dir can do, decided by `cortiq info`.
#[derive(Clone, Copy, PartialEq, Eq, Debug, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum Modality {
Chat,
Image,
Video,
}
/// `cortiq info` output → modality. The arch line names the family; the media
/// packs also carry their pipeline components (DiT/VAE/vocoder) in the text.
fn classify_info(info: &str) -> Modality {
let lower = info.to_ascii_lowercase();
let arch = lower
.lines()
.find_map(|l| l.trim().strip_prefix("arch:"))
.unwrap_or("")
.trim()
.to_string();
// generic arch suffixes first (e.g. "lumina2-image", "mmh3-video"), then
// known family names as a fallback for packs without the suffix
if arch.contains("video") || arch.contains("mmh3") || arch.contains("minimax") {
return Modality::Video;
}
if arch.contains("image") || arch.contains("lumina") || arch.contains("diffusion") {
return Modality::Image;
}
if lower.contains("text-to-video") {
return Modality::Video;
}
if lower.contains("text-to-image") {
return Modality::Image;
}
Modality::Chat
}
/// One local `.cmf` with its detected modality.
#[derive(Clone, Serialize)]
pub struct MediaModel {
pub name: String, // file stem
pub path: String,
pub size_bytes: u64,
pub modality: Modality,
}
#[derive(Default)]
struct InfoCache(Mutex<HashMap<String, (u64, u64, Modality)>>); // path → (mtime, size, modality)
/// Modality of one `.cmf`, via `cortiq info` — cached by (path, mtime, size),
/// so the process runs once per file version.
pub async fn classify_file(path: &Path, cortiq_bin: &str) -> Modality {
static CACHE: std::sync::OnceLock<InfoCache> = std::sync::OnceLock::new();
let cache = CACHE.get_or_init(InfoCache::default);
let size = crate::import::path_size(path);
let mtime = std::fs::metadata(path)
.ok()
.and_then(|m| m.modified().ok())
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_secs())
.unwrap_or(0);
let key = path.to_string_lossy().to_string();
if let Some(k) = cache
.0
.lock()
.unwrap()
.get(&key)
.filter(|(m, s, _)| *m == mtime && *s == size)
.map(|(_, _, k)| *k)
{
return k;
}
let output = tokio::process::Command::new(cortiq_bin)
.arg("info")
.arg(path)
.output()
.await;
let text = output
.map(|o| {
format!(
"{}{}",
String::from_utf8_lossy(&o.stdout),
String::from_utf8_lossy(&o.stderr)
)
})
.unwrap_or_default();
let k = classify_info(&text);
cache.0.lock().unwrap().insert(key, (mtime, size, k));
k
}
/// Scan models_dir and classify every `.cmf` (file or sharded dir).
pub async fn list_media_models(models_dir: &str, cortiq_bin: &str) -> Vec<MediaModel> {
let mut out = Vec::new();
let Ok(rd) = std::fs::read_dir(models_dir) else {
return out;
};
for entry in rd.filter_map(|e| e.ok()) {
let path = entry.path();
let name = path
.file_name()
.unwrap_or_default()
.to_string_lossy()
.to_string();
if !name.to_ascii_lowercase().ends_with(".cmf") {
continue;
}
let size = crate::import::path_size(&path);
if size == 0 {
continue; // empty or still being written
}
let modality = classify_file(&path, cortiq_bin).await;
out.push(MediaModel {
name: name
.trim_end_matches(".cmf")
.trim_end_matches(".CMF")
.to_string(),
path: path.to_string_lossy().to_string(),
size_bytes: size,
modality,
});
}
out.sort_by(|a, b| a.name.cmp(&b.name));
out
}
/// Parameters for one generation, straight from the playground form.
#[derive(Clone, Debug, serde::Deserialize)]
pub struct MediaParams {
pub model: String, // file stem under models_dir
pub kind: String, // image | video
pub prompt: String,
#[serde(default)]
pub width: Option<u32>,
#[serde(default)]
pub height: Option<u32>,
#[serde(default)]
pub steps: Option<u32>,
#[serde(default)]
pub cfg: Option<f32>, // image only
#[serde(default)]
pub seed: Option<u64>,
#[serde(default)]
pub frames: Option<u32>, // video only
#[serde(default)]
pub quality: Option<u32>, // video only: AVI JPEG quality
/// Optional first/last keyframe for video, base64-encoded binary P6 PPM.
#[serde(default)]
pub first_frame_b64: Option<String>,
#[serde(default)]
pub last_frame_b64: Option<String>,
}
/// Live state of one generation.
#[derive(Clone, Serialize, serde::Deserialize)]
pub struct MediaJob {
pub id: String,
pub kind: String, // image | video
pub model: String,
pub prompt: String,
pub width: u32,
pub height: u32,
pub steps: u32,
pub seed: u64,
pub frames: Option<u32>,
pub state: String, // queued | running | done | error | cancelled
pub progress: Option<f32>,
pub phase: Option<String>,
pub log: Vec<String>,
pub started: u64,
pub finished: Option<u64>,
/// Files this job produced: name → relative kind ("image", "video", "audio", "mp4").
pub outputs: Vec<String>,
pub out_bytes: Option<u64>,
}
/// Everything needed to actually launch a queued job when its turn comes.
struct Prepared {
bin: String,
args: Vec<String>,
gpu: bool,
dir: PathBuf,
out_path: PathBuf,
kind: String,
}
#[derive(Default)]
pub struct MediaStore {
jobs: Mutex<HashMap<String, MediaJob>>,
cancels: Mutex<HashMap<String, Arc<Notify>>>,
/// One generation runs at a time — these models take the whole GPU/RAM
/// budget — but new requests queue up instead of being rejected.
queue: Mutex<std::collections::VecDeque<String>>,
current: Mutex<Option<String>>,
prepared: Mutex<HashMap<String, Prepared>>,
persist: Mutex<Option<PathBuf>>,
}
impl MediaStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
/// Load saved generations (their files are still under media_dir) and start
/// persisting. Jobs that were queued/running when the gateway died settle
/// as errors — the process is gone.
pub fn attach_persistence(&self, path: PathBuf) {
if let Ok(text) = std::fs::read_to_string(&path) {
if let Ok(mut jobs) = serde_json::from_str::<Vec<MediaJob>>(&text) {
let mut g = self.jobs.lock().unwrap();
for mut j in jobs.drain(..) {
if j.state == "running" || j.state == "queued" {
j.state = "error".into();
j.log.push("✗ interrupted by a gateway restart".into());
j.finished = Some(now());
}
g.insert(j.id.clone(), j);
}
}
}
*self.persist.lock().unwrap() = Some(path);
}
fn save(&self) {
let Some(path) = self.persist.lock().unwrap().clone() else {
return;
};
let jobs: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
if let Ok(json) = serde_json::to_vec(&jobs) {
let tmp = path.with_extension("json.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &path);
}
}
}
/// How many jobs sit in the queue ahead of `id` (0 = next up).
pub fn queue_position(&self, id: &str) -> Option<usize> {
self.queue.lock().unwrap().iter().position(|x| x == id)
}
pub fn list(&self) -> Vec<MediaJob> {
let mut v: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
v.sort_by_key(|j| std::cmp::Reverse(j.started));
v.truncate(20);
v
}
pub fn get(&self, id: &str) -> Option<MediaJob> {
self.jobs.lock().unwrap().get(id).cloned()
}
pub fn cancel(self: &Arc<Self>, id: &str, media_dir: &Path) -> bool {
let state = self
.jobs
.lock()
.unwrap()
.get(id)
.map(|j| j.state.clone())
.unwrap_or_default();
match state.as_str() {
"queued" => {
self.queue.lock().unwrap().retain(|x| x != id);
self.prepared.lock().unwrap().remove(id);
let _ = std::fs::remove_dir_all(media_dir.join(id));
self.set_cancelled(id);
true
}
"running" => {
if let Some(n) = self.cancels.lock().unwrap().get(id) {
n.notify_one();
true
} else {
false
}
}
_ => false,
}
}
/// Remove a settled job and its output directory. Err = still running.
pub fn delete(&self, id: &str, media_dir: &Path) -> Result<bool, String> {
let job = self.jobs.lock().unwrap().get(id).cloned();
let Some(job) = job else { return Ok(false) };
if job.state == "running" || job.state == "queued" {
return Err("cancel the running generation first".into());
}
let _ = std::fs::remove_dir_all(media_dir.join(id));
self.jobs.lock().unwrap().remove(id);
self.save();
Ok(true)
}
fn push_line(&self, id: &str, line: String) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
j.log.push(line);
let n = j.log.len();
if n > 120 {
j.log.drain(0..n - 120);
}
}
}
fn set_progress(&self, id: &str, frac: f32, phase: Option<String>) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
let f = frac.clamp(0.0, 1.0);
if j.progress.is_none() || f > j.progress.unwrap() {
j.progress = Some(f);
}
if let Some(p) = phase {
j.phase = Some(p);
}
}
}
fn finish(self: &Arc<Self>, id: &str, ok: bool, outputs: Vec<String>, out_bytes: u64) {
let mut g = self.jobs.lock().unwrap();
if let Some(j) = g.get_mut(id) {
if j.state == "cancelled" {
drop(g);
self.after_slot_freed(id);
return;
}
j.finished = Some(now());
j.state = if ok { "done" } else { "error" }.into();
if ok {
j.progress = Some(1.0);
j.outputs = outputs;
j.out_bytes = Some(out_bytes);
}
}
drop(g);
self.after_slot_freed(id);
}
fn set_cancelled(self: &Arc<Self>, id: &str) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
j.state = "cancelled".into();
j.finished = Some(now());
}
self.after_slot_freed(id);
}
fn after_slot_freed(self: &Arc<Self>, id: &str) {
{
let mut cur = self.current.lock().unwrap();
if cur.as_deref() == Some(id) {
*cur = None;
}
}
self.cancels.lock().unwrap().remove(id);
self.save();
promote_next(self.clone());
}
}
/// Start the next queued generation, if the slot is free.
fn promote_next(store: Arc<MediaStore>) {
let next = {
let mut cur = store.current.lock().unwrap();
if cur.is_some() {
return;
}
let Some(id) = store.queue.lock().unwrap().pop_front() else {
return;
};
*cur = Some(id.clone());
id
};
let Some(p) = store.prepared.lock().unwrap().remove(&next) else {
// job was cancelled while queued and slipped through — free the slot
*store.current.lock().unwrap() = None;
return;
};
if let Some(j) = store.jobs.lock().unwrap().get_mut(&next) {
j.state = "running".into();
j.phase = Some("starting".into());
j.started = now();
}
store.save();
let cancel = Arc::new(Notify::new());
store
.cancels
.lock()
.unwrap()
.insert(next.clone(), cancel.clone());
let s = store.clone();
tokio::spawn(async move {
run_generation(
s, next, p.bin, p.args, p.gpu, p.dir, p.out_path, p.kind, cancel,
)
.await;
});
}
/// Progress markers in `cortiq imagine`/`animate` stdout. The exact wording has
/// varied between releases, so accept anything that carries an X/Y pair near a
/// step/frame word, a bare percentage, or the converter-style `@PROGRESS`.
fn parse_media_progress(line: &str) -> Option<(f32, Option<String>)> {
let l = line.trim();
if let Some(rest) = l.strip_prefix("@PROGRESS ") {
let mut it = rest.splitn(2, ' ');
let frac: f32 = it.next()?.trim().parse().ok()?;
let phase = it
.next()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty());
return Some((frac.clamp(0.0, 1.0), phase));
}
let lower = l.to_ascii_lowercase();
// "step 12/30", "denoise 3/4", "frame 7/39" — any word followed by x/y
for word in ["step", "frame", "sigma", "denois"] {
if let Some(pos) = lower.find(word) {
let tail = &lower[pos..];
if let Some((x, y)) = extract_ratio(tail) {
return Some((x / y, Some(l.to_string())));
}
}
}
// bare "42%" (avoid matching sizes like "42%faster")
if let Some(idx) = lower.find('%') {
let head: String = lower[..idx]
.chars()
.rev()
.take_while(|c| c.is_ascii_digit() || *c == '.')
.collect();
let num: String = head.chars().rev().collect();
if let Ok(p) = num.parse::<f32>() {
if (0.0..=100.0).contains(&p) {
return Some((p / 100.0, None));
}
}
}
None
}
/// First "N/M" pair in the string with M > 0.
fn extract_ratio(s: &str) -> Option<(f32, f32)> {
let bytes = s.as_bytes();
let mut i = 0;
while i < bytes.len() {
if bytes[i].is_ascii_digit() {
let start = i;
while i < bytes.len() && bytes[i].is_ascii_digit() {
i += 1;
}
if i < bytes.len() && bytes[i] == b'/' {
let x: f32 = s[start..i].parse().ok()?;
let ystart = i + 1;
let mut j = ystart;
while j < bytes.len() && bytes[j].is_ascii_digit() {
j += 1;
}
if j > ystart {
if let Ok(y) = s[ystart..j].parse::<f32>() {
if y > 0.0 && x <= y {
return Some((x, y));
}
}
}
}
} else {
i += 1;
}
}
None
}
/// Root for generated files: `<data>/media`.
pub fn media_dir() -> PathBuf {
crate::config::data_dir().join("media")
}
const B64: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
/// Minimal base64 decode (standard alphabet, padding optional) — the keyframes
/// are the only base64 input the gateway takes; not worth a dependency.
fn b64_decode(s: &str) -> Option<Vec<u8>> {
let mut lut = [255u8; 256];
for (i, c) in B64.iter().enumerate() {
lut[*c as usize] = i as u8;
}
let mut out = Vec::with_capacity(s.len() * 3 / 4);
let mut buf = 0u32;
let mut bits = 0u32;
for c in s.bytes() {
if c == b'=' || c == b'\n' || c == b'\r' {
continue;
}
let v = lut[c as usize];
if v == 255 {
return None;
}
buf = (buf << 6) | v as u32;
bits += 6;
if bits >= 8 {
bits -= 8;
out.push((buf >> bits) as u8);
}
}
Some(out)
}
/// Kick off a generation. Returns the job id, or an error when another one is
/// already running / the model file is missing.
pub fn start_generation(
store: Arc<MediaStore>,
cmf: &crate::config::CmfCfg,
p: MediaParams,
) -> Result<String, String> {
if p.prompt.trim().is_empty() {
return Err("empty prompt".into());
}
let kind = p.kind.as_str();
if kind != "image" && kind != "video" {
return Err(format!("unknown kind '{kind}'"));
}
let model_path = Path::new(&cmf.models_dir).join(format!("{}.cmf", p.model));
if !model_path.exists() {
return Err(format!("model file not found: {}", model_path.display()));
}
let id = gen_media_id();
let dir = media_dir().join(&id);
if let Err(e) = std::fs::create_dir_all(&dir) {
return Err(format!("cannot create media dir {}: {e}", dir.display()));
}
// Optional keyframes (video): decode to PPM files next to the output.
let mut first_frame: Option<PathBuf> = None;
let mut last_frame: Option<PathBuf> = None;
for (b64, slot, name) in [
(&p.first_frame_b64, &mut first_frame, "first.ppm"),
(&p.last_frame_b64, &mut last_frame, "last.ppm"),
] {
if let Some(data) = b64.as_deref().filter(|s| !s.is_empty()) {
let bytes = b64_decode(data).ok_or("invalid keyframe base64")?;
if !bytes.starts_with(b"P6") {
return Err("keyframe must be a binary P6 PPM".into());
}
let path = dir.join(name);
std::fs::write(&path, bytes).map_err(|e| format!("write keyframe: {e}"))?;
*slot = Some(path);
}
}
let width = p.width.unwrap_or(512).clamp(64, 2048);
let height = p
.height
.unwrap_or(if kind == "video" { 288 } else { 512 })
.clamp(64, 2048);
let steps = p
.steps
.unwrap_or(if kind == "video" { 4 } else { 30 })
.clamp(1, 200);
let seed = p.seed.unwrap_or(42);
let frames = p.frames.unwrap_or(39).clamp(1, 1000);
let out_name = if kind == "image" {
"out.ppm"
} else {
"out.avi"
};
let out_path = dir.join(out_name);
let mut args: Vec<String> = if kind == "image" {
vec![
"imagine".into(),
model_path.to_string_lossy().to_string(),
"--prompt".into(),
p.prompt.clone(),
"--width".into(),
width.to_string(),
"--height".into(),
height.to_string(),
"--steps".into(),
steps.to_string(),
"--cfg".into(),
format!("{}", p.cfg.unwrap_or(4.0)),
"--seed".into(),
seed.to_string(),
"--out".into(),
out_path.to_string_lossy().to_string(),
]
} else {
let mut a = vec![
"animate".into(),
model_path.to_string_lossy().to_string(),
"--prompt".into(),
p.prompt.clone(),
"--width".into(),
width.to_string(),
"--height".into(),
height.to_string(),
"--frames".into(),
frames.to_string(),
"--steps".into(),
steps.to_string(),
"--seed".into(),
seed.to_string(),
"--quality".into(),
p.quality.unwrap_or(92).clamp(10, 100).to_string(),
"--out".into(),
out_path.to_string_lossy().to_string(),
];
if let Some(f) = &first_frame {
a.push("--first-frame".into());
a.push(f.to_string_lossy().to_string());
}
if let Some(f) = &last_frame {
a.push("--last-frame".into());
a.push(f.to_string_lossy().to_string());
}
a
};
// keep the model path first for readability in logs
let _ = &mut args;
store.jobs.lock().unwrap().insert(
id.clone(),
MediaJob {
id: id.clone(),
kind: kind.to_string(),
model: p.model.clone(),
prompt: p.prompt.clone(),
width,
height,
steps,
seed,
frames: (kind == "video").then_some(frames),
state: "queued".into(),
progress: None,
phase: None,
log: vec![format!("→ cortiq {}", args.join(" "))],
started: now(),
finished: None,
outputs: Vec::new(),
out_bytes: None,
},
);
store.prepared.lock().unwrap().insert(
id.clone(),
Prepared {
bin: cmf.cortiq_bin.clone(),
args,
gpu: cmf.gpu,
dir,
out_path,
kind: kind.to_string(),
},
);
store.queue.lock().unwrap().push_back(id.clone());
store.save();
promote_next(store.clone());
Ok(id)
}
#[allow(clippy::too_many_arguments)]
async fn run_generation(
store: Arc<MediaStore>,
id: String,
bin: String,
args: Vec<String>,
gpu: bool,
dir: PathBuf,
out_path: PathBuf,
kind: String,
cancel: Arc<Notify>,
) {
let mut cmd = tokio::process::Command::new(&bin);
cmd.args(&args)
.env("CMF_GPU", if gpu { "1" } else { "0" })
.stdout(Stdio::piped())
.stderr(Stdio::piped());
let mut child = match cmd.spawn() {
Ok(c) => c,
Err(e) => {
store.push_line(&id, format!("✗ failed to start cortiq: {e}"));
store.finish(&id, false, Vec::new(), 0);
return;
}
};
let mut readers = Vec::new();
if let Some(o) = child.stdout.take() {
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), o)));
}
if let Some(e) = child.stderr.take() {
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), e)));
}
let status = tokio::select! {
s = child.wait() => s,
_ = cancel.notified() => {
let _ = child.start_kill();
let _ = child.wait().await;
for r in readers { r.abort(); }
let _ = std::fs::remove_dir_all(&dir);
store.push_line(&id, "✗ cancelled".into());
store.set_cancelled(&id);
return;
}
};
for r in readers {
let _ = r.await;
}
let ok = status.map(|s| s.success()).unwrap_or(false);
if !ok || !out_path.exists() {
store.push_line(&id, "✗ generation failed — see the log above".into());
store.finish(&id, false, Vec::new(), 0);
return;
}
let mut outputs = vec![if kind == "image" { "image" } else { "video" }.to_string()];
if kind == "video" {
// `animate` writes a .wav next to the .avi
if dir.join("out.wav").exists() {
outputs.push("audio".into());
}
// opportunistic browser-friendly remux when ffmpeg is present
if let Ok(p) = which_ffmpeg() {
store.push_line(&id, "→ remuxing to mp4 (ffmpeg found)".into());
let mp4 = dir.join("out.mp4");
let mut f = tokio::process::Command::new(p);
f.arg("-y").arg("-i").arg(dir.join("out.avi"));
if dir.join("out.wav").exists() {
f.arg("-i")
.arg(dir.join("out.wav"))
.args(["-map", "0:v", "-map", "1:a", "-c:a", "aac"]);
}
f.args(["-c:v", "libx264", "-pix_fmt", "yuv420p"]).arg(&mp4);
f.stdout(Stdio::null()).stderr(Stdio::null());
if matches!(f.status().await, Ok(s) if s.success()) && mp4.exists() {
outputs.push("mp4".into());
}
}
}
let bytes = crate::import::path_size(&out_path);
store.push_line(
&id,
format!("✓ done — {}", crate::import::format_bytes(bytes)),
);
store.finish(&id, true, outputs, bytes);
}
/// Strip ANSI color escapes that `tracing` writes even into a pipe.
fn strip_ansi(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut chars = s.chars();
while let Some(c) = chars.next() {
if c == '\u{1b}' {
for d in chars.by_ref() {
if d.is_ascii_alphabetic() {
break;
}
}
} else {
out.push(c);
}
}
out
}
async fn read_lines(store: Arc<MediaStore>, id: String, stream: impl tokio::io::AsyncRead + Unpin) {
let mut lines = BufReader::new(stream).lines();
while let Ok(Some(l)) = lines.next_line().await {
let l = strip_ansi(l.trim());
if l.is_empty() {
continue;
}
// the Metal backend logs one line per pipeline — dozens per run, all identical
if l.contains("Metal GPU path") {
continue;
}
if let Some((frac, phase)) = parse_media_progress(&l) {
store.set_progress(&id, frac, phase);
} else {
store.push_line(&id, l);
}
}
}
fn which_ffmpeg() -> Result<String, ()> {
for p in [
"ffmpeg",
"/usr/local/bin/ffmpeg",
"/opt/homebrew/bin/ffmpeg",
] {
if std::process::Command::new(p)
.arg("-version")
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
{
return Ok(p.to_string());
}
}
Err(())
}
/// (relative file, content-type) for an output kind of a finished job.
pub fn output_file(kind: &str) -> Option<(&'static str, &'static str)> {
match kind {
"image" => Some(("out.ppm", "image/x-portable-pixmap")),
"video" => Some(("out.avi", "video/x-msvideo")),
"audio" => Some(("out.wav", "audio/wav")),
"mp4" => Some(("out.mp4", "video/mp4")),
_ => None,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn progress_parses_common_shapes() {
assert_eq!(parse_media_progress("step 15/30").unwrap().0, 0.5);
assert_eq!(
parse_media_progress("Denoising step 3/4 ...").unwrap().0,
0.75
);
assert_eq!(parse_media_progress("frame 39/39 decoded").unwrap().0, 1.0);
assert_eq!(parse_media_progress("@PROGRESS 0.25 vae").unwrap().0, 0.25);
assert!(parse_media_progress("loaded 2361 tensors").is_none());
}
#[test]
fn classify_by_arch() {
assert_eq!(classify_info(" Arch: lumina2\n"), Modality::Image);
assert_eq!(classify_info(" Arch: mmh3\n"), Modality::Video);
assert_eq!(classify_info(" Arch: qwen3\n"), Modality::Chat);
}
#[test]
fn b64_roundtrip() {
assert_eq!(b64_decode("UDYgd2g=").unwrap(), b"P6 wh");
assert!(b64_decode("!!").is_none());
}
}
+56 -4
View File
@@ -232,8 +232,16 @@ impl Pipeline {
match live.router.route(&text, prof).await { match live.router.route(&text, prof).await {
Ok(Some(dec)) => { Ok(Some(dec)) => {
let src = dec.source.clone(); let src = dec.source.clone();
let tier = dec.complexity_tier.clone(); let (cands, task_rule) = live
(dec, src, live.routing.candidates(&tier)) .routing
.candidates_for(&dec.task_label, &dec.complexity_tier);
// surface which rule chose the head of the chain
let src = if task_rule {
format!("{src}+task-rule")
} else {
src
};
(dec, src, cands)
} }
Ok(None) | Err(_) => { Ok(None) | Err(_) => {
tracing::warn!( tracing::warn!(
@@ -542,6 +550,11 @@ impl Pipeline {
cost_usd: 0.0, cost_usd: 0.0,
failover: attempt > 0, failover: attempt > 0,
}; };
// for the client-facing usage estimate when the provider
// sends none: the prompt size is known here, the reply is
// measured inside the tap
let prompt_chars: usize =
req.messages.iter().map(|m| m.content.chars().count()).sum();
let tapped = tap_stream( let tapped = tap_stream(
stream, stream,
state.stats.clone(), state.stats.clone(),
@@ -551,6 +564,7 @@ impl Pipeline {
protocol, protocol,
directive, directive,
started, started,
prompt_chars,
); );
return Ok((info, Box::pin(tapped))); return Ok((info, Box::pin(tapped)));
} }
@@ -722,6 +736,7 @@ fn tap_stream(
protocol: String, protocol: String,
directive: String, directive: String,
started: Instant, started: Instant,
prompt_chars: usize,
) -> impl futures::Stream<Item = Result<bytes::Bytes>> { ) -> impl futures::Stream<Item = Result<bytes::Bytes>> {
use futures::StreamExt; use futures::StreamExt;
async_stream::stream! { async_stream::stream! {
@@ -731,12 +746,15 @@ fn tap_stream(
let mut completion_tokens = 0u32; let mut completion_tokens = 0u32;
let mut got_usage = false; let mut got_usage = false;
let mut delta_chars = 0usize; let mut delta_chars = 0usize;
let mut delta_chunks = 0u32;
let mut errored: Option<String> = None; let mut errored: Option<String> = None;
while let Some(item) = inner.next().await { while let Some(item) = inner.next().await {
let mut done_marker = false;
match &item { match &item {
Ok(bytes) => { Ok(bytes) => {
if let Ok(s) = std::str::from_utf8(bytes) { if let Ok(s) = std::str::from_utf8(bytes) {
done_marker = s.contains("data: [DONE]");
buf.push_str(s); buf.push_str(s);
while let Some(idx) = buf.find("\n\n") { while let Some(idx) = buf.find("\n\n") {
let evt: String = buf.drain(..idx + 2).collect(); let evt: String = buf.drain(..idx + 2).collect();
@@ -765,6 +783,7 @@ fn tap_stream(
v["choices"][0]["delta"]["content"].as_str() v["choices"][0]["delta"]["content"].as_str()
{ {
delta_chars += c.chars().count(); delta_chars += c.chars().count();
delta_chunks += 1;
} }
} }
} }
@@ -774,12 +793,45 @@ fn tap_stream(
} }
Err(e) => errored = Some(e.to_string()), Err(e) => errored = Some(e.to_string()),
} }
// The local cortiq server (and some providers) never send a usage
// chunk. Clients still deserve token counts, so ahead of [DONE] we
// inject one final OpenAI-shaped usage event with the gateway's
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
// mistakes it for the tokenizer's truth.
if done_marker && !got_usage {
let est_completion = if delta_chunks > 0 {
delta_chunks
} else {
(delta_chars / 4) as u32
};
let est_prompt = (prompt_chars / 4) as u32;
let evt = serde_json::json!({
"object": "chat.completion.chunk",
"model": info.selected_model,
"choices": [],
"usage": {
"prompt_tokens": est_prompt,
"completion_tokens": est_completion,
"total_tokens": est_prompt + est_completion,
},
"cortiq_estimated": true,
});
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
prompt_tokens = est_prompt;
completion_tokens = est_completion;
got_usage = true;
}
yield item; yield item;
} }
if !got_usage && completion_tokens == 0 { if !got_usage && completion_tokens == 0 {
// estimate when the server didn't return usage (~4 chars per token) // stream ended without a [DONE] marker — estimate for the stats
completion_tokens = (delta_chars / 4) as u32; completion_tokens = if delta_chunks > 0 {
delta_chunks
} else {
(delta_chars / 4) as u32
};
prompt_tokens = (prompt_chars / 4) as u32;
} }
let cost = provider.price(prompt_tokens, completion_tokens); let cost = provider.price(prompt_tokens, completion_tokens);
stats.record(RequestRecord { stats.record(RequestRecord {
+17
View File
@@ -71,6 +71,23 @@ impl RouterClient {
/// Classify text. Returns `Ok(None)` on network error/timeout — /// Classify text. Returns `Ok(None)` on network error/timeout —
/// the caller treats this as "router unavailable" and uses the default tier. /// the caller treats this as "router unavailable" and uses the default tier.
/// `GET /v1/taxonomies` — the account's taxonomies with their label sets.
/// `Ok(None)` when the router is disabled or has no key configured.
pub async fn taxonomies(&self) -> anyhow::Result<Option<serde_json::Value>> {
if self.url.is_empty() {
return Ok(None);
}
let mut req = self.http.get(format!("{}/v1/taxonomies", self.url));
if let Some(k) = &self.api_key {
req = req.bearer_auth(k);
}
let resp = req.send().await?;
if !resp.status().is_success() {
anyhow::bail!("router /v1/taxonomies: HTTP {}", resp.status());
}
Ok(Some(resp.json().await?))
}
pub async fn route(&self, text: &str, profile: &str) -> anyhow::Result<Option<RouteDecision>> { pub async fn route(&self, text: &str, profile: &str) -> anyhow::Result<Option<RouteDecision>> {
let mut body = serde_json::json!({ let mut body = serde_json::json!({
"input": { "text": text }, "input": { "text": text },
+92 -17
View File
@@ -3,38 +3,70 @@
use crate::config::{Config, TierTargets}; use crate::config::{Config, TierTargets};
fn to_list(t: &TierTargets) -> Vec<String> {
match t {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
}
}
pub struct RoutingTable { pub struct RoutingTable {
tiers: std::collections::HashMap<String, Vec<String>>, tiers: std::collections::HashMap<String, Vec<String>>,
/// Per-task rules — a matching rule's models lead the candidate chain.
tasks: std::collections::HashMap<String, Vec<String>>,
default: String, default: String,
} }
impl RoutingTable { impl RoutingTable {
pub fn from_config(cfg: &Config) -> Self { pub fn from_config(cfg: &Config) -> Self {
let tiers = cfg
.routing
.tiers
.iter()
.map(|(tier, targets)| {
let list = match targets {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
(tier.clone(), list)
})
.collect();
Self { Self {
tiers, tiers: cfg
.routing
.tiers
.iter()
.map(|(tier, targets)| (tier.clone(), to_list(targets)))
.collect(),
tasks: cfg
.routing
.tasks
.iter()
.map(|(task, targets)| (task.clone(), to_list(targets)))
.filter(|(_, v)| !v.is_empty())
.collect(),
default: cfg.routing.default.clone(), default: cfg.routing.default.clone(),
} }
} }
/// Candidates for `(task, tier)`, in preference order, deduplicated:
/// the task rule's models first (when one exists), then the tier chain —
/// so a task pin adds preference without ever weakening failover.
/// Returns `(candidates, task_rule_matched)`.
pub fn candidates_for(&self, task: &str, tier: &str) -> (Vec<String>, bool) {
let mut out: Vec<String> = Vec::new();
let matched = if let Some(rule) = self.tasks.get(task) {
out.extend(rule.iter().cloned());
true
} else {
false
};
let tier_chain = self
.tiers
.get(tier)
.cloned()
.unwrap_or_else(|| vec![self.default.clone()]);
for id in tier_chain {
if !out.contains(&id) {
out.push(id);
}
}
out.retain(|id| !id.is_empty());
(out, matched)
}
/// Candidates for the given complexity tier, in preference order. /// Candidates for the given complexity tier, in preference order.
/// If the tier is not defined, the sole candidate is `default`. /// If the tier is not defined, the sole candidate is `default`.
pub fn candidates(&self, tier: &str) -> Vec<String> { pub fn candidates(&self, tier: &str) -> Vec<String> {
self.tiers self.candidates_for("", tier).0
.get(tier)
.cloned()
.unwrap_or_else(|| vec![self.default.clone()])
} }
/// Default model (used when the router is unavailable). /// Default model (used when the router is unavailable).
@@ -46,3 +78,46 @@ impl RoutingTable {
// TODO(v0.2): cost_aware mode and circuit breaker — pick the cheapest available // TODO(v0.2): cost_aware mode and circuit breaker — pick the cheapest available
// model no lower than min_class[tier], respecting max_cost_usd_per_request, // model no lower than min_class[tier], respecting max_cost_usd_per_request,
// and skip models whose circuit breaker is open. // and skip models whose circuit breaker is open.
#[cfg(test)]
mod tests {
use super::*;
fn table(tasks: &[(&str, &[&str])], tiers: &[(&str, &[&str])], default: &str) -> RoutingTable {
RoutingTable {
tiers: tiers
.iter()
.map(|(k, v)| (k.to_string(), v.iter().map(|s| s.to_string()).collect()))
.collect(),
tasks: tasks
.iter()
.map(|(k, v)| (k.to_string(), v.iter().map(|s| s.to_string()).collect()))
.collect(),
default: default.to_string(),
}
}
#[test]
fn task_rule_leads_tier_chain_follows() {
let t = table(
&[("translation", &["local"])],
&[("low", &["cheap", "local"]), ("high", &["big"])],
"cheap",
);
let (c, matched) = t.candidates_for("translation", "high");
assert!(matched);
assert_eq!(c, vec!["local", "big"]);
// dedup: rule model already in the tier chain appears once, first
let (c, _) = t.candidates_for("translation", "low");
assert_eq!(c, vec!["local", "cheap"]);
}
#[test]
fn no_rule_falls_back_to_tier_then_default() {
let t = table(&[], &[("low", &["a"])], "d");
assert_eq!(t.candidates_for("qa", "low").0, vec!["a"]);
let (c, matched) = t.candidates_for("qa", "unknown-tier");
assert!(!matched);
assert_eq!(c, vec!["d"]);
}
}
+40
View File
@@ -71,6 +71,17 @@ pub struct AppState {
pub stats: Arc<Stats>, pub stats: Arc<Stats>,
pub promotion: Arc<crate::promotion::Promotion>, pub promotion: Arc<crate::promotion::Promotion>,
pub imports: Arc<crate::import::JobStore>, pub imports: Arc<crate::import::JobStore>,
/// Image/video generation jobs (cortiq imagine / animate).
pub media: Arc<crate::media::MediaStore>,
/// One-click local model benchmarks (cortiq bench --json).
pub bench: Arc<crate::bench::BenchStore>,
/// Host metrics sampler (CPU / RAM / disk) for the dashboard.
pub sysmon: Arc<crate::sysmon::SysMon>,
/// Newest cortiq-gateway version on crates.io, when newer than ours.
pub gateway_update: Arc<std::sync::Mutex<Option<String>>>,
/// Admin console reachable from other machines without a token — the
/// footgun banner flag (set at startup, shown by the SPA).
pub admin_open_public: std::sync::atomic::AtomicBool,
/// Managed local CMF model server (install/update/spawn lifecycle + status). /// Managed local CMF model server (install/update/spawn lifecycle + status).
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>, pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
pub cache: Arc<SemanticCache>, pub cache: Arc<SemanticCache>,
@@ -113,6 +124,30 @@ impl SharedState {
let stats = Stats::new(&cfg.stats); let stats = Stats::new(&cfg.stats);
let promotion = crate::promotion::Promotion::new(promotion_cfg(&cfg.shadow)); let promotion = crate::promotion::Promotion::new(promotion_cfg(&cfg.shadow));
let imports = crate::import::JobStore::new(); let imports = crate::import::JobStore::new();
let media = crate::media::MediaStore::new();
let bench = crate::bench::BenchStore::new();
// jobs survive restarts: files under <data>/, reconciled on load
let data = crate::config::data_dir();
let _ = std::fs::create_dir_all(&data);
imports.attach_persistence(data.join("jobs-import.json"));
media.attach_persistence(data.join("jobs-media.json"));
let sysmon = crate::sysmon::spawn(cfg.cmf.models_dir.clone());
let gateway_update: Arc<std::sync::Mutex<Option<String>>> =
Arc::new(std::sync::Mutex::new(None));
{
// check crates.io for a newer gateway once at start and twice a day
let slot = gateway_update.clone();
tokio::spawn(async move {
loop {
if let Some(latest) = crate::cmf_runtime::latest_gateway_version().await {
let newer =
crate::cmf_runtime::version_lt(env!("CARGO_PKG_VERSION"), &latest);
*slot.lock().unwrap() = newer.then_some(latest);
}
tokio::time::sleep(std::time::Duration::from_secs(12 * 3600)).await;
}
});
}
let cmf = crate::cmf_runtime::CmfRuntime::new(); let cmf = crate::cmf_runtime::CmfRuntime::new();
let cache = SemanticCache::new(&cfg.cache); let cache = SemanticCache::new(&cfg.cache);
let router_status = RouterLastStatus::default(); let router_status = RouterLastStatus::default();
@@ -122,6 +157,11 @@ impl SharedState {
stats, stats,
promotion, promotion,
imports, imports,
media,
bench,
sysmon,
gateway_update,
admin_open_public: std::sync::atomic::AtomicBool::new(false),
cmf, cmf,
cache, cache,
secrets, secrets,
+86
View File
@@ -0,0 +1,86 @@
//! Host metrics sampler for the admin dashboard: CPU load, RAM, free disk at
//! the models dir, and the gateway's own RSS. One background task refreshes a
//! snapshot every 2 s; the API endpoint only reads it.
use serde::Serialize;
use std::sync::{Arc, Mutex};
#[derive(Clone, Default, Serialize)]
pub struct Snapshot {
pub ts: u64,
/// Whole-machine CPU usage 0..100 (average over all cores).
pub cpu_percent: f32,
pub cores: usize,
pub load1: f64,
pub ram_total: u64,
pub ram_used: u64,
/// Disk that holds models_dir.
pub disk_total: u64,
pub disk_free: u64,
/// Resident memory of the gateway process itself.
pub process_rss: u64,
}
pub struct SysMon {
snap: Mutex<Snapshot>,
}
impl SysMon {
pub fn snapshot(&self) -> Snapshot {
self.snap.lock().unwrap().clone()
}
}
fn now() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// Start the sampler. `models_dir` picks which disk's free space is reported.
pub fn spawn(models_dir: String) -> Arc<SysMon> {
let mon = Arc::new(SysMon {
snap: Mutex::new(Snapshot::default()),
});
let ret = mon.clone();
tokio::task::spawn_blocking(move || {
use sysinfo::{Disks, ProcessesToUpdate, System};
let mut sys = System::new();
let pid = sysinfo::Pid::from_u32(std::process::id());
let models_abs = std::fs::canonicalize(&models_dir)
.unwrap_or_else(|_| std::path::PathBuf::from(&models_dir));
loop {
sys.refresh_cpu_usage();
sys.refresh_memory();
let _ = sys.refresh_processes(ProcessesToUpdate::Some(&[pid]), true);
let disks = Disks::new_with_refreshed_list();
// the disk whose mount point is the longest prefix of models_dir
let mut best: Option<(usize, u64, u64)> = None;
for d in disks.list() {
let mp = d.mount_point();
if models_abs.starts_with(mp) {
let len = mp.as_os_str().len();
if best.map(|(l, _, _)| len > l).unwrap_or(true) {
best = Some((len, d.total_space(), d.available_space()));
}
}
}
let (disk_total, disk_free) = best.map(|(_, t, f)| (t, f)).unwrap_or((0, 0));
let snap = Snapshot {
ts: now(),
cpu_percent: sys.global_cpu_usage(),
cores: sys.cpus().len(),
load1: System::load_average().one,
ram_total: sys.total_memory(),
ram_used: sys.used_memory(),
disk_total,
disk_free,
process_rss: sys.process(pid).map(|p| p.memory()).unwrap_or(0),
};
*mon.snap.lock().unwrap() = snap;
std::thread::sleep(std::time::Duration::from_secs(2));
}
});
ret
}
+72
View File
@@ -585,6 +585,78 @@ body.modal-open { overflow: hidden; }
/* Featured gallery: the list grows as new models are published, so give it more /* Featured gallery: the list grows as new models are published, so give it more
room than the search grid before it starts scrolling. */ room than the search grid before it starts scrolling. */
.featured-section .hf-grid { max-height: min(62vh, 760px); } .featured-section .hf-grid { max-height: min(62vh, 760px); }
/* Routing: per-task rule rows */
.taskrow { align-items: flex-start; }
.taskname { min-width: 190px; line-height: 1.35; }
.taskname .mono { font-size: 10.5px; opacity: .75; }
.chips-empty { padding: 4px 0; font-style: italic; }
/* Shell notices (open admin warning, update available) */
.notice {
display: flex; align-items: center; gap: 8px;
padding: 8px 18px; font-size: 13px;
border-bottom: 1px solid var(--border-soft, #232833);
}
.notice.warn { background: color-mix(in srgb, #e5a54b 12%, var(--bg)); }
.notice.info { background: color-mix(in srgb, var(--accent) 10%, var(--bg)); }
.notice .icon-btn { margin-left: auto; }
/* Dashboard host metrics */
.sysgrid { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 16px; }
.sysitem .k { color: var(--muted); font-size: 12px; }
.sysitem .v { font-weight: 600; }
.sysbar {
height: 8px; border-radius: 6px; overflow: hidden; margin: 6px 0;
background: var(--bg-2, #12151c); border: 1px solid var(--border, #2a2f3a);
}
.sysbar i { display: block; height: 100%; background: var(--accent, #5b8def); }
.sysbar i.bad { background: #e5534b; }
/* Log viewer */
.logs-view { max-height: 68vh; }
/* Chat dialogue bubbles */
.chat-log { max-height: 56vh; overflow-y: auto; display: flex; flex-direction: column; gap: 8px; }
.chat-msg { display: flex; }
.chat-msg.user { justify-content: flex-end; }
.chat-bubble {
max-width: 85%; padding: 8px 12px; border-radius: 12px;
white-space: pre-wrap; word-break: break-word; font-size: 13.5px;
background: var(--bg-2, #12151c); border: 1px solid var(--border-soft, #232833);
}
.chat-msg.user .chat-bubble {
background: color-mix(in srgb, var(--accent) 16%, var(--bg-2));
border-color: color-mix(in srgb, var(--accent) 35%, transparent);
}
/* Generation gallery */
.gal-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(110px, 1fr)); gap: 8px; }
.gal-item {
position: relative; cursor: pointer; border: 1px solid var(--border, #2a2f3a);
border-radius: 8px; overflow: hidden; background: var(--bg-2, #12151c);
}
.gal-item:hover { border-color: var(--accent, #5b8def); }
.gal-thumb { width: 100%; display: block; min-height: 64px; }
.gal-video { display: flex; align-items: center; justify-content: center; font-size: 28px; height: 72px; }
.gal-cap { font-size: 10px; color: var(--muted); padding: 3px 6px; }
.gal-del {
position: absolute; top: 2px; right: 2px; z-index: 2;
border: 0; border-radius: 50%; width: 18px; height: 18px; line-height: 16px;
background: rgba(0,0,0,.55); color: #fff; cursor: pointer; font-size: 12px;
}
/* Bench + badges */
.bench-row { gap: 10px; padding: 6px 0; border-top: 1px solid var(--border-soft, #232833); }
.bench-row:first-of-type { border-top: 0; }
.badge.warn { background: rgba(229,165,75,.15); color: #e5a54b; }
.snippet { max-height: 260px; }
/* Playground media tabs: generated image / video frame */
.media-out { display: flex; flex-direction: column; gap: 8px; }
.media-canvas {
max-width: 100%; height: auto; border-radius: 8px;
border: 1px solid var(--border, #2a2f3a); background: #000;
}
/* Live download progress inside a featured card (empty until a job runs) */ /* Live download progress inside a featured card (empty until a job runs) */
.card-prog:empty { display: none; } .card-prog:empty { display: none; }
.card-prog { margin-top: 6px; } .card-prog { margin-top: 6px; }
+70 -2
View File
@@ -68,6 +68,7 @@ export const api = {
// routing // routing
getRouting: () => j("GET", "/routing"), getRouting: () => j("GET", "/routing"),
routingLabels: () => j("GET", "/routing/labels"),
putRouting: (b) => j("PUT", "/routing", b), putRouting: (b) => j("PUT", "/routing", b),
// protocols // protocols
@@ -118,6 +119,21 @@ export const api = {
registerImport: (job) => registerImport: (job) =>
j("POST", "/import/" + encodeURIComponent(job) + "/register"), j("POST", "/import/" + encodeURIComponent(job) + "/register"),
// media generation (cortiq imagine / animate over local .cmf)
mediaModels: () => j("GET", "/media/models"),
mediaGenerate: (b) => j("POST", "/media", b),
mediaJobs: () => j("GET", "/media"),
mediaJob: (id) => j("GET", "/media/" + encodeURIComponent(id)),
mediaCancel: (id) => j("POST", "/media/" + encodeURIComponent(id) + "/cancel"),
mediaDelete: (id) => j("DELETE", "/media/" + encodeURIComponent(id)),
// system / logs / lifecycle
system: () => j("GET", "/system"),
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
restart: () => j("POST", "/restart"),
benchStart: (model) => j("POST", "/bench", { model }),
benchList: () => j("GET", "/bench"),
// first-run onboarding // first-run onboarding
setupStatus: () => j("GET", "/setup/status"), setupStatus: () => j("GET", "/setup/status"),
// Connects every finished download as a local model unless register === false. // Connects every finished download as a local model unless register === false.
@@ -134,6 +150,21 @@ export const api = {
), ),
}; };
// Raw-body POST (no JSON encoding) — the config import takes TOML text as-is.
export async function postRaw(path, text) {
const tok = getToken();
const headers = { "content-type": "application/toml" };
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch(BASE + path, { method: "POST", headers, body: text });
const body = await r.text();
if (!r.ok) {
let msg = body;
try { msg = JSON.parse(body).error?.message || body; } catch { /* raw */ }
throw new Error(msg || r.statusText);
}
return body ? JSON.parse(body) : null;
}
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk, // Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
// returns the routing decision read from X-Cortiq-* response headers. // returns the routing decision read from X-Cortiq-* response headers.
export async function testStream(body, onDelta) { export async function testStream(body, onDelta) {
@@ -155,6 +186,10 @@ export async function testStream(body, onDelta) {
const reader = r.body.getReader(); const reader = r.body.getReader();
const dec = new TextDecoder(); const dec = new TextDecoder();
let buf = ""; let buf = "";
const t0 = performance.now();
let ttftMs = null; // time to first content token
let usage = null; // OpenAI-style final-chunk usage, when the provider sends it
let usageEstimated = false; // gateway-injected estimate rather than tokenizer truth
for (;;) { for (;;) {
const { done, value } = await reader.read(); const { done, value } = await reader.read();
if (done) break; if (done) break;
@@ -167,8 +202,16 @@ export async function testStream(body, onDelta) {
const data = line.slice(5).trim(); const data = line.slice(5).trim();
if (!data || data === "[DONE]") continue; if (!data || data === "[DONE]") continue;
try { try {
const d = JSON.parse(data).choices?.[0]?.delta?.content; const chunk = JSON.parse(data);
if (d) onDelta(d); if (chunk.usage) {
usage = chunk.usage;
usageEstimated = !!chunk.cortiq_estimated;
}
const d = chunk.choices?.[0]?.delta?.content;
if (d) {
if (ttftMs == null) ttftMs = Math.round(performance.now() - t0);
onDelta(d);
}
} catch { } catch {
/* ignore malformed chunk */ /* ignore malformed chunk */
} }
@@ -182,5 +225,30 @@ export async function testStream(body, onDelta) {
selected_model: h.get("x-cortiq-selected-model") || "", selected_model: h.get("x-cortiq-selected-model") || "",
route_source: h.get("x-cortiq-route-source") || "", route_source: h.get("x-cortiq-route-source") || "",
cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"), cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"),
usage,
usage_estimated: usageEstimated,
ttft_ms: ttftMs,
}; };
} }
// Fetch a media job's output with the admin token and hand back an object URL
// (plain <img src>/<video src> cannot carry the Authorization header).
export async function mediaFileUrl(job, kind) {
const tok = getToken();
const headers = {};
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
if (!r.ok) throw new Error((await r.text()) || r.statusText);
return { url: URL.createObjectURL(await r.blob()), size: +(r.headers.get("content-length") || 0) };
}
// Same fetch, raw bytes — for outputs the browser cannot display natively
// (P6 PPM, MJPEG AVI) that we decode in JS.
export async function mediaFileBytes(job, kind) {
const tok = getToken();
const headers = {};
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
if (!r.ok) throw new Error((await r.text()) || r.statusText);
return r.arrayBuffer();
}
+24 -1
View File
@@ -11,6 +11,7 @@ import { renderProtocols } from "./views/protocols.js";
import { renderKeys } from "./views/keys.js"; import { renderKeys } from "./views/keys.js";
import { renderPlayground } from "./views/playground.js"; import { renderPlayground } from "./views/playground.js";
import { renderSettings } from "./views/settings.js"; import { renderSettings } from "./views/settings.js";
import { renderLogs } from "./views/logs.js";
import { renderOnboarding } from "./views/onboarding.js"; import { renderOnboarding } from "./views/onboarding.js";
const ROUTES = [ const ROUTES = [
@@ -21,6 +22,7 @@ const ROUTES = [
{ id: "protocols", icon: "⇆", view: renderProtocols }, { id: "protocols", icon: "⇆", view: renderProtocols },
{ id: "keys", icon: "🔑", view: renderKeys }, { id: "keys", icon: "🔑", view: renderKeys },
{ id: "playground", icon: "✦", view: renderPlayground }, { id: "playground", icon: "✦", view: renderPlayground },
{ id: "logs", icon: "📜", view: renderLogs },
{ id: "settings", icon: "⚙", view: renderSettings }, { id: "settings", icon: "⚙", view: renderSettings },
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true }, { id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
]; ];
@@ -52,6 +54,27 @@ const FAVICON =
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="%233b82f6"/><stop offset=".5" stop-color="%238b5cf6"/><stop offset="1" stop-color="%2306b6d4"/></linearGradient></defs><rect width="32" height="32" rx="8" fill="url(%23g)"/><path d="M9 16h14M17 10l6 6-6 6" stroke="white" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>' '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="%233b82f6"/><stop offset=".5" stop-color="%238b5cf6"/><stop offset="1" stop-color="%2306b6d4"/></linearGradient></defs><rect width="32" height="32" rx="8" fill="url(%23g)"/><path d="M9 16h14M17 10l6 6-6 6" stroke="white" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>'
); );
// One-line banners under the topbar: open-admin warning, update available.
function shellNotices() {
const host = h("div", {});
const m = appState.meta || {};
if (m.admin_open_public && !sessionStorage.getItem("hide_open_admin")) {
const close = h("button", { class: "icon-btn", title: "×", onClick: (e) => { sessionStorage.setItem("hide_open_admin", "1"); e.target.closest(".notice").remove(); } }, "×");
host.appendChild(
h("div", { class: "notice warn" }, "⚠ " + t("notice.openAdmin"), " ",
h("a", { href: "#/settings" }, t("nav.settings") + " →"), close)
);
}
if (m.update_available) {
host.appendChild(
h("div", { class: "notice info" },
"⬆ " + t("notice.update", { v: m.update_available }),
" ", h("code", { class: "mono" }, "cargo install cortiq-gateway"))
);
}
return host;
}
function sidebar(activeId) { function sidebar(activeId) {
const visibleRoutes = ROUTES.filter((r) => !r.hidden); const visibleRoutes = ROUTES.filter((r) => !r.hidden);
return h( return h(
@@ -221,7 +244,7 @@ function renderShell() {
{ class: "app" }, { class: "app" },
sidebar(route.id), sidebar(route.id),
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }), h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" })) h("main", { class: "main" }, topbar(route.id), shellNotices(), h("div", { class: "content", id: "view" }))
) )
); );
renderView(); renderView();
+95
View File
@@ -468,4 +468,99 @@ export default {
"onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen", "onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen",
"onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.", "onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.",
"onboarding.unregistered": "Ebenfalls abgemeldet: {ids}", "onboarding.unregistered": "Ebenfalls abgemeldet: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Bild",
"pg.tab.video": "Video",
"pg.generate": "Generieren",
"pg.generating": "Wird generiert…",
"pg.genFailed": "Generierung fehlgeschlagen — siehe Log.",
"pg.result": "Ergebnis",
"pg.genMetrics": "Generierungsmetriken",
"pg.imagePromptPlaceholder": "ein Rotfuchs im Schnee bei Sonnenuntergang, fotorealistisch, detailliertes Fell",
"pg.videoPromptPlaceholder": "ein Corgi mit Kochmütze wirft einen Pfannkuchen, Brutzeln und ein fröhliches Bellen",
"pg.imageEmpty": "Das generierte Bild erscheint hier.",
"pg.videoEmpty": "Der generierte Clip (mit Ton) erscheint hier.",
"pg.width": "Breite",
"pg.height": "Höhe",
"pg.steps": "Schritte",
"pg.seed": "Seed",
"pg.randomSeed": "Zufälliger Seed",
"pg.cfgHint": "Guidance-Stärke; ≤ 1 schaltet CFG ab und halbiert die Arbeit.",
"pg.frames": "Frames",
"pg.framesHint": "Bei 24 fps; auf das Raster des Modells gerundet.",
"pg.quality": "JPEG-Qualität",
"pg.videoSizeHint": "Vielfache von 32; die trainierte kurze Kante ist 768.",
"pg.videoStepsHint": "Die Turbo-LoRA ist auf 4 Schritte trainiert.",
"pg.firstFrame": "Erstes Frame (optional)",
"pg.lastFrame": "Letztes Frame (optional)",
"pg.keyframeHint": "Macht den Lauf zu Keyframe-to-Video.",
"pg.resolution": "Auflösung",
"pg.genTime": "Generierungszeit",
"pg.timePerStep": "Pro Schritt",
"pg.timePerFrame": "Pro Frame",
"pg.fileSize": "Dateigröße",
"pg.duration": "Dauer",
"pg.play": "Abspielen",
"pg.noMediaModels": "Noch kein lokales {kind}-Modell. Lade eines (z. B. aus der infosave-Sammlung) unter Import herunter:",
"pg.tokSpeed": "Geschwindigkeit",
"pg.ttft": "Erstes Token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Logs",
"logs.subtitle": "Jüngstes Gateway-Log — das, was das Terminal zeigen würde.",
"logs.filterPlaceholder": "filtern…",
"logs.pause": "Pause",
"logs.resume": "Fortsetzen",
"logs.empty": "Nichts gefunden.",
"notice.openAdmin": "Die Konsole ist ohne Token offen und von anderen Rechnern erreichbar. Setze einen Token, wenn diese Maschine nicht privat ist.",
"notice.update": "Update verfügbar: v{v}.",
"dash.sys.title": "System",
"dash.sys.cores": "Kerne",
"dash.sys.disk": "Festplatte (Modelle)",
"dash.sys.free": "frei",
"dash.sys.gateway": "Gateway",
"fit.disk": "zu wenig Speicherplatz",
"fit.diskTitle": "Der Download ist größer als der freie Platz auf der Modell-Festplatte.",
"fit.ram": "passt evtl. nicht in den RAM",
"fit.ramTitle": "Das Modell ist nahe an oder größer als der RAM dieser Maschine — es läuft womöglich nicht.",
"pg.eta": "übrig",
"pg.min": "Min",
"pg.queued": "In der Warteschlange — Position {n}. Startet, sobald die aktuelle Generierung fertig ist.",
"pg.gallery": "Letzte Generierungen",
"pg.galDeleteConfirm": "Diese Generierung samt Dateien löschen?",
"pg.clearChat": "Leeren",
"keys.snip.title": "Deine App verbinden",
"keys.snip.hint": "Füge eines der Beispiele in dein Projekt ein — nur der Schlüssel-Platzhalter muss ersetzt werden.",
"keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
"keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
"keys.snip.keyPh": "dein API-Schlüssel",
"settings.restart": "Gateway neu starten",
"settings.restartConfirm": "Gateway neu starten? Laufende Anfragen brechen ab; lokale Modellserver starten ebenfalls neu.",
"settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
"bench.title": "Geschwindigkeit lokaler Modelle",
"bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
"bench.run": "Messen",
"bench.running": "misst…",
"bench.failed": "fehlgeschlagen",
"bench.none": "Noch keine lokalen .cmf-Chat-Modelle.",
// task-based routing
"task.chitchat": "Smalltalk",
"task.code": "Code",
"task.creative-writing": "Kreatives Schreiben",
"task.extraction": "Datenextraktion",
"task.math": "Mathematik",
"task.qa": "Fragen & Antworten",
"task.summarization": "Zusammenfassung",
"task.translation": "Übersetzung",
"routing.tasks.title": "Aufgabenbasiertes Routing",
"routing.tasks.hint": "Die Modelle einer Regel kommen für ihre Aufgabe zuerst; die Komplexitätskette bleibt als Fallback. Leer = nur nach Komplexität.",
"routing.tasks.inherit": "nach Komplexität (keine Regel)",
"routing.tasks.share": "{p}% der Anfragen (7 T)",
"routing.tasks.fromRouter": "Labels live vom Router",
"routing.tasks.builtin": "Router nicht erreichbar — eingebauter Labelsatz",
"routing.tasks.addCustom": "Aufgabe hinzufügen",
"routing.tasks.addPlaceholder": "eigenes Label…",
}; };
+95
View File
@@ -461,4 +461,99 @@ export default {
"onboarding.deleteFile": "Delete the .cmf file from disk", "onboarding.deleteFile": "Delete the .cmf file from disk",
"onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.", "onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.",
"onboarding.unregistered": "Also unregistered: {ids}", "onboarding.unregistered": "Also unregistered: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Image",
"pg.tab.video": "Video",
"pg.generate": "Generate",
"pg.generating": "Generating…",
"pg.genFailed": "Generation failed — see the log.",
"pg.result": "Result",
"pg.genMetrics": "Generation metrics",
"pg.imagePromptPlaceholder": "a red fox sitting in snow at sunset, photorealistic, detailed fur",
"pg.videoPromptPlaceholder": "a corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark",
"pg.imageEmpty": "The generated image will appear here.",
"pg.videoEmpty": "The generated clip (with audio) will appear here.",
"pg.width": "Width",
"pg.height": "Height",
"pg.steps": "Steps",
"pg.seed": "Seed",
"pg.randomSeed": "Random seed",
"pg.cfgHint": "Guidance scale; ≤ 1 disables CFG and halves the work.",
"pg.frames": "Frames",
"pg.framesHint": "At 24 fps; snapped to the model's grid.",
"pg.quality": "JPEG quality",
"pg.videoSizeHint": "Multiples of 32; the trained short edge is 768.",
"pg.videoStepsHint": "The Turbo LoRA is trained for 4 steps.",
"pg.firstFrame": "First frame (optional)",
"pg.lastFrame": "Last frame (optional)",
"pg.keyframeHint": "Turns the run into keyframe-to-video.",
"pg.resolution": "Resolution",
"pg.genTime": "Generation time",
"pg.timePerStep": "Per step",
"pg.timePerFrame": "Per frame",
"pg.fileSize": "File size",
"pg.duration": "Duration",
"pg.play": "Play",
"pg.noMediaModels": "No local {kind} model yet. Download one (e.g. from the infosave collection) in Import:",
"pg.tokSpeed": "Speed",
"pg.ttft": "First token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Logs",
"logs.subtitle": "Recent gateway log — what the terminal would show.",
"logs.filterPlaceholder": "filter…",
"logs.pause": "Pause",
"logs.resume": "Resume",
"logs.empty": "Nothing matches.",
"notice.openAdmin": "The admin console is open without a token and reachable from other machines. Set a token if this box is not private.",
"notice.update": "Update available: v{v}.",
"dash.sys.title": "System",
"dash.sys.cores": "cores",
"dash.sys.disk": "Disk (models)",
"dash.sys.free": "free",
"dash.sys.gateway": "gateway",
"fit.disk": "not enough disk",
"fit.diskTitle": "The download is larger than the free space on the models disk.",
"fit.ram": "may not fit in RAM",
"fit.ramTitle": "The model is close to or larger than this machine's RAM — it may not run.",
"pg.eta": "left",
"pg.min": "min",
"pg.queued": "In queue — position {n}. It starts as soon as the current generation finishes.",
"pg.gallery": "Recent generations",
"pg.galDeleteConfirm": "Delete this generation and its files?",
"pg.clearChat": "Clear",
"keys.snip.title": "Connect your app",
"keys.snip.hint": "Paste one of these into your project — only the API key placeholder needs replacing.",
"keys.snip.withKey": "Snippets below include the key you just created.",
"keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
"keys.snip.keyPh": "your API key",
"settings.restart": "Restart gateway",
"settings.restartConfirm": "Restart the gateway? Requests in flight will be dropped; local model servers restart too.",
"settings.restarting": "Restarting — the page reloads when the gateway is back…",
"bench.title": "Local model speed",
"bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
"bench.run": "Measure",
"bench.running": "measuring…",
"bench.failed": "failed",
"bench.none": "No local .cmf chat models yet.",
// task-based routing
"task.chitchat": "Small talk",
"task.code": "Code",
"task.creative-writing": "Creative writing",
"task.extraction": "Data extraction",
"task.math": "Math",
"task.qa": "Q&A",
"task.summarization": "Summarization",
"task.translation": "Translation",
"routing.tasks.title": "Task-based routing",
"routing.tasks.hint": "A rule's models go first for that task; the complexity chain below still applies as fallback. Empty = route by complexity only.",
"routing.tasks.inherit": "by complexity (no rule)",
"routing.tasks.share": "{p}% of requests (7d)",
"routing.tasks.fromRouter": "labels live from the router",
"routing.tasks.builtin": "router unreachable — built-in label set",
"routing.tasks.addCustom": "Add task",
"routing.tasks.addPlaceholder": "custom label…",
}; };
+95
View File
@@ -468,4 +468,99 @@ export default {
"onboarding.deleteFile": "Eliminar el archivo .cmf del disco", "onboarding.deleteFile": "Eliminar el archivo .cmf del disco",
"onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.", "onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.",
"onboarding.unregistered": "También se dio de baja: {ids}", "onboarding.unregistered": "También se dio de baja: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Imagen",
"pg.tab.video": "Vídeo",
"pg.generate": "Generar",
"pg.generating": "Generando…",
"pg.genFailed": "La generación falló — revisa el registro.",
"pg.result": "Resultado",
"pg.genMetrics": "Métricas de generación",
"pg.imagePromptPlaceholder": "un zorro rojo en la nieve al atardecer, fotorrealista, pelaje detallado",
"pg.videoPromptPlaceholder": "un corgi con gorro de chef voltea una tortita, chisporroteo y un ladrido alegre",
"pg.imageEmpty": "La imagen generada aparecerá aquí.",
"pg.videoEmpty": "El clip generado (con audio) aparecerá aquí.",
"pg.width": "Ancho",
"pg.height": "Alto",
"pg.steps": "Pasos",
"pg.seed": "Seed",
"pg.randomSeed": "Seed aleatorio",
"pg.cfgHint": "Escala de guidance; ≤ 1 desactiva el CFG y reduce el trabajo a la mitad.",
"pg.frames": "Fotogramas",
"pg.framesHint": "A 24 fps; se ajusta a la rejilla del modelo.",
"pg.quality": "Calidad JPEG",
"pg.videoSizeHint": "Múltiplos de 32; el lado corto entrenado es 768.",
"pg.videoStepsHint": "La LoRA Turbo está entrenada para 4 pasos.",
"pg.firstFrame": "Primer fotograma (opcional)",
"pg.lastFrame": "Último fotograma (opcional)",
"pg.keyframeHint": "Convierte la ejecución en keyframe-to-video.",
"pg.resolution": "Resolución",
"pg.genTime": "Tiempo de generación",
"pg.timePerStep": "Por paso",
"pg.timePerFrame": "Por fotograma",
"pg.fileSize": "Tamaño del archivo",
"pg.duration": "Duración",
"pg.play": "Reproducir",
"pg.noMediaModels": "Aún no hay un modelo {kind} local. Descarga uno (p. ej. de la colección infosave) en Importar:",
"pg.tokSpeed": "Velocidad",
"pg.ttft": "Primer token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Registros",
"logs.subtitle": "Registro reciente de la pasarela — lo que mostraría la terminal.",
"logs.filterPlaceholder": "filtrar…",
"logs.pause": "Pausa",
"logs.resume": "Reanudar",
"logs.empty": "Nada coincide.",
"notice.openAdmin": "La consola está abierta sin token y accesible desde otras máquinas. Configura un token si esta máquina no es privada.",
"notice.update": "Actualización disponible: v{v}.",
"dash.sys.title": "Sistema",
"dash.sys.cores": "núcleos",
"dash.sys.disk": "Disco (modelos)",
"dash.sys.free": "libre",
"dash.sys.gateway": "pasarela",
"fit.disk": "no hay espacio suficiente",
"fit.diskTitle": "La descarga supera el espacio libre del disco de modelos.",
"fit.ram": "puede no caber en RAM",
"fit.ramTitle": "El modelo se acerca o supera la RAM de esta máquina — puede que no funcione.",
"pg.eta": "restante",
"pg.min": "min",
"pg.queued": "En cola — posición {n}. Arranca en cuanto termine la generación actual.",
"pg.gallery": "Generaciones recientes",
"pg.galDeleteConfirm": "¿Eliminar esta generación y sus archivos?",
"pg.clearChat": "Limpiar",
"keys.snip.title": "Conecta tu aplicación",
"keys.snip.hint": "Pega uno de estos ejemplos en tu proyecto — solo hay que sustituir el marcador de la clave.",
"keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
"keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
"keys.snip.keyPh": "tu clave API",
"settings.restart": "Reiniciar la pasarela",
"settings.restartConfirm": "¿Reiniciar la pasarela? Las peticiones en curso se cortarán; los servidores de modelos locales también se reinician.",
"settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
"bench.title": "Velocidad de modelos locales",
"bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
"bench.run": "Medir",
"bench.running": "midiendo…",
"bench.failed": "falló",
"bench.none": "Aún no hay modelos de chat .cmf locales.",
// task-based routing
"task.chitchat": "Charla",
"task.code": "Código",
"task.creative-writing": "Escritura creativa",
"task.extraction": "Extracción de datos",
"task.math": "Matemáticas",
"task.qa": "Preguntas y respuestas",
"task.summarization": "Resumen",
"task.translation": "Traducción",
"routing.tasks.title": "Enrutado por tarea",
"routing.tasks.hint": "Los modelos de una regla van primero para su tarea; la cadena por complejidad sigue como respaldo. Vacío = solo por complejidad.",
"routing.tasks.inherit": "por complejidad (sin regla)",
"routing.tasks.share": "{p}% de las peticiones (7 d)",
"routing.tasks.fromRouter": "etiquetas en vivo del router",
"routing.tasks.builtin": "router inaccesible — conjunto integrado",
"routing.tasks.addCustom": "Añadir tarea",
"routing.tasks.addPlaceholder": "etiqueta propia…",
}; };
+95
View File
@@ -468,4 +468,99 @@ export default {
"onboarding.deleteFile": "Supprimer le fichier .cmf du disque", "onboarding.deleteFile": "Supprimer le fichier .cmf du disque",
"onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.", "onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.",
"onboarding.unregistered": "Également désenregistré : {ids}", "onboarding.unregistered": "Également désenregistré : {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Image",
"pg.tab.video": "Vidéo",
"pg.generate": "Générer",
"pg.generating": "Génération…",
"pg.genFailed": "Échec de la génération — voir le journal.",
"pg.result": "Résultat",
"pg.genMetrics": "Métriques de génération",
"pg.imagePromptPlaceholder": "un renard roux dans la neige au coucher du soleil, photoréaliste, fourrure détaillée",
"pg.videoPromptPlaceholder": "un corgi en toque de chef fait sauter une crêpe, grésillements et aboiement joyeux",
"pg.imageEmpty": "L'image générée apparaîtra ici.",
"pg.videoEmpty": "Le clip généré (avec le son) apparaîtra ici.",
"pg.width": "Largeur",
"pg.height": "Hauteur",
"pg.steps": "Étapes",
"pg.seed": "Seed",
"pg.randomSeed": "Seed aléatoire",
"pg.cfgHint": "Force du guidance ; ≤ 1 désactive le CFG et divise le travail par deux.",
"pg.frames": "Images",
"pg.framesHint": "À 24 fps ; arrondi à la grille du modèle.",
"pg.quality": "Qualité JPEG",
"pg.videoSizeHint": "Multiples de 32 ; le petit côté entraîné est 768.",
"pg.videoStepsHint": "La LoRA Turbo est entraînée pour 4 étapes.",
"pg.firstFrame": "Première image (optionnel)",
"pg.lastFrame": "Dernière image (optionnel)",
"pg.keyframeHint": "Transforme la génération en keyframe-to-video.",
"pg.resolution": "Résolution",
"pg.genTime": "Temps de génération",
"pg.timePerStep": "Par étape",
"pg.timePerFrame": "Par image",
"pg.fileSize": "Taille du fichier",
"pg.duration": "Durée",
"pg.play": "Lire",
"pg.noMediaModels": "Pas encore de modèle {kind} local. Téléchargez-en un (par ex. de la collection infosave) dans Importer :",
"pg.tokSpeed": "Vitesse",
"pg.ttft": "Premier token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Journaux",
"logs.subtitle": "Journal récent de la passerelle — ce que montrerait le terminal.",
"logs.filterPlaceholder": "filtrer…",
"logs.pause": "Pause",
"logs.resume": "Reprendre",
"logs.empty": "Rien ne correspond.",
"notice.openAdmin": "La console est ouverte sans jeton et accessible depuis d'autres machines. Définissez un jeton si cette machine n'est pas privée.",
"notice.update": "Mise à jour disponible : v{v}.",
"dash.sys.title": "Système",
"dash.sys.cores": "cœurs",
"dash.sys.disk": "Disque (modèles)",
"dash.sys.free": "libre",
"dash.sys.gateway": "passerelle",
"fit.disk": "espace disque insuffisant",
"fit.diskTitle": "Le téléchargement dépasse l'espace libre du disque des modèles.",
"fit.ram": "risque de ne pas tenir en RAM",
"fit.ramTitle": "Le modèle approche ou dépasse la RAM de cette machine — il pourrait ne pas tourner.",
"pg.eta": "restant",
"pg.min": "min",
"pg.queued": "En file d'attente — position {n}. Démarre dès la fin de la génération en cours.",
"pg.gallery": "Générations récentes",
"pg.galDeleteConfirm": "Supprimer cette génération et ses fichiers ?",
"pg.clearChat": "Effacer",
"keys.snip.title": "Connecter votre application",
"keys.snip.hint": "Collez un de ces exemples dans votre projet — seul le espace réservé de la clé est à remplacer.",
"keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
"keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
"keys.snip.keyPh": "votre clé API",
"settings.restart": "Redémarrer la passerelle",
"settings.restartConfirm": "Redémarrer la passerelle ? Les requêtes en cours seront coupées ; les serveurs de modèles locaux redémarrent aussi.",
"settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
"bench.title": "Vitesse des modèles locaux",
"bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
"bench.run": "Mesurer",
"bench.running": "mesure…",
"bench.failed": "échec",
"bench.none": "Pas encore de modèles de chat .cmf locaux.",
// task-based routing
"task.chitchat": "Bavardage",
"task.code": "Code",
"task.creative-writing": "Écriture créative",
"task.extraction": "Extraction de données",
"task.math": "Mathématiques",
"task.qa": "Questions-réponses",
"task.summarization": "Résumé",
"task.translation": "Traduction",
"routing.tasks.title": "Routage par tâche",
"routing.tasks.hint": "Les modèles d'une règle passent en premier pour leur tâche ; la chaîne par complexité reste en secours. Vide = complexité seule.",
"routing.tasks.inherit": "par complexité (aucune règle)",
"routing.tasks.share": "{p}% des requêtes (7 j)",
"routing.tasks.fromRouter": "étiquettes reçues du routeur",
"routing.tasks.builtin": "routeur injoignable — jeu d'étiquettes intégré",
"routing.tasks.addCustom": "Ajouter une tâche",
"routing.tasks.addPlaceholder": "étiquette personnalisée…",
}; };
+95
View File
@@ -460,4 +460,99 @@ export default {
"onboarding.deleteFile": "Удалить файл .cmf с диска", "onboarding.deleteFile": "Удалить файл .cmf с диска",
"onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.", "onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.",
"onboarding.unregistered": "Также отключено из моделей: {ids}", "onboarding.unregistered": "Также отключено из моделей: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Чат",
"pg.tab.image": "Изображение",
"pg.tab.video": "Видео",
"pg.generate": "Сгенерировать",
"pg.generating": "Генерация…",
"pg.genFailed": "Генерация не удалась — смотрите лог.",
"pg.result": "Результат",
"pg.genMetrics": "Метрики генерации",
"pg.imagePromptPlaceholder": "рыжая лиса в снегу на закате, фотореалистично, детальный мех",
"pg.videoPromptPlaceholder": "корги в поварском колпаке подбрасывает блин, шипение сковороды и весёлый лай",
"pg.imageEmpty": "Сгенерированное изображение появится здесь.",
"pg.videoEmpty": "Сгенерированный ролик (со звуком) появится здесь.",
"pg.width": "Ширина",
"pg.height": "Высота",
"pg.steps": "Шаги",
"pg.seed": "Seed",
"pg.randomSeed": "Случайный seed",
"pg.cfgHint": "Сила guidance; ≤ 1 отключает CFG и вдвое ускоряет генерацию.",
"pg.frames": "Кадры",
"pg.framesHint": "При 24 fps; округляется до сетки модели.",
"pg.quality": "Качество JPEG",
"pg.videoSizeHint": "Кратно 32; обученная короткая сторона — 768.",
"pg.videoStepsHint": "Turbo LoRA обучена на 4 шага.",
"pg.firstFrame": "Первый кадр (необязательно)",
"pg.lastFrame": "Последний кадр (необязательно)",
"pg.keyframeHint": "Превращает запуск в keyframe-to-video.",
"pg.resolution": "Разрешение",
"pg.genTime": "Время генерации",
"pg.timePerStep": "На шаг",
"pg.timePerFrame": "На кадр",
"pg.fileSize": "Размер файла",
"pg.duration": "Длительность",
"pg.play": "Играть",
"pg.noMediaModels": "Локальной модели ({kind}) ещё нет. Скачайте её (например, из коллекции infosave) во вкладке Импорт:",
"pg.tokSpeed": "Скорость",
"pg.ttft": "Первый токен",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Логи",
"logs.subtitle": "Последние записи журнала шлюза — то, что показал бы терминал.",
"logs.filterPlaceholder": "фильтр…",
"logs.pause": "Пауза",
"logs.resume": "Продолжить",
"logs.empty": "Ничего не найдено.",
"notice.openAdmin": "Панель управления открыта без токена и доступна с других машин. Задайте токен, если эта машина не приватная.",
"notice.update": "Доступно обновление: v{v}.",
"dash.sys.title": "Система",
"dash.sys.cores": "ядер",
"dash.sys.disk": "Диск (модели)",
"dash.sys.free": "свободно",
"dash.sys.gateway": "шлюз",
"fit.disk": "не хватит места",
"fit.diskTitle": "Загрузка больше свободного места на диске с моделями.",
"fit.ram": "может не влезть в RAM",
"fit.ramTitle": "Модель сравнима с объёмом RAM этой машины или больше — может не запуститься.",
"pg.eta": "осталось",
"pg.min": "мин",
"pg.queued": "В очереди — позиция {n}. Стартует, как только закончится текущая генерация.",
"pg.gallery": "Последние генерации",
"pg.galDeleteConfirm": "Удалить эту генерацию и её файлы?",
"pg.clearChat": "Очистить",
"keys.snip.title": "Подключите ваше приложение",
"keys.snip.hint": "Вставьте один из примеров в свой проект — заменить нужно только плейсхолдер ключа.",
"keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
"keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
"keys.snip.keyPh": "ваш API-ключ",
"settings.restart": "Перезапустить шлюз",
"settings.restartConfirm": "Перезапустить шлюз? Текущие запросы оборвутся; серверы локальных моделей тоже перезапустятся.",
"settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
"bench.title": "Скорость локальных моделей",
"bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
"bench.run": "Замерить",
"bench.running": "замер…",
"bench.failed": "ошибка",
"bench.none": "Локальных чат-моделей .cmf пока нет.",
// task-based routing
"task.chitchat": "Светская беседа",
"task.code": "Код",
"task.creative-writing": "Творческие тексты",
"task.extraction": "Извлечение данных",
"task.math": "Математика",
"task.qa": "Вопросы и ответы",
"task.summarization": "Суммаризация",
"task.translation": "Перевод",
"routing.tasks.title": "Распределение по задачам",
"routing.tasks.hint": "Модели правила идут первыми для своей задачи; цепочка по сложности остаётся резервом. Пусто = только по сложности.",
"routing.tasks.inherit": "по сложности (без правила)",
"routing.tasks.share": "{p}% запросов (7д)",
"routing.tasks.fromRouter": "метки получены с роутера",
"routing.tasks.builtin": "роутер недоступен — встроенный набор меток",
"routing.tasks.addCustom": "Добавить задачу",
"routing.tasks.addPlaceholder": "своя метка…",
}; };
+95
View File
@@ -468,4 +468,99 @@ export default {
"onboarding.deleteFile": ".cmf dosyasını diskten sil", "onboarding.deleteFile": ".cmf dosyasını diskten sil",
"onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.", "onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.",
"onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}", "onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Sohbet",
"pg.tab.image": "Görsel",
"pg.tab.video": "Video",
"pg.generate": "Oluştur",
"pg.generating": "Oluşturuluyor…",
"pg.genFailed": "Oluşturma başarısız — günlüğe bakın.",
"pg.result": "Sonuç",
"pg.genMetrics": "Oluşturma metrikleri",
"pg.imagePromptPlaceholder": "gün batımında karda oturan kızıl tilki, fotogerçekçi, ayrıntılı tüyler",
"pg.videoPromptPlaceholder": "aşçı şapkalı bir corgi krep çeviriyor, cızırtı sesleri ve neşeli bir havlama",
"pg.imageEmpty": "Oluşturulan görsel burada görünecek.",
"pg.videoEmpty": "Oluşturulan klip (sesiyle) burada görünecek.",
"pg.width": "Genişlik",
"pg.height": "Yükseklik",
"pg.steps": "Adım",
"pg.seed": "Seed",
"pg.randomSeed": "Rastgele seed",
"pg.cfgHint": "Guidance gücü; ≤ 1 CFG'yi kapatır ve işi yarıya indirir.",
"pg.frames": "Kare",
"pg.framesHint": "24 fps'te; modelin ızgarasına yuvarlanır.",
"pg.quality": "JPEG kalitesi",
"pg.videoSizeHint": "32'nin katları; eğitilen kısa kenar 768.",
"pg.videoStepsHint": "Turbo LoRA 4 adım için eğitildi.",
"pg.firstFrame": "İlk kare (isteğe bağlı)",
"pg.lastFrame": "Son kare (isteğe bağlı)",
"pg.keyframeHint": "Çalışmayı keyframe-to-video'ya çevirir.",
"pg.resolution": "Çözünürlük",
"pg.genTime": "Oluşturma süresi",
"pg.timePerStep": "Adım başına",
"pg.timePerFrame": "Kare başına",
"pg.fileSize": "Dosya boyutu",
"pg.duration": "Süre",
"pg.play": "Oynat",
"pg.noMediaModels": "Henüz yerel bir {kind} modeli yok. İçe aktarma sekmesinden bir tane indirin (ör. infosave koleksiyonundan):",
"pg.tokSpeed": "Hız",
"pg.ttft": "İlk token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Günlükler",
"logs.subtitle": "Ağ geçidinin son günlüğü — terminalde görecekleriniz.",
"logs.filterPlaceholder": "filtrele…",
"logs.pause": "Duraklat",
"logs.resume": "Devam et",
"logs.empty": "Eşleşen yok.",
"notice.openAdmin": "Yönetim konsolu belirteçsiz açık ve diğer makinelerden erişilebilir. Bu makine özel değilse bir belirteç ayarlayın.",
"notice.update": "Güncelleme mevcut: v{v}.",
"dash.sys.title": "Sistem",
"dash.sys.cores": "çekirdek",
"dash.sys.disk": "Disk (modeller)",
"dash.sys.free": "boş",
"dash.sys.gateway": "ağ geçidi",
"fit.disk": "disk alanı yetmez",
"fit.diskTitle": "İndirme, model diskindeki boş alandan büyük.",
"fit.ram": "RAM'e sığmayabilir",
"fit.ramTitle": "Model bu makinenin RAM'ine yakın ya da ondan büyük — çalışmayabilir.",
"pg.eta": "kaldı",
"pg.min": "dk",
"pg.queued": "Kuyrukta — {n}. sırada. Mevcut üretim biter bitmez başlar.",
"pg.gallery": "Son üretimler",
"pg.galDeleteConfirm": "Bu üretim ve dosyaları silinsin mi?",
"pg.clearChat": "Temizle",
"keys.snip.title": "Uygulamanı bağla",
"keys.snip.hint": "Örneklerden birini projene yapıştır — yalnızca anahtar yer tutucusunu değiştirmen gerekir.",
"keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
"keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
"keys.snip.keyPh": "API anahtarın",
"settings.restart": "Ağ geçidini yeniden başlat",
"settings.restartConfirm": "Ağ geçidi yeniden başlatılsın mı? Devam eden istekler kesilir; yerel model sunucuları da yeniden başlar.",
"settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
"bench.title": "Yerel model hızı",
"bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
"bench.run": "Ölç",
"bench.running": "ölçülüyor…",
"bench.failed": "başarısız",
"bench.none": "Henüz yerel .cmf sohbet modeli yok.",
// task-based routing
"task.chitchat": "Sohbet",
"task.code": "Kod",
"task.creative-writing": "Yaratıcı yazarlık",
"task.extraction": "Veri çıkarma",
"task.math": "Matematik",
"task.qa": "Soru-cevap",
"task.summarization": "Özetleme",
"task.translation": "Çeviri",
"routing.tasks.title": "Göreve göre yönlendirme",
"routing.tasks.hint": "Kuralın modelleri kendi görevi için önce gelir; karmaşıklık zinciri yedek olarak kalır. Boş = yalnızca karmaşıklığa göre.",
"routing.tasks.inherit": "karmaşıklığa göre (kural yok)",
"routing.tasks.share": "isteklerin %{p}'i (7 g)",
"routing.tasks.fromRouter": "etiketler yönlendiriciden canlı",
"routing.tasks.builtin": "yönlendiriciye ulaşılamıyor — yerleşik etiket seti",
"routing.tasks.addCustom": "Görev ekle",
"routing.tasks.addPlaceholder": "özel etiket…",
}; };
+95
View File
@@ -468,4 +468,99 @@ export default {
"onboarding.deleteFile": "从磁盘删除该 .cmf 文件", "onboarding.deleteFile": "从磁盘删除该 .cmf 文件",
"onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。", "onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。",
"onboarding.unregistered": "同时已取消注册:{ids}", "onboarding.unregistered": "同时已取消注册:{ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "对话",
"pg.tab.image": "图像",
"pg.tab.video": "视频",
"pg.generate": "生成",
"pg.generating": "生成中……",
"pg.genFailed": "生成失败 — 请查看日志。",
"pg.result": "结果",
"pg.genMetrics": "生成指标",
"pg.imagePromptPlaceholder": "夕阳下雪地里的红狐狸,照片级真实感,毛发细节丰富",
"pg.videoPromptPlaceholder": "戴厨师帽的柯基翻煎饼,滋滋声和欢快的犬吠",
"pg.imageEmpty": "生成的图像将显示在这里。",
"pg.videoEmpty": "生成的短片(含音频)将显示在这里。",
"pg.width": "宽度",
"pg.height": "高度",
"pg.steps": "步数",
"pg.seed": "Seed",
"pg.randomSeed": "随机 seed",
"pg.cfgHint": "引导强度;≤ 1 关闭 CFG,工作量减半。",
"pg.frames": "帧数",
"pg.framesHint": "24 fps;按模型网格取整。",
"pg.quality": "JPEG 质量",
"pg.videoSizeHint": "32 的倍数;训练短边为 768。",
"pg.videoStepsHint": "Turbo LoRA 按 4 步训练。",
"pg.firstFrame": "首帧(可选)",
"pg.lastFrame": "末帧(可选)",
"pg.keyframeHint": "使运行变为关键帧生视频。",
"pg.resolution": "分辨率",
"pg.genTime": "生成耗时",
"pg.timePerStep": "每步",
"pg.timePerFrame": "每帧",
"pg.fileSize": "文件大小",
"pg.duration": "时长",
"pg.play": "播放",
"pg.noMediaModels": "还没有本地{kind}模型。请在“导入模型”中下载一个(例如 infosave 合集):",
"pg.tokSpeed": "速度",
"pg.ttft": "首个 token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "日志",
"logs.subtitle": "网关最近的日志 — 终端里会看到的内容。",
"logs.filterPlaceholder": "过滤……",
"logs.pause": "暂停",
"logs.resume": "继续",
"logs.empty": "没有匹配内容。",
"notice.openAdmin": "管理控制台未设置令牌即可从其他机器访问。若这台机器不是私有的,请设置令牌。",
"notice.update": "有可用更新:v{v}。",
"dash.sys.title": "系统",
"dash.sys.cores": "核",
"dash.sys.disk": "磁盘(模型)",
"dash.sys.free": "可用",
"dash.sys.gateway": "网关",
"fit.disk": "磁盘空间不足",
"fit.diskTitle": "下载大小超过模型磁盘的可用空间。",
"fit.ram": "可能超出内存",
"fit.ramTitle": "模型接近或超过本机内存 — 可能无法运行。",
"pg.eta": "剩余",
"pg.min": "分钟",
"pg.queued": "排队中 — 第 {n} 位。当前生成结束后立即开始。",
"pg.gallery": "最近的生成",
"pg.galDeleteConfirm": "删除这次生成及其文件?",
"pg.clearChat": "清空",
"keys.snip.title": "接入你的应用",
"keys.snip.hint": "把示例粘贴到你的项目里 — 只需替换密钥占位符。",
"keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
"keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
"keys.snip.keyPh": "你的 API 密钥",
"settings.restart": "重启网关",
"settings.restartConfirm": "重启网关?进行中的请求会中断;本地模型服务器也会重启。",
"settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
"bench.title": "本地模型速度",
"bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s,一次一个模型。",
"bench.run": "测速",
"bench.running": "测速中……",
"bench.failed": "失败",
"bench.none": "还没有本地 .cmf 聊天模型。",
// task-based routing
"task.chitchat": "闲聊",
"task.code": "代码",
"task.creative-writing": "创意写作",
"task.extraction": "数据提取",
"task.math": "数学",
"task.qa": "问答",
"task.summarization": "摘要",
"task.translation": "翻译",
"routing.tasks.title": "按任务路由",
"routing.tasks.hint": "规则中的模型对该任务优先;下方的复杂度链仍作为回退。留空 = 仅按复杂度。",
"routing.tasks.inherit": "按复杂度(无规则)",
"routing.tasks.share": "{p}% 的请求(7 天)",
"routing.tasks.fromRouter": "标签实时来自路由器",
"routing.tasks.builtin": "路由器不可达 — 使用内置标签集",
"routing.tasks.addCustom": "添加任务",
"routing.tasks.addPlaceholder": "自定义标签……",
}; };
+167
View File
@@ -0,0 +1,167 @@
// Browser-side codecs for cortiq's dependency-free media outputs:
// - P6 PPM → canvas (cortiq imagine links no image library)
// - MJPEG+PCM AVI → frame list + canvas player synced to the .wav
// (cortiq animate never touches ffmpeg; neither do we)
/// Parse binary P6 PPM into {width, height, rgb: Uint8Array}.
export function parsePpm(buf) {
const b = new Uint8Array(buf);
// header: "P6" <ws> width <ws> height <ws> maxval <single ws> raster
let pos = 0;
const isWs = (c) => c === 32 || c === 9 || c === 10 || c === 13;
function token() {
while (pos < b.length && (isWs(b[pos]) || b[pos] === 35 /* # comment */)) {
if (b[pos] === 35) while (pos < b.length && b[pos] !== 10) pos++;
else pos++;
}
let start = pos;
while (pos < b.length && !isWs(b[pos])) pos++;
return String.fromCharCode(...b.slice(start, pos));
}
if (token() !== "P6") throw new Error("not a P6 PPM");
const width = parseInt(token());
const height = parseInt(token());
const maxval = parseInt(token());
pos++; // single whitespace after maxval
if (!width || !height || !maxval) throw new Error("bad PPM header");
return { width, height, rgb: b.subarray(pos, pos + width * height * 3) };
}
/// Draw a parsed PPM onto a (new or given) canvas. Returns the canvas.
export function ppmToCanvas(buf, canvas) {
const { width, height, rgb } = parsePpm(buf);
const c = canvas || document.createElement("canvas");
c.width = width;
c.height = height;
const ctx = c.getContext("2d");
const img = ctx.createImageData(width, height);
for (let i = 0, j = 0; i < width * height; i++, j += 3) {
img.data[i * 4] = rgb[j];
img.data[i * 4 + 1] = rgb[j + 1];
img.data[i * 4 + 2] = rgb[j + 2];
img.data[i * 4 + 3] = 255;
}
ctx.putImageData(img, 0, 0);
return c;
}
/// Rasterize an uploaded image file to WxH (stretched, as the engine does with
/// the first frame) and return it as base64-encoded binary P6 PPM.
export async function fileToPpmB64(file, width, height) {
const bmp = await createImageBitmap(file);
const c = document.createElement("canvas");
c.width = width;
c.height = height;
const ctx = c.getContext("2d");
ctx.drawImage(bmp, 0, 0, width, height);
const img = ctx.getImageData(0, 0, width, height).data;
const header = `P6\n${width} ${height}\n255\n`;
const out = new Uint8Array(header.length + width * height * 3);
for (let i = 0; i < header.length; i++) out[i] = header.charCodeAt(i);
let p = header.length;
for (let i = 0; i < width * height; i++) {
out[p++] = img[i * 4];
out[p++] = img[i * 4 + 1];
out[p++] = img[i * 4 + 2];
}
// chunked btoa — the frame can be a few MB
let bin = "";
for (let i = 0; i < out.length; i += 0x8000)
bin += String.fromCharCode.apply(null, out.subarray(i, i + 0x8000));
return btoa(bin);
}
/// Extract JPEG frames from an MJPEG AVI (RIFF): walk chunks from 'movi',
/// collect '00dc'/'00dj' payloads as Blobs.
export function aviJpegFrames(buf) {
const b = new Uint8Array(buf);
const dv = new DataView(buf);
const fourcc = (o) => String.fromCharCode(b[o], b[o + 1], b[o + 2], b[o + 3]);
if (fourcc(0) !== "RIFF" || fourcc(8) !== "AVI ") throw new Error("not an AVI");
// find the 'movi' LIST
let movi = -1;
for (let o = 12; o + 12 <= b.length; ) {
const id = fourcc(o);
const size = dv.getUint32(o + 4, true);
if (id === "LIST" && fourcc(o + 8) === "movi") {
movi = o + 12;
break;
}
o += 8 + size + (size & 1);
}
if (movi < 0) throw new Error("no movi list");
const frames = [];
for (let o = movi; o + 8 <= b.length; ) {
const id = fourcc(o);
const size = dv.getUint32(o + 4, true);
if (id === "00dc" || id === "00dj") {
if (size > 0) frames.push(new Blob([b.subarray(o + 8, o + 8 + size)], { type: "image/jpeg" }));
} else if (id === "LIST") {
o += 12; // descend (rec lists)
continue;
} else if (id === "idx1") {
break;
}
o += 8 + size + (size & 1);
}
return frames;
}
/// Canvas player for extracted MJPEG frames at a fixed fps, synced to an
/// optional <audio> element (the .wav). Returns {canvas, play, pause, seek0, playing}.
export function mjpegPlayer(frames, fps, audio) {
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
let bitmaps = null; // decoded lazily on first play
let raf = null;
let t0 = 0;
let startFrame = 0;
const state = { playing: false };
async function decodeAll() {
if (bitmaps) return;
bitmaps = await Promise.all(frames.map((f) => createImageBitmap(f)));
if (bitmaps.length) {
canvas.width = bitmaps[0].width;
canvas.height = bitmaps[0].height;
ctx.drawImage(bitmaps[0], 0, 0);
}
}
function drawAt(i) {
if (bitmaps && bitmaps[i]) ctx.drawImage(bitmaps[i], 0, 0);
}
function tick() {
// audio clock when available (keeps A/V in sync), wall clock otherwise
const t = audio && !audio.paused ? audio.currentTime : (performance.now() - t0) / 1000;
const i = Math.min(frames.length - 1, startFrame + Math.floor(t * fps));
drawAt(i);
if (i >= frames.length - 1) {
pause();
return;
}
raf = requestAnimationFrame(tick);
}
async function play() {
await decodeAll();
state.playing = true;
startFrame = 0;
t0 = performance.now();
if (audio) {
audio.currentTime = 0;
audio.play().catch(() => {});
}
raf = requestAnimationFrame(tick);
}
function pause() {
state.playing = false;
if (raf) cancelAnimationFrame(raf);
raf = null;
if (audio) audio.pause();
}
async function seek0() {
pause();
await decodeAll();
drawAt(0);
}
return { canvas, play, pause, seek0, state, decodeAll };
}
+83
View File
@@ -27,6 +27,87 @@ export async function renderDashboard() {
let groupby = "model"; let groupby = "model";
const root = h("div"); const root = h("div");
// ── host metrics: CPU sparkline + RAM / disk bars, refreshed every 2 s ──
const sysHost = h("div", {});
const cpuHistory = []; // last ~60 samples (2 min)
function fmtG(n) {
return (n / 1e9).toFixed(1) + " GB";
}
function bar(frac, cls) {
return h("div", { class: "sysbar" }, h("i", { class: cls || "", style: `width:${Math.min(100, frac * 100)}%` }));
}
function sparkline(values, w, hgt) {
const c = document.createElement("canvas");
c.width = w;
c.height = hgt;
const ctx = c.getContext("2d");
const accent = getComputedStyle(document.documentElement).getPropertyValue("--accent").trim() || "#5b8def";
ctx.strokeStyle = accent;
ctx.lineWidth = 1.5;
ctx.beginPath();
const n = Math.max(values.length, 2);
values.forEach((v, i) => {
const x = (i / (n - 1)) * (w - 2) + 1;
const y = hgt - 2 - (Math.min(v, 100) / 100) * (hgt - 4);
if (i === 0) ctx.moveTo(x, y);
else ctx.lineTo(x, y);
});
ctx.stroke();
return c;
}
async function refreshSystem() {
let sys;
try {
sys = await api.system();
} catch {
return;
}
if (!sys.ts) return;
cpuHistory.push(sys.cpu_percent || 0);
if (cpuHistory.length > 60) cpuHistory.shift();
const ramFrac = sys.ram_total ? sys.ram_used / sys.ram_total : 0;
const diskFrac = sys.disk_total ? (sys.disk_total - sys.disk_free) / sys.disk_total : 0;
mount(
sysHost,
h(
"div",
{ class: "card" },
h("div", { class: "card-title" }, t("dash.sys.title")),
h(
"div",
{ class: "sysgrid" },
h("div", { class: "sysitem" },
h("div", { class: "flex" },
h("span", { class: "k grow" }, `CPU · ${sys.cores} ${t("dash.sys.cores")}`),
h("span", { class: "v mono" }, (sys.cpu_percent || 0).toFixed(0) + "%")),
sparkline(cpuHistory, 220, 36),
h("div", { class: "muted small" }, `load ${Number(sys.load1 || 0).toFixed(1)}`)),
h("div", { class: "sysitem" },
h("div", { class: "flex" },
h("span", { class: "k grow" }, "RAM"),
h("span", { class: "v mono" }, `${fmtG(sys.ram_used)} / ${fmtG(sys.ram_total)}`)),
bar(ramFrac, ramFrac > 0.9 ? "bad" : ""),
h("div", { class: "muted small" }, t("dash.sys.gateway") + " " + (sys.process_rss / 1e6).toFixed(0) + " MB")),
h("div", { class: "sysitem" },
h("div", { class: "flex" },
h("span", { class: "k grow" }, t("dash.sys.disk")),
h("span", { class: "v mono" }, t("dash.sys.free") + " " + fmtG(sys.disk_free)),
),
bar(diskFrac, diskFrac > 0.92 ? "bad" : ""),
h("div", { class: "muted small" }, `${fmtG(sys.disk_total - sys.disk_free)} / ${fmtG(sys.disk_total)}`))
)
)
);
}
const sysTimer = setInterval(() => {
if (!root.isConnected) {
clearInterval(sysTimer);
return;
}
refreshSystem();
}, 2000);
refreshSystem();
async function load() { async function load() {
const rangeSecs = range; const rangeSecs = range;
let stats, recent, health; let stats, recent, health;
@@ -86,6 +167,8 @@ export async function renderDashboard() {
kpi(num(tot.failovers), t("dash.kpi.failovers")) kpi(num(tot.failovers), t("dash.kpi.failovers"))
), ),
sysHost,
h( h(
"div", "div",
{ class: "grid cols-2" }, { class: "grid cols-2" },
+16 -2
View File
@@ -78,7 +78,19 @@ function descLine(desc) {
return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null; return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null;
} }
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v) }. /// Disk/RAM sanity for a model of `bytes` size against the live system
/// snapshot (ctx.sys): running a model needs it in RAM; downloading needs disk.
function fitWarnings(bytes, sys) {
const out = [];
if (!bytes || !sys) return out;
if (sys.disk_free && bytes > sys.disk_free)
out.push(h("span", { class: "badge bad", title: t("fit.diskTitle") }, "⚠ " + t("fit.disk")));
else if (sys.ram_total && bytes > sys.ram_total * 0.75)
out.push(h("span", { class: "badge warn", title: t("fit.ramTitle") }, "⚠ " + t("fit.ram")));
return out;
}
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v), sys? }.
/// Live progress lands in the per-variant host so a poll can repaint it in place, /// Live progress lands in the per-variant host so a poll can repaint it in place,
/// without re-rendering the grid under the user's cursor. /// without re-rendering the grid under the user's cursor.
export function featuredCard(m, ctx) { export function featuredCard(m, ctx) {
@@ -101,6 +113,7 @@ export function featuredCard(m, ctx) {
h("div", { class: "hf-meta" }, h("div", { class: "hf-meta" },
h("span", { class: "badge" }, badge), h("span", { class: "badge" }, badge),
descBadge(desc), descBadge(desc),
...fitWarnings(m.size_bytes, ctx.sys),
isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null), isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null),
descLine(desc), descLine(desc),
h("div", { class: "muted small" }, m.id), h("div", { class: "muted small" }, m.id),
@@ -149,7 +162,8 @@ export function featuredCard(m, ctx) {
h("div", { class: "hf-meta" }, h("div", { class: "hf-meta" },
h("span", { class: "badge" }, badge), h("span", { class: "badge" }, badge),
size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null, size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null,
descBadge(desc) descBadge(desc),
...(done0 ? [] : fitWarnings((v0 && v0.size_bytes) || m.size_bytes, ctx.sys))
), ),
descLine(desc), descLine(desc),
h("div", { class: "muted small" }, m.id), h("div", { class: "muted small" }, m.id),
+3 -1
View File
@@ -424,8 +424,10 @@ export function renderImport() {
async function renderFeatured(reload = true) { async function renderFeatured(reload = true) {
if (reload) featured = await loadFeatured(); if (reload) featured = await loadFeatured();
let sys = null;
try { sys = await api.system(); } catch { /* metrics optional */ }
featuredHosts = new Map(); featuredHosts = new Map();
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured }; const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured, sys };
mount(featuredGrid, ...(featured.length mount(featuredGrid, ...(featured.length
? featured.map((m) => featuredCard(m, ctx)) ? featured.map((m) => featuredCard(m, ctx))
: [h("div", { class: "muted pad" }, t("import.noResults"))])); : [h("div", { class: "muted pad" }, t("import.noResults"))]));
+64 -1
View File
@@ -48,6 +48,68 @@ export async function renderKeys() {
const input = h("input", { value: key, readonly: true, class: "mono" }); const input = h("input", { value: key, readonly: true, class: "mono" });
const copyBtn = h("button", { class: "btn", onClick: () => { input.select(); navigator.clipboard?.writeText(key); toast(t("common.copied"), "good"); } }, t("common.copy")); const copyBtn = h("button", { class: "btn", onClick: () => { input.select(); navigator.clipboard?.writeText(key); toast(t("common.copied"), "good"); } }, t("common.copy"));
modal(t("common.ok"), h("div", {}, h("p", {}, t("keys.created")), h("div", { class: "flex" }, input, copyBtn)), null); modal(t("common.ok"), h("div", {}, h("p", {}, t("keys.created")), h("div", { class: "flex" }, input, copyBtn)), null);
lastCreatedKey = key;
reload();
}
let lastCreatedKey = null;
// "Connect your app" — copy-paste snippets with this gateway's real URL.
// Keys are stored hashed-masked in the list, so the key itself is a
// placeholder unless one was just created on this page.
function snippetsCard(openMode) {
const base = location.origin + "/v1";
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
const snips = {
curl: `curl ${base}/chat/completions \\
-H "Authorization: Bearer ${key}" \\
-H "Content-Type: application/json" \\
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
python: `from openai import OpenAI
client = OpenAI(base_url="${base}", api_key="${key}")
resp = client.chat.completions.create(
model="cortiq-auto", # авто-роутинг; или точное имя модели
messages=[{"role": "user", "content": "Привет!"}],
)
print(resp.choices[0].message.content)`,
javascript: `import OpenAI from "openai";
const client = new OpenAI({ baseURL: "${base}", apiKey: "${key}" });
const resp = await client.chat.completions.create({
model: "cortiq-auto",
messages: [{ role: "user", content: "Привет!" }],
});
console.log(resp.choices[0].message.content);`,
};
let cur = "curl";
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
const seg = h(
"div",
{ class: "seg" },
...Object.keys(snips).map((k, i) =>
h("button", { class: i === 0 ? "active" : "", onClick: (e) => {
cur = k;
pre.textContent = snips[k];
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
e.target.classList.add("active");
} }, k)
)
);
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
copy.addEventListener("click", async () => {
await navigator.clipboard.writeText(pre.textContent);
toast(t("common.copied"), "ok");
});
return h(
"div",
{ class: "card" },
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" },
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px" }, seg, h("div", { class: "grow" }), copy),
pre
);
} }
async function reload() { async function reload() {
@@ -119,7 +181,8 @@ export async function renderKeys() {
) )
) )
) )
) ),
snippetsCard(!!data.open_mode)
); );
} }
+84
View File
@@ -0,0 +1,84 @@
// Gateway log viewer — the in-memory ring served by /admin/api/logs, for
// everyone who never sees the terminal (Docker, autostart, double-clicked exe).
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
export function renderLogs() {
const root = h("div");
let timer = null;
let paused = false;
let filter = "";
let level = "";
const pre = h("pre", { class: "conv-log logs-view" }, t("common.loading"));
const filterIn = h("input", { placeholder: t("logs.filterPlaceholder"), style: "max-width:260px" });
const levelSel = h(
"select",
{ style: "max-width:130px" },
h("option", { value: "" }, t("common.all")),
...["ERROR", "WARN", "INFO", "DEBUG"].map((l) => h("option", { value: l }, l))
);
const pauseBtn = h("button", { class: "btn ghost sm" }, "⏸ " + t("logs.pause"));
const copyBtn = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
let lines = [];
function paint() {
let shown = lines;
if (level) shown = shown.filter((l) => l.includes(` ${level} `) || l.includes(`${level} `));
if (filter) shown = shown.filter((l) => l.toLowerCase().includes(filter));
pre.textContent = shown.length ? shown.join("\n") : t("logs.empty");
if (!paused) pre.scrollTop = pre.scrollHeight;
}
async function refresh() {
if (paused) return;
try {
lines = (await api.logs(500)).lines || [];
paint();
} catch {
/* transient */
}
}
filterIn.addEventListener("input", () => {
filter = filterIn.value.trim().toLowerCase();
paint();
});
levelSel.addEventListener("change", () => {
level = levelSel.value;
paint();
});
pauseBtn.addEventListener("click", () => {
paused = !paused;
pauseBtn.textContent = (paused ? "▶ " : "⏸ ") + t(paused ? "logs.resume" : "logs.pause");
if (!paused) refresh();
});
copyBtn.addEventListener("click", async () => {
try {
await navigator.clipboard.writeText(pre.textContent);
toast(t("common.copied"), "ok");
} catch (e) {
toast(String(e.message || e), "err");
}
});
timer = setInterval(() => {
if (!root.isConnected) {
clearInterval(timer);
return;
}
refresh();
}, 2000);
refresh();
mount(
root,
h("div", { class: "view-head" }, h("h2", {}, t("nav.logs")), h("div", { class: "muted" }, t("logs.subtitle"))),
h(
"div",
{ class: "card" },
h("div", { class: "flex", style: "gap:8px; flex-wrap:wrap; margin-bottom:10px" }, filterIn, levelSel, pauseBtn, copyBtn),
pre
)
);
return root;
}
+54 -1
View File
@@ -315,10 +315,63 @@ export async function renderModels() {
) )
) )
) )
) ),
benchCard()
); );
} }
// ── one-click benchmark of local .cmf files (cortiq bench --json) ──
function benchCard() {
const host = h("div", { class: "card" }, h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
h("div", { class: "empty" }, t("common.loading")));
let timer = null;
async function paint() {
let files = [];
let runs = {};
try {
files = (await api.mediaModels()).models || [];
for (const r of (await api.benchList()).runs || []) runs[r.model] = r;
} catch {
return;
}
const chat = files.filter((m) => m.modality === "chat");
const anyRunning = Object.values(runs).some((r) => r.state === "running");
const rows = chat.map((m) => {
const r = runs[m.name];
const go = h("button", { class: "btn sm ghost", disabled: anyRunning ? true : null }, "⏱ " + t("bench.run"));
go.addEventListener("click", async () => {
try {
await api.benchStart(m.name);
paint();
if (!timer) timer = setInterval(() => {
if (!host.isConnected) { clearInterval(timer); return; }
paint();
}, 3000);
} catch (e) {
toast(String(e.message || e), "bad");
}
});
let result = h("span", { class: "muted" }, "—");
if (r?.state === "running") result = h("span", { class: "flex" }, h("span", { class: "spinner" }), " ", t("bench.running"));
else if (r?.state === "done") result = h("b", { class: "mono" }, (r.tok_s != null ? r.tok_s.toFixed(1) : "?") + " tok/s");
else if (r?.state === "error") result = h("span", { class: "err", title: (r.log || []).slice(-2).join(" · ") }, t("bench.failed"));
return h("div", { class: "flex bench-row" },
h("span", { class: "mono grow" }, m.name),
h("span", { class: "muted small" }, (m.size_bytes / 1e9).toFixed(2) + " GB"),
result,
go);
});
mount(host,
h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
...(rows.length ? rows : [h("div", { class: "empty" }, t("bench.none"))]));
if (!anyRunning && timer) { clearInterval(timer); timer = null; }
}
paint();
return host;
}
await reload(); await reload();
return root; return root;
} }
+5 -2
View File
@@ -45,6 +45,7 @@ export function renderOnboarding() {
const root = h("div", { class: "onb-wrap" }); const root = h("div", { class: "onb-wrap" });
let jobsPoll = null; let jobsPoll = null;
let jobs = []; let jobs = [];
let sysSnapshot = null; // disk/RAM для плашек "не влезет" на карточках
let jobStates = new Map(); // job id → last seen state, to spot completions let jobStates = new Map(); // job id → last seen state, to spot completions
let hosts = new Map(); // job id → card progress host (rebuilt on every render) let hosts = new Map(); // job id → card progress host (rebuilt on every render)
let jobsHost = h("div", {}); let jobsHost = h("div", {});
@@ -131,6 +132,7 @@ export function renderOnboarding() {
async function loadFeatured() { async function loadFeatured() {
featured = await fetchFeatured(); featured = await fetchFeatured();
try { sysSnapshot = await api.system(); } catch { /* optional */ }
featuredLoaded = true; featuredLoaded = true;
} }
@@ -199,7 +201,7 @@ export function renderOnboarding() {
); );
} else if (step === 1) { } else if (step === 1) {
const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip"))); const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip")));
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete }; const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete, sys: sysSnapshot };
// the list is fetched from HuggingFace — say so instead of showing a blank grid // the list is fetched from HuggingFace — say so instead of showing a blank grid
const grid = featured.length const grid = featured.length
? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx))) ? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx)))
@@ -290,7 +292,8 @@ export function renderOnboarding() {
// let the toast be readable before the reload swaps the page // let the toast be readable before the reload swaps the page
await new Promise((res) => setTimeout(res, 900)); await new Promise((res) => setTimeout(res, 900));
} }
location.hash = "#/models"; // land where the reward is: try the model right away
location.hash = "#/playground";
location.reload(); location.reload();
}); });
const skipBtn = h("button", { class: "btn ghost", onClick: async () => { const skipBtn = h("button", { class: "btn ghost", onClick: async () => {
+541 -36
View File
@@ -1,17 +1,64 @@
// Playground — send a prompt through the live pipeline, inspect routing decision. // Playground — three tabs over the live gateway:
import { h, mount, money, ms } from "../ui.js"; // chat → the routing pipeline (with streaming, token counts, tok/s, TTFT)
// image → `cortiq imagine` over a local text-to-image .cmf (Lumina)
// video → `cortiq animate` over a local text-to-video .cmf (MiniMax-H3)
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
import { h, mount, toast, money, ms } from "../ui.js";
import { t } from "../i18n.js"; import { t } from "../i18n.js";
import { api, testStream } from "../api.js"; import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
import { appState } from "../app.js"; import { appState } from "../app.js";
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
function fmtBytes(n) {
if (!n && n !== 0) return "—";
if (n >= 1e9) return (n / 1e9).toFixed(2) + " GB";
if (n >= 1e6) return (n / 1e6).toFixed(1) + " MB";
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
return n + " B";
}
function kv(k, v) {
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
}
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
}
export async function renderPlayground() { export async function renderPlayground() {
const meta = appState.meta || (await api.meta()); const meta = appState.meta || (await api.meta());
const modelsData = await api.listModels(); const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id); const ids = (modelsData.models || []).map((m) => m.id);
let mediaModels = [];
try {
mediaModels = (await api.mediaModels()).models || [];
} catch {
/* media API unavailable — tabs will show the hint */
}
const imageModels = mediaModels.filter((m) => m.modality === "image");
const videoModels = mediaModels.filter((m) => m.modality === "video");
let mode = "auto";
const root = h("div"); const root = h("div");
let tab = "chat";
// ── tabs ──
const tabsSeg = h(
"div",
{ class: "seg" },
h("button", { class: "active", "data-tab": "chat" }, "💬 " + t("pg.tab.chat")),
h("button", { "data-tab": "image" }, "🖼 " + t("pg.tab.image")),
h("button", { "data-tab": "video" }, "🎬 " + t("pg.tab.video"))
);
const panelHost = h("div", {});
tabsSeg.querySelectorAll("button").forEach((b) =>
b.addEventListener("click", () => {
tab = b.dataset.tab;
tabsSeg.querySelectorAll("button").forEach((x) => x.classList.toggle("active", x === b));
mount(panelHost, panels[tab]);
})
);
// ════════════════════════════ chat panel ════════════════════════════
let mode = "auto";
const sysIn = h("textarea", { placeholder: t("pg.systemPlaceholder"), rows: 2 }); const sysIn = h("textarea", { placeholder: t("pg.systemPlaceholder"), rows: 2 });
const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 }); const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 });
const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 }); const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 });
@@ -44,16 +91,33 @@ export async function renderPlayground() {
modeExtra = next; modeExtra = next;
} }
const out = h("div", { class: "answer" }, h("div", { class: "empty" }, t("pg.empty"))); const chatLog = []; // the running conversation: {role, content}
const out = h("div", { class: "answer chat-log" }, h("div", { class: "empty" }, t("pg.empty")));
const decision = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—")); const decision = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
function bubble(role, text) {
return h("div", { class: "chat-msg " + role }, h("div", { class: "chat-bubble" }, text));
}
function renderChat(liveText) {
const items = chatLog.map((m) => bubble(m.role, m.content));
if (liveText != null) items.push(bubble("assistant", liveText || "…"));
mount(out, ...(items.length ? items : [h("div", { class: "empty" }, t("pg.empty"))]));
out.scrollTop = out.scrollHeight;
}
const clearBtn = h("button", { class: "btn ghost sm" }, "🧹 " + t("pg.clearChat"));
clearBtn.addEventListener("click", () => {
chatLog.length = 0;
renderChat();
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
});
const sendBtn = h("button", { class: "btn primary" }, t("pg.send")); const sendBtn = h("button", { class: "btn primary" }, t("pg.send"));
sendBtn.addEventListener("click", async () => { sendBtn.addEventListener("click", async () => {
const prompt = promptIn.value.trim(); const prompt = promptIn.value.trim();
if (!prompt) return; if (!prompt) return;
const messages = []; const messages = [];
if (sysIn.value.trim()) messages.push({ role: "system", content: sysIn.value.trim() }); if (sysIn.value.trim()) messages.push({ role: "system", content: sysIn.value.trim() });
messages.push({ role: "user", content: prompt }); messages.push(...chatLog, { role: "user", content: prompt });
let model = "cortiq-auto"; let model = "cortiq-auto";
if (mode === "auto") { if (mode === "auto") {
if (profileSel.value) model = "cortiq-auto:" + profileSel.value; if (profileSel.value) model = "cortiq-auto:" + profileSel.value;
@@ -62,7 +126,9 @@ export async function renderPlayground() {
} }
sendBtn.disabled = true; sendBtn.disabled = true;
sendBtn.textContent = t("pg.sending"); sendBtn.textContent = t("pg.sending");
mount(out, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.sending"))); chatLog.push({ role: "user", content: prompt });
promptIn.value = "";
renderChat("");
const body = { const body = {
model, model,
messages, messages,
@@ -73,12 +139,13 @@ export async function renderPlayground() {
if (streamCb.checked) { if (streamCb.checked) {
const started = performance.now(); const started = performance.now();
let acc = ""; let acc = "";
out.textContent = "";
const info = await testStream(body, (d) => { const info = await testStream(body, (d) => {
acc += d; acc += d;
out.textContent = acc; renderChat(acc);
}); });
out.textContent = acc || "—"; chatLog.push({ role: "assistant", content: acc || "—" });
renderChat();
const latency = Math.round(performance.now() - started);
renderDecision({ renderDecision({
cortiq: { cortiq: {
task_label: info.task_label, task_label: info.task_label,
@@ -88,32 +155,53 @@ export async function renderPlayground() {
cost_usd: info.cost_usd, cost_usd: info.cost_usd,
failover: false, failover: false,
}, },
usage: {}, usage: info.usage || {},
latency_ms: Math.round(performance.now() - started), usage_estimated: info.usage_estimated,
latency_ms: latency,
ttft_ms: info.ttft_ms,
}); });
} else { } else {
const r = await api.test(body); const r = await api.test(body);
if (r.ok) { if (r.ok) {
mount(out, r.answer || h("span", { class: "muted" }, "—")); chatLog.push({ role: "assistant", content: r.answer || "—" });
renderChat();
renderDecision(r); renderDecision(r);
} else { } else {
mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || "")); chatLog.pop(); // the user turn failed — let them retry it
renderChat();
promptIn.value = prompt;
mount(out, out.firstChild || "", h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—")); mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
} }
} }
} catch (e) { } catch (e) {
mount(out, h("div", { class: "callout" }, String(e.message || e))); chatLog.pop();
renderChat();
promptIn.value = prompt;
toast(String(e.message || e), "err");
} }
sendBtn.disabled = false; sendBtn.disabled = false;
sendBtn.textContent = t("pg.send"); sendBtn.textContent = t("pg.send");
}); });
// Cmd/Ctrl+Enter sends
promptIn.addEventListener("keydown", (e) => {
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
e.preventDefault();
sendBtn.click();
}
});
function kv(k, v) {
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
}
function renderDecision(r) { function renderDecision(r) {
const c = r.cortiq || {}; const c = r.cortiq || {};
const score = c.complexity?.score ?? 0; const score = c.complexity?.score ?? 0;
const u = r.usage || {};
const haveUsage = u.completion_tokens != null;
// generation speed over the produce phase (after first token when known)
let tokS = null;
if (haveUsage && r.latency_ms) {
const genMs = r.ttft_ms != null ? Math.max(1, r.latency_ms - r.ttft_ms) : r.latency_ms;
tokS = (u.completion_tokens / (genMs / 1000)).toFixed(1);
}
mount( mount(
decision, decision,
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
@@ -127,17 +215,278 @@ export async function renderPlayground() {
), ),
kv(t("pg.selected"), h("span", { class: "mono" }, c.selected_model || "—")), kv(t("pg.selected"), h("span", { class: "mono" }, c.selected_model || "—")),
kv(t("pg.source"), c.route_source || "—"), kv(t("pg.source"), c.route_source || "—"),
kv(t("pg.tokens"), `${r.usage?.prompt_tokens ?? 0}/${r.usage?.completion_tokens ?? 0}`), kv(t("pg.tokens"), haveUsage ? `${r.usage_estimated ? "≈ " : ""}${u.prompt_tokens ?? 0} / ${u.completion_tokens ?? 0}` : "—"),
kv(t("pg.tokSpeed"), tokS != null ? tokS + " tok/s" : "—"),
kv(t("pg.ttft"), r.ttft_ms != null ? ms(r.ttft_ms) : "—"),
kv(t("pg.cost"), money(c.cost_usd)), kv(t("pg.cost"), money(c.cost_usd)),
kv(t("pg.latency"), ms(r.latency_ms)), kv(t("pg.latency"), ms(r.latency_ms)),
kv(t("pg.failover"), c.failover ? "yes" : "no") kv(t("pg.failover"), c.failover ? "yes" : "no")
); );
} }
mount( const chatPanel = h(
root, "div",
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))), { class: "pg" },
h( h(
"div",
{},
h(
"div",
{ class: "card" },
field(t("pg.system"), sysIn),
field(t("pg.prompt"), promptIn),
field(t("pg.mode"), modeSeg),
modeExtra,
h("div", { class: "row" }, field(t("pg.temp"), tempIn), field(t("pg.maxTokens"), maxTokIn)),
h(
"div",
{ class: "flex", style: "justify-content:space-between; gap:8px" },
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
h("div", { class: "flex", style: "gap:8px" }, clearBtn, sendBtn)
)
),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
),
decision
);
// ═══════════════════════ shared media machinery ═══════════════════════
let mediaPoll = null;
function stopMediaPoll() {
if (mediaPoll) clearInterval(mediaPoll);
mediaPoll = null;
}
// one generation at a time server-side → one active job client-side
function watchJob(id, ui, onDone) {
stopMediaPoll();
const started = performance.now();
mediaPoll = setInterval(async () => {
if (!root.isConnected) return stopMediaPoll();
let j;
try {
j = await api.mediaJob(id);
} catch {
return;
}
const elapsed = (performance.now() - started) / 1000;
const pct = j.progress != null ? Math.round(j.progress * 100) : null;
// honest ETA once there is enough signal: total ≈ elapsed / progress
let eta = "";
if (j.state === "running" && j.progress > 0.03) {
const left = elapsed / j.progress - elapsed;
if (left > 1) eta = ` · ${t("pg.eta")} ~${left > 90 ? Math.round(left / 60) + " " + t("pg.min") : Math.round(left) + " s"}`;
}
let statusLine;
if (j.state === "queued") {
const pos = (j.queue_position ?? 0) + 1;
statusLine = t("pg.queued", { n: pos });
} else {
statusLine =
(j.phase || j.state) +
(pct != null && j.state === "running" ? ` · ${pct}%` : "") +
` · ${elapsed.toFixed(0)}s` + eta;
}
mount(
ui.bar,
h("div", { class: "pbar" },
j.state === "running"
? (pct != null
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
: h("div", { class: "pbar-fill run" }))
: j.state === "queued"
? h("div", { class: "pbar-fill run" })
: h("div", { class: "pbar-fill " + (j.state === "done" ? "ok" : "err"), style: "width:100%" })
),
h("div", { class: "muted small" }, statusLine),
(j.log || []).length && j.state === "running" ? h("pre", { class: "conv-log", style: "max-height:90px" }, j.log.slice(-5).join("\n")) : null
);
if (j.state !== "running" && j.state !== "queued") {
stopMediaPoll();
ui.goBtn.disabled = false;
ui.cancelBtn.style.display = "none";
if (j.state === "done") onDone(j);
else if (j.state === "error") toast(t("pg.genFailed"), "err");
}
}, 1000);
}
function mediaControls(goBtn, onCancelId) {
const cancelBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
cancelBtn.addEventListener("click", async () => {
try {
await api.mediaCancel(onCancelId.id);
} catch (e) {
toast(e.message, "err");
}
});
return cancelBtn;
}
function noModelsHint(kindLabel) {
return h(
"div",
{ class: "card" },
h("div", { class: "callout" },
t("pg.noMediaModels", { kind: kindLabel }),
" ",
h("a", { href: "#/import" }, t("settings.cmf.importLink") + " ↗"))
);
}
function dice(seedIn) {
const b = h("button", { class: "btn ghost sm", title: t("pg.randomSeed") }, "🎲");
b.addEventListener("click", () => (seedIn.value = Math.floor(Math.random() * 1e9)));
return b;
}
// ── gallery: past generations of one kind, loaded lazily ──
function buildGallery(kind, onOpen) {
const host = h("div", {});
async function refresh() {
let jobs = [];
try {
jobs = ((await api.mediaJobs()).jobs || []).filter((j) => j.kind === kind && j.state === "done");
} catch {
return;
}
if (!jobs.length) {
mount(host);
return;
}
const grid = h("div", { class: "gal-grid" });
for (const j of jobs.slice(0, 12)) {
const del = h("button", { class: "gal-del", title: t("import.delete") }, "×");
del.addEventListener("click", async (e) => {
e.stopPropagation();
if (!window.confirm(t("pg.galDeleteConfirm"))) return;
try {
await api.mediaDelete(j.id);
refresh();
} catch (err) {
toast(err.message, "err");
}
});
const cap = h("div", { class: "gal-cap" }, `${j.width}×${j.height} · seed ${j.seed}`);
const item = h("div", { class: "gal-item", title: j.prompt }, del, cap);
if (kind === "image") {
// lazy thumbnail: decode the PPM only when the item scrolls into view
const thumb = h("canvas", { class: "gal-thumb" });
item.prepend(thumb);
const io = new IntersectionObserver(async (entries) => {
if (!entries.some((x) => x.isIntersecting)) return;
io.disconnect();
try {
const buf = await mediaFileBytes(j.id, "image");
const full = ppmToCanvas(buf);
thumb.width = 128;
thumb.height = Math.round((128 * full.height) / full.width) || 128;
thumb.getContext("2d").drawImage(full, 0, 0, thumb.width, thumb.height);
} catch {
/* file gone */
}
});
io.observe(item);
} else {
item.prepend(h("div", { class: "gal-thumb gal-video" }, "🎬"));
}
item.addEventListener("click", () => onOpen(j));
grid.appendChild(item);
}
mount(host, h("div", { class: "card-title", style: "margin-top:14px" }, t("pg.gallery")), grid);
}
refresh();
return { host, refresh };
}
// ════════════════════════════ image panel ════════════════════════════
let imagePanel;
if (!imageModels.length) {
imagePanel = noModelsHint(t("pg.tab.image"));
} else {
const imModel = h("select", {}, ...imageModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
const imPrompt = h("textarea", { placeholder: t("pg.imagePromptPlaceholder"), rows: 3 });
const imW = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
const imH = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
const imSteps = h("input", { type: "number", min: 1, max: 100, value: 30 });
const imCfg = h("input", { type: "number", step: "0.5", value: 4 });
const imSeed = h("input", { type: "number", value: 42 });
const imBar = h("div", {});
const imOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.imageEmpty")));
const imMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
const imGo = h("button", { class: "btn primary" }, "🖼 " + t("pg.generate"));
const imCancelRef = { id: null };
const imCancel = mediaControls(imGo, imCancelRef);
imGo.addEventListener("click", async () => {
const prompt = imPrompt.value.trim();
if (!prompt) return;
imGo.disabled = true;
imCancel.style.display = "";
mount(imOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
const params = {
kind: "image",
model: imModel.value,
prompt,
width: +imW.value,
height: +imH.value,
steps: +imSteps.value || 30,
cfg: parseFloat(imCfg.value) || 4,
seed: +imSeed.value || 42,
};
try {
const r = await api.mediaGenerate(params);
imCancelRef.id = r.job;
const t0 = performance.now();
watchJob(r.job, { bar: imBar, goBtn: imGo, cancelBtn: imCancel }, async (j) => {
mount(imBar);
const secs = (performance.now() - t0) / 1000;
await showImageResult(j, secs);
imGallery.refresh();
});
} catch (e) {
imGo.disabled = false;
imCancel.style.display = "none";
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
}
});
async function showImageResult(j, secs) {
try {
const buf = await mediaFileBytes(j.id, "image");
const canvas = ppmToCanvas(buf);
canvas.className = "media-canvas";
const dl = h("button", { class: "btn sm" }, "⬇ PNG");
dl.addEventListener("click", () =>
canvas.toBlob((b) => {
const a = h("a", { href: URL.createObjectURL(b), download: `cortiq-${j.id}.png` });
a.click();
}, "image/png")
);
mount(imOut, canvas, h("div", { class: "actions" }, dl));
const rows = [
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
kv(t("pg.resolution"), `${j.width}×${j.height}`),
kv(t("pg.steps"), String(j.steps)),
kv(t("pg.seed"), String(j.seed)),
kv(t("pg.fileSize"), fmtBytes(j.out_bytes)),
];
if (secs != null) {
rows.push(kv(t("pg.genTime"), secs.toFixed(1) + " s"));
rows.push(kv(t("pg.timePerStep"), (secs / j.steps).toFixed(2) + " s"));
} else if (j.finished && j.started) {
const dur = j.finished - j.started;
rows.push(kv(t("pg.genTime"), dur + " s"));
rows.push(kv(t("pg.timePerStep"), (dur / j.steps).toFixed(2) + " s"));
}
mount(imMetrics, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), ...rows);
} catch (e) {
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
}
}
const imGallery = buildGallery("image", (j) => {
imSeed.value = j.seed;
imSteps.value = j.steps;
showImageResult(j, null);
});
imagePanel = h(
"div", "div",
{ class: "pg" }, { class: "pg" },
h( h(
@@ -146,22 +495,178 @@ export async function renderPlayground() {
h( h(
"div", "div",
{ class: "card" }, { class: "card" },
h("label", { class: "field" }, h("span", {}, t("pg.system")), sysIn), field(t("pg.model"), imModel),
h("label", { class: "field" }, h("span", {}, t("pg.prompt")), promptIn), field(t("pg.prompt"), imPrompt),
h("label", { class: "field" }, h("span", {}, t("pg.mode")), modeSeg), h("div", { class: "row" }, field(t("pg.width"), imW), field(t("pg.height"), imH)),
modeExtra, h("div", { class: "row" },
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("pg.temp")), tempIn), h("label", { class: "field" }, h("span", {}, t("pg.maxTokens")), maxTokIn)), field(t("pg.steps"), imSteps),
h( field("CFG", imCfg, t("pg.cfgHint"))),
"div", h("div", { class: "row" },
{ class: "flex", style: "justify-content:space-between" }, field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, imSeed, dice(imSeed)))),
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")), h("div", { class: "actions" }, imCancel, imGo),
sendBtn imBar
)
), ),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out) h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), imOut, imGallery.host)
), ),
decision imMetrics
) );
}
// ════════════════════════════ video panel ════════════════════════════
let videoPanel;
if (!videoModels.length) {
videoPanel = noModelsHint(t("pg.tab.video"));
} else {
const vModel = h("select", {}, ...videoModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
const vPrompt = h("textarea", { placeholder: t("pg.videoPromptPlaceholder"), rows: 3 });
const vW = h("input", { type: "number", step: 32, value: 512 });
const vH = h("input", { type: "number", step: 32, value: 288 });
const vFrames = h("input", { type: "number", value: 39 });
const vSteps = h("input", { type: "number", value: 4 });
const vSeed = h("input", { type: "number", value: 42 });
const vQuality = h("input", { type: "number", min: 10, max: 100, value: 92 });
const vFirst = h("input", { type: "file", accept: "image/*" });
const vLast = h("input", { type: "file", accept: "image/*" });
const vBar = h("div", {});
const vOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.videoEmpty")));
const vMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
const vGo = h("button", { class: "btn primary" }, "🎬 " + t("pg.generate"));
const vCancelRef = { id: null };
const vCancel = mediaControls(vGo, vCancelRef);
vGo.addEventListener("click", async () => {
const prompt = vPrompt.value.trim();
if (!prompt) return;
vGo.disabled = true;
vCancel.style.display = "";
mount(vOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
const params = {
kind: "video",
model: vModel.value,
prompt,
width: +vW.value || 512,
height: +vH.value || 288,
frames: +vFrames.value || 39,
steps: +vSteps.value || 4,
seed: +vSeed.value || 42,
quality: +vQuality.value || 92,
};
try {
if (vFirst.files[0]) params.first_frame_b64 = await fileToPpmB64(vFirst.files[0], params.width, params.height);
if (vLast.files[0]) params.last_frame_b64 = await fileToPpmB64(vLast.files[0], params.width, params.height);
const r = await api.mediaGenerate(params);
vCancelRef.id = r.job;
const t0 = performance.now();
watchJob(r.job, { bar: vBar, goBtn: vGo, cancelBtn: vCancel }, async (j) => {
mount(vBar);
vGallery.refresh();
const secs = (performance.now() - t0) / 1000;
try {
await showVideoResult(j, vOut);
const durS = (j.frames || 0) / 24;
mount(
vMetrics,
h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))),
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
kv(t("pg.resolution"), `${j.width}×${j.height}`),
kv(t("pg.frames"), `${j.frames} @ 24 fps`),
kv(t("pg.duration"), durS.toFixed(1) + " s"),
kv(t("pg.steps"), String(j.steps)),
kv(t("pg.seed"), String(j.seed)),
kv(t("pg.genTime"), secs.toFixed(1) + " s"),
kv(t("pg.timePerFrame"), j.frames ? (secs / j.frames).toFixed(2) + " s" : "—"),
kv(t("pg.fileSize"), fmtBytes(j.out_bytes))
);
} catch (e) {
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
}
});
} catch (e) {
vGo.disabled = false;
vCancel.style.display = "none";
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
}
});
async function showVideoResult(j, host) {
const downloads = h("div", { class: "actions" });
for (const kind of j.outputs || []) {
const label = { video: "⬇ AVI", audio: "⬇ WAV", mp4: "⬇ MP4" }[kind];
if (!label) continue;
const b = h("button", { class: "btn sm" }, label);
b.addEventListener("click", async () => {
const { url } = await mediaFileUrl(j.id, kind);
const ext = kind === "video" ? "avi" : kind === "audio" ? "wav" : "mp4";
h("a", { href: url, download: `cortiq-${j.id}.${ext}` }).click();
});
downloads.appendChild(b);
}
if ((j.outputs || []).includes("mp4")) {
// ffmpeg was present server-side → native playback
const { url } = await mediaFileUrl(j.id, "mp4");
mount(host, h("video", { src: url, controls: true, class: "media-canvas" }), downloads);
return;
}
// no ffmpeg anywhere: decode the MJPEG AVI ourselves, sync to the wav
const buf = await mediaFileBytes(j.id, "video");
const frames = aviJpegFrames(buf);
let audio = null;
if ((j.outputs || []).includes("audio")) {
const { url } = await mediaFileUrl(j.id, "audio");
audio = h("audio", { src: url });
}
const player = mjpegPlayer(frames, 24, audio);
player.canvas.className = "media-canvas";
await player.seek0();
const playBtn = h("button", { class: "btn primary sm" }, "▶ " + t("pg.play"));
playBtn.addEventListener("click", async () => {
if (player.state.playing) {
player.pause();
playBtn.textContent = "▶ " + t("pg.play");
} else {
playBtn.textContent = "⏸";
await player.play();
}
});
mount(host, player.canvas, h("div", { class: "actions" }, playBtn, ...downloads.childNodes ? [downloads] : []));
}
const vGallery = buildGallery("video", (j) => showVideoResult(j, vOut));
videoPanel = h(
"div",
{ class: "pg" },
h(
"div",
{},
h(
"div",
{ class: "card" },
field(t("pg.model"), vModel),
field(t("pg.prompt"), vPrompt),
h("div", { class: "row" }, field(t("pg.width"), vW, t("pg.videoSizeHint")), field(t("pg.height"), vH)),
h("div", { class: "row" }, field(t("pg.frames"), vFrames, t("pg.framesHint")), field(t("pg.steps"), vSteps, t("pg.videoStepsHint"))),
h("div", { class: "row" },
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, vSeed, dice(vSeed))),
field(t("pg.quality"), vQuality)),
h("div", { class: "row" }, field(t("pg.firstFrame"), vFirst, t("pg.keyframeHint")), field(t("pg.lastFrame"), vLast)),
h("div", { class: "actions" }, vCancel, vGo),
vBar
),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), vOut, vGallery.host)
),
vMetrics
);
}
const panels = { chat: chatPanel, image: imagePanel, video: videoPanel };
mount(
root,
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
tabsSeg,
panelHost
); );
mount(panelHost, panels.chat);
return root; return root;
} }
+117 -30
View File
@@ -4,71 +4,155 @@ import { t } from "../i18n.js";
import { api } from "../api.js"; import { api } from "../api.js";
import { appState } from "../app.js"; import { appState } from "../app.js";
// Known labels of the hosted taxonomy — used for localized names/descriptions
// and as the offline fallback; the live list comes from /routing/labels.
const TASK_META = {
chitchat: { icon: "💬" },
code: { icon: "⌨" },
"creative-writing": { icon: "✍" },
extraction: { icon: "📑" },
math: { icon: "∑" },
qa: { icon: "❓" },
summarization: { icon: "📋" },
translation: { icon: "🌍" },
};
export async function renderRouting() { export async function renderRouting() {
const meta = appState.meta || (await api.meta()); const meta = appState.meta || (await api.meta());
const [routing, modelsData] = await Promise.all([api.getRouting(), api.listModels()]); const [routing, modelsData] = await Promise.all([api.getRouting(), api.listModels()]);
const ids = (modelsData.models || []).map((m) => m.id); const ids = (modelsData.models || []).map((m) => m.id);
// Task labels: live from the router when reachable, built-in set otherwise;
// labels that already have a rule are always shown (custom taxonomies).
let taskLabels = Object.keys(TASK_META);
let labelsSource = "builtin";
try {
const r = await api.routingLabels();
if (r.labels && r.labels.length) {
taskLabels = r.labels;
labelsSource = "router";
}
} catch {
/* offline — builtin list */
}
for (const k of Object.keys(routing.tasks || {})) {
if (!taskLabels.includes(k)) taskLabels.push(k);
}
// Share of recent traffic per task — shows which rules actually matter.
let taskShare = {};
try {
const stat = await api.stats("7d", "label");
const total = (stat.breakdown || []).reduce((a, b) => a + (b.requests || 0), 0);
if (total > 0) {
for (const b of stat.breakdown || []) taskShare[b.key] = (b.requests || 0) / total;
}
} catch {
/* no stats yet */
}
// local editable state — ensure standard tiers exist for a friendly editor // local editable state — ensure standard tiers exist for a friendly editor
const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {}); const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {});
const tasks = {};
for (const l of taskLabels) tasks[l] = (routing.tasks || {})[l] || [];
const st = { const st = {
tiers, tiers,
tasks,
default: routing.default || ids[0] || "", default: routing.default || ids[0] || "",
policy: routing.policy || { mode: "fixed_table", min_class: {} }, policy: routing.policy || { mode: "fixed_table", min_class: {} },
}; };
const root = h("div"); const root = h("div");
function moveChip(tier, i, dir) { function moveChip(map, key, i, dir) {
const arr = st.tiers[tier]; const arr = map[key];
const ni = i + dir; const ni = i + dir;
if (ni < 0 || ni >= arr.length) return; if (ni < 0 || ni >= arr.length) return;
[arr[i], arr[ni]] = [arr[ni], arr[i]]; [arr[i], arr[ni]] = [arr[ni], arr[i]];
render(); render();
} }
function removeChip(tier, i) { function removeChip(map, key, i) {
st.tiers[tier].splice(i, 1); map[key].splice(i, 1);
render(); render();
} }
function addChip(tier, id) { function addChip(map, key, id) {
if (id && !st.tiers[tier].includes(id)) st.tiers[tier].push(id); if (id && !map[key].includes(id)) map[key].push(id);
render(); render();
} }
function chipEditor(map, key, emptyLabel) {
function tierRow(tier) { const arr = map[key];
const arr = st.tiers[tier];
const avail = ids.filter((id) => !arr.includes(id)); const avail = ids.filter((id) => !arr.includes(id));
const addSel = h( const addSel = h(
"select", "select",
{ { style: "width:auto", onChange: (e) => addChip(map, key, e.target.value) },
style: "width:auto",
onChange: (e) => {
addChip(tier, e.target.value);
},
},
h("option", { value: "" }, t("routing.addModel")), h("option", { value: "" }, t("routing.addModel")),
...avail.map((id) => h("option", { value: id }, id)) ...avail.map((id) => h("option", { value: id }, id))
); );
return h(
"div",
{ class: "chips" },
arr.length === 0 && emptyLabel ? h("span", { class: "muted small chips-empty" }, emptyLabel) : null,
...arr.map((id, i) =>
h(
"div",
{ class: "chip" },
h("span", { class: "ord" }, i + 1),
id,
h("span", { class: "arrow", title: "←", onClick: () => moveChip(map, key, i, -1) }, "◄"),
h("span", { class: "arrow", title: "→", onClick: () => moveChip(map, key, i, 1) }, "►"),
h("span", { class: "x", onClick: () => removeChip(map, key, i) }, "×")
)
),
avail.length ? addSel : null
);
}
function tierRow(tier) {
return h( return h(
"div", "div",
{ class: "tierrow" }, { class: "tierrow" },
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)), h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
chipEditor(st.tiers, tier)
);
}
function taskRow(label) {
const known = TASK_META[label];
const name = known ? t("task." + label) : label;
const share = taskShare[label];
return h(
"div",
{ class: "tierrow taskrow" },
h( h(
"div", "div",
{ class: "chips" }, { class: "tname taskname" },
...arr.map((id, i) => h("div", {}, (known ? known.icon + " " : "") + name),
h( h("div", { class: "muted small mono" }, label),
"div", share != null ? h("div", { class: "muted small" }, t("routing.tasks.share", { p: Math.round(share * 100) })) : null
{ class: "chip" }, ),
h("span", { class: "ord" }, i + 1), chipEditor(st.tasks, label, t("routing.tasks.inherit"))
id, );
h("span", { class: "arrow", title: "←", onClick: () => moveChip(tier, i, -1) }, "◄"), }
h("span", { class: "arrow", title: "→", onClick: () => moveChip(tier, i, 1) }, "►"),
h("span", { class: "x", onClick: () => removeChip(tier, i) }, "×") function tasksCard() {
) const addIn = h("input", { placeholder: t("routing.tasks.addPlaceholder"), style: "max-width:220px" });
), const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("routing.tasks.addCustom"));
avail.length ? addSel : null addBtn.addEventListener("click", () => {
) const v = addIn.value.trim().toLowerCase();
if (!v || st.tasks[v]) return;
st.tasks[v] = [];
render();
});
return h(
"div",
{ class: "card" },
h("div", { class: "card-head" },
h("h3", {}, "🎯 " + t("routing.tasks.title")),
h("span", { class: "card-head sub" },
labelsSource === "router" ? t("routing.tasks.fromRouter") : t("routing.tasks.builtin"))),
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.tasks.hint")),
...Object.keys(st.tasks).map(taskRow),
h("div", { class: "flex", style: "gap:8px; margin-top:10px" }, addIn, addBtn)
); );
} }
@@ -116,7 +200,9 @@ export async function renderRouting() {
saveBtn.addEventListener("click", async () => { saveBtn.addEventListener("click", async () => {
saveBtn.disabled = true; saveBtn.disabled = true;
try { try {
await api.putRouting({ tiers: st.tiers, default: st.default, policy: st.policy }); const tasks = {};
for (const [k, v] of Object.entries(st.tasks)) if (v.length) tasks[k] = v;
await api.putRouting({ tiers: st.tiers, tasks, default: st.default, policy: st.policy });
toast(t("toast.saved"), "good"); toast(t("toast.saved"), "good");
} catch (e) { } catch (e) {
toast(String(e.message || e), "bad"); toast(String(e.message || e), "bad");
@@ -143,6 +229,7 @@ export async function renderRouting() {
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))), h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
...Object.keys(st.tiers).map(tierRow) ...Object.keys(st.tiers).map(tierRow)
), ),
tasksCard(),
h( h(
"div", "div",
{ class: "card" }, { class: "card" },
+46 -10
View File
@@ -1,7 +1,7 @@
// Settings — router/route/breaker/log/telemetry/stats + export/import config. // Settings — router/route/breaker/log/telemetry/stats + export/import config.
import { h, mount, toast } from "../ui.js"; import { h, mount, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js"; import { t } from "../i18n.js";
import { api } from "../api.js"; import { api, getToken, postRaw } from "../api.js";
import { appState, SITE_URL } from "../app.js"; import { appState, SITE_URL } from "../app.js";
// A small "?" badge with a hover/focus tooltip explaining a setting. // A small "?" badge with a hover/focus tooltip explaining a setting.
@@ -480,22 +480,33 @@ export async function renderSettings() {
}); });
async function exportConfig() { async function exportConfig() {
const cfg = await api.getConfig(); // the real gateway.toml from disk — comments and all
const blob = new Blob([JSON.stringify(cfg, null, 2)], { type: "application/json" }); const tok = getToken();
const url = URL.createObjectURL(blob); const headers = {};
const a = h("a", { href: url, download: "gateway-config.json" }); if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch("/admin/api/config/export", { headers });
if (!r.ok) {
toast(await r.text(), "bad");
return;
}
const url = URL.createObjectURL(await r.blob());
const a = h("a", { href: url, download: "gateway.toml" });
document.body.appendChild(a); document.body.appendChild(a);
a.click(); a.click();
a.remove(); a.remove();
URL.revokeObjectURL(url); URL.revokeObjectURL(url);
} }
const fileIn = h("input", { type: "file", accept: ".json,application/json", style: "display:none" }); const fileIn = h("input", { type: "file", accept: ".toml,.json,application/json", style: "display:none" });
fileIn.addEventListener("change", async () => { fileIn.addEventListener("change", async () => {
const f = fileIn.files[0]; const f = fileIn.files[0];
if (!f) return; if (!f) return;
try { try {
const cfg = JSON.parse(await f.text()); const text = await f.text();
await api.putConfig(cfg); if (f.name.endsWith(".json")) {
await api.putConfig(JSON.parse(text)); // legacy JSON export
} else {
await postRaw("/config/import", text); // gateway.toml — validated + hot-reloaded
}
toast(t("toast.imported"), "good"); toast(t("toast.imported"), "good");
} catch (e) { } catch (e) {
toast(String(e.message || e), "bad"); toast(String(e.message || e), "bad");
@@ -503,6 +514,29 @@ export async function renderSettings() {
fileIn.value = ""; fileIn.value = "";
}); });
const restartBtn = h("button", { class: "btn danger" }, "⟳ " + t("settings.restart"));
restartBtn.addEventListener("click", async () => {
if (!(await confirmDialog(t("settings.restartConfirm")))) return;
restartBtn.disabled = true;
try {
await api.restart();
toast(t("settings.restarting"), "good");
// the gateway comes back on the same port — reload once it answers
const poll = setInterval(async () => {
try {
await api.health();
clearInterval(poll);
location.reload();
} catch {
/* still restarting */
}
}, 1000);
} catch (e) {
toast(String(e.message || e), "bad");
restartBtn.disabled = false;
}
});
mount( mount(
root, root,
h( h(
@@ -623,7 +657,9 @@ export async function renderSettings() {
{ class: "flex wrap" }, { class: "flex wrap" },
h("button", { class: "btn", onClick: exportConfig }, t("settings.export")), h("button", { class: "btn", onClick: exportConfig }, t("settings.export")),
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")), h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
fileIn fileIn,
h("div", { class: "grow" }),
restartBtn
) )
) )
); );