Compare commits

...
42 Commits
Author SHA1 Message Date
infosave2007 f65f131c41 fix(routing): offered tools must not make a local model unroutable
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Verified against a real Open WebUI container: it attaches its ~26 built-in
tools (knowledge bases, notes, calendar, ...) to every single message, and
resolve() treated that offer as a hard capability requirement. On a pool of
tools-less local models a plain "hello" therefore failed with "No healthy
model available that satisfies required capabilities", and a pinned local
model was reported as "not available - not in the pool (deleted?)".

In the OpenAI contract `tools` is an offer; only tool_choice: "required" (or
a named function) makes the call impossible without them. Keep preferring a
tool-capable model, but fall back to serving the request with the tools
stripped - together with tool_choice/parallel_tool_calls, which a provider
rejects on their own. A pin whose only shortcoming is tool support is now
honoured; an unknown pin and tool_choice: "required" still fail loudly.

The downgrade is never silent: X-Cortiq-Tools-Dropped: true, plus
cortiq.tools_dropped in the echo block.
2026-08-21 09:59:28 +03:00
infosave2007 b339c2b622 fix(openai): strict OpenAI wire compatibility — Open WebUI can chat again
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
/v1/chat/completions answered ordinary client payloads with a bare HTTP 422
(a plain-text body no OpenAI client can parse), so Open WebUI showed neither
an answer nor an error — it just hung. Parse the body leniently instead:
content as string/null/content-part array, tools: null, stream: null, a
string reasoning_effort, max_completion_tokens, messages[].name and
tool_call_id, and a missing Content-Type. Unreadable bodies now come back as
an OpenAI error envelope with 400.

Responses are strict OpenAI JSON: no "tool_calls": null, finish_reason mapped
to the spec's closed set (cortiq serve's max_tokens -> length), never an empty
id, and `model` echoes the id the client requested — the model that actually
answered stays in cortiq.selected_model and X-Cortiq-Selected-Model.

Streaming is re-normalised rather than forwarded verbatim: a leading role
delta, message-shaped chunks converted to delta, one terminal finish_reason,
always a trailing data: [DONE], and an SSE error event when the upstream dies
mid-flight. Adds X-Accel-Buffering: no so a fronting proxy cannot buffer the
stream into a hang.

GET /v1/models now lists managed [[cmf.servers]] models with their live
running/healthy state, so clients can pin a local .cmf directly.

Also strips client bookkeeping keys (chat_id, session_id, metadata, features,
...) from the upstream call and drops the leftover [to_wire] debug prints.
2026-08-21 09:10:31 +03:00
infosave2007 34093330f6 chore: bump v0.2.44
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 17:58:25 +03:00
infosave2007 faddf945d7 fix(config): a foreign project must not capture the config path
default_config_path treated any directory holding Cargo.toml plus a
config/ folder as a gateway checkout, so running cortiq-gateway inside
an unrelated project died with "cannot read config config/gateway.toml"
instead of using the per-user home. Now ./config/gateway.toml counts
only when the file is actually readable, the repo-checkout heuristic
requires Cargo.toml to be cortiq-gateway's own, and everything else
falls through to $CORTIQ_GATEWAY_HOME / ~/.cortiq-gateway — typing
cortiq-gateway in any directory picks up the same configuration.
2026-08-08 17:57:45 +03:00
infosave2007 255d989e83 feat(admin): power controls — stop everything & power off, restart everything
Settings gains a second service button next to restart: stop every managed
local model server and exit the gateway process (nothing is respawned; in
Docker the restart policy decides). The SPA replaces itself with a static
"gateway is off" screen since the API is gone. The restart button is
relabeled "Restart everything" to say what it always did — the gateway
re-execs itself and managed servers come back with it. Translated in all
seven languages.
2026-08-08 17:33:56 +03:00
infosave2007 0d9d24125a style: rustfmt after resolve() signature change 2026-08-08 16:26:07 +03:00
infosave2007 1c59906b49 chore: bump v0.2.43
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 16:12:17 +03:00
infosave2007 19f666e7b1 feat(cmf): auto-detect tool support of managed local servers
cortiq serve >=0.5.62 advertises the loaded model's capabilities in
/healthz ({"capabilities":{"tools":true}} when the chat template has a
tools branch). The gateway now reads that handshake when the managed
server becomes healthy and treats detected tool support as equivalent
to declared caps in the candidate filter — so a low-tier chain made of
a local model no longer widens agent (tools) traffic to cloud models
once the local server can actually drive tools.

Also adds an optional manual "caps" override to [[cmf.servers]] for
older serve builds that do not advertise capabilities.
2026-08-08 15:52:20 +03:00
infosave2007 dbcd9827f6 fix(pipeline): capability filtering widens the chain instead of killing the request
OpenCode (like every coding agent) sends tool definitions with each request.
The router rated the prompt low → the low-tier chain contained only the
local model → the tools filter emptied the chain → hard failure: "No
healthy model available that satisfies required capabilities", with the
agent stuck in a retry loop.

On an AUTO route whose filtered chain comes up empty, the pipeline now
widens to any pool model satisfying the required capabilities — preferring
the routing table's own medium→high order (an agentic request deserves a
capable model), then the rest of the pool — and logs the widening. Pinned
requests still fail loudly as before. Verified with OpenCode's exact
request shape: cortiq-auto + tools at tier low now answers from
deepseek in 1.2s instead of erroring.
2026-08-08 14:25:00 +03:00
infosave2007 b689180717 fix(keys): snippets survive being pasted into real zsh
The copyable text opened with a "# hint" line — but default interactive
zsh has interactivecomments off, so the line was parsed as a command, and
the "!" inside «(перезапишет!)» fired history expansion: pasting the
OpenCode snippet died with "zsh: event not found: )" before doing
anything. Hints now live in the UI above the snippet; the clipboard gets
commands only. Verified by piping the actual rendered snippet through an
interactive zsh: the config file lands intact.
2026-08-08 14:16:22 +03:00
infosave2007 f385c2254b fix(ui): install-command chip and bands slider readable in the light theme
The chip used var(--panel-2, #23252c) — a variable that doesn't exist with
a hard dark fallback, so in the light theme the install command was a dark
box with dark text. Same class of bug in the bands slider track and handle
labels (invented --panel-2/--fg vars). All moved to the real theme
variables (--panel/--border/--text) via a .cmd-chip class; verified by
computed styles in the light theme (text rgb(15,23,42) on white) and
screenshots of both views.
2026-08-08 14:12:39 +03:00
infosave2007 9607a38792 feat(keys): paste-your-key field — snippets carry the real key instantly
Next to the copy button: a key input and an «Insert key» button (Enter works
too). The pasted key substitutes into every harness/SDK snippet immediately
and survives tab and OS switches; a freshly created key pre-fills the field.
No more copying a snippet and then hand-editing the placeholder. 2 strings
×7 languages.
2026-08-08 14:07:11 +03:00
infosave2007 239ddc767c feat(keys): harness snippets for every OS + install command when missing
The harness auto-setup scripts were bash-only (heredocs, mkdir -p) — useless
on Windows. Every scripty snippet now exists in two flavors behind a
🐧🍎/🪟 switch (auto-detected from the browser): bash heredocs for
Linux/macOS and PowerShell here-strings ($env:, New-Item, @'..'@,
curl.exe with backtick continuations) for Windows. SDK snippets and GUI
instructions stay OS-independent.

And when detection says a harness is NOT installed on the gateway machine,
the status line now offers its one-line install command for the selected OS
(claude.ai install.sh / install.ps1, npm -g for Codex and OpenCode-on-
Windows, code --install-extension for Continue/Cline) with a copy button.
1 string ×7 languages.
2026-08-08 14:05:47 +03:00
infosave2007 057abaa95a feat!: API keys actually gate the gateway — auth, attribution, rate limits, budgets
The office-readiness audit found the foundation missing: configured API keys
were decorative. Any bearer — or none — passed every model endpoint, per-key
rate limits existed only in the form, and all real traffic was recorded as
"anonymous", leaving the per-key spend table permanently blank.

A middleware on the model surface (/v1/*, /route, /mcp) now enforces keys:
- no keys configured → open mode as before (the console already warns);
- keys exist → Bearer or x-api-key must match (401 invalid_api_key);
- per-key rate_per_min → fixed one-minute window, 429 with Retry-After;
- NEW per-key monthly_budget_usd → 429 insufficient_quota once the account's
  month-to-date spend crosses the ceiling;
- the matched account rides the request into stats — the keys table's spend
  columns finally see real traffic.

Month-to-date spend is tracked per account in stats and persisted in the
totals sidecar with a watermark (no double counting across restarts). The
key dialog gains a budget field; the table shows "budget $N/mo" in red once
exhausted. All verified live: 401 wrong/missing key, 200 + attribution with
the right key, 429 + Retry-After over the limit, 429 insufficient_quota on
the second request past a $0.000001 budget.

Also: the dashboard shows the OpenRouter credit balance (GET
/api/v1/credits via the stored key, cached 5 min, hidden without a key, red
under $5 — live: $16.18), the server drains connections on Ctrl-C instead
of dropping them, request bodies are capped at 32 MB, and docs/DEPLOY(.ru)
cover office deployment: systemd/launchd/Windows service, nginx/caddy TLS,
security checklist, backups. 10 strings ×7 languages.
2026-08-08 14:01:16 +03:00
infosave2007 dd864eed22 fix(cache): honest settings card — clean threshold, live stats, no-embedder warning
Full audit of the semantic cache against a live instance (mock embedder,
identical requests): the mechanism works — miss at 3.1s, then hits at 2ms
with entries/hits/saved-$ counting. Two honest gaps fixed:

- the threshold round-tripped through f32 and rendered as
  0.9200000166893005 in both the TOML and the settings input — the config
  field is f64 now (writes exactly what you typed) and the input rounds
  legacy values to 2 decimals;
- the card now shows proof of life (entries / hits / hit-rate / saved $)
  and, when the cache is enabled with NO embedding model available, a
  warning that it is silently doing nothing and how to fix it — previously
  that state looked identical to a working cache.

Also audited in the same pass, no changes needed: Prometheus /metrics
(toggle on/off applies live, 404 when off), and all 7 protocol switches
(each gates its endpoint instantly both ways). 4 strings ×7 languages.
2026-08-08 13:21:43 +03:00
infosave2007 c1ff675d80 chore: release 0.2.42
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 12:18:04 +03:00
infosave2007 9ef1af3ae1 feat(ui): the continuous complexity score shows next to the tier
Benchmark (routing mode) and the dashboard's recent-requests list now print
the router's raw 0..1 score under/next to the tier badge — people can see
what different prompts actually weigh and tune the boundary sliders from
real data. Tooltip explains the link to the Routing sliders (7 languages).

Caught before commit: naming the field row.score collided with the answer-
correctness fraction from evaluate() and dragged the benchmark's quality KPI
down to the average complexity — renamed to cscore, KPI verified back at
100%. Dashboard tier badges also render translated names now, and the
Russian locale calls the dashboard «Рабочий стол».
2026-08-08 12:11:07 +03:00
infosave2007 6f4d83ce13 chore: release 0.2.41
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 09:55:20 +03:00
infosave2007 0040bfd7e9 fix(routing): the high zone is unmistakably red
At 35% opacity over the dark background the amber medium zone and the
light-red high zone read as the same color right of the medium handle.
Saturation stepped up per zone (45/55/65%) and high uses a deeper red —
green / amber / red now read at a glance.
2026-08-08 09:53:25 +03:00
infosave2007 10b5101757 fix(i18n): complexity tier names are translated everywhere they're shown
low/medium/high leaked as raw English into the bands legend, the tier badges
on the Routing page, the pair complexity selects, the benchmark's tier column
and the Playground decision panel. New trTier() helper renders the human name
(ru: лёгкий/средний/сложный, de: leicht/mittel/schwer, fr/es/zh/tr likewise);
custom tier names pass through untouched, badge colors stay keyed by the raw
tier. Verified in the browser in Russian and German.
2026-08-08 09:51:38 +03:00
infosave2007 788c87c565 feat(routing): linked complexity-boundary sliders — your own low/medium/high
The tier boundaries lived on the router and were global for every customer
of a hosted router; per-gateway tuning meant hand-editing score ranges. The
Routing page now has one track with two draggable handles (the low|medium
and medium|high boundaries) over three colored zones: handles push each
other (low can't cross medium — it shoves it along, and vice versa), the
numbers ride on the handles and the legend updates live, so nobody types
ranges by hand. Unchecked = the router's own bands, untouched behavior.

Under the hood the gateway re-maps the router's CONTINUOUS complexity score
with the user's boundaries ([routing.bands] low/medium in gateway.toml,
validated 0 < low < medium < 1) before tier chains, escalation and stats —
verified live: with bands at 0.70/0.72 a score-0.27 request that used to be
medium routed low, and back after restoring the defaults.

2 new strings in all 7 languages.
2026-08-08 09:48:36 +03:00
infosave2007 179c414a0e fix(benchmark): retry the SAME hard prompt after a pause before reaching for a fresh one
Difficulty is judged in the background: the first-ever occurrence of a hard
prompt routes by prior (often low/medium), the oracle's judgement lands in
the cache a couple of seconds later. The route-judged retry fired instantly
with a FRESH template — which has the same first-encounter fate — so the
test could go red exactly when the system was working as designed. The retry
now re-asks the same prompt after 4s (steady-state answer from the cache),
and only then tries a fresh template.
2026-08-08 09:23:53 +03:00
infosave2007 115b04f20e chore: release 0.2.40
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 09:13:12 +03:00
infosave2007 b24d4cdbab fix(routing): the task-rule badge only when the rule actually chose the head
route_source reported "+task-rule" whenever a rule existed for the task —
so a math prompt routed low by the ordinary tier chain still wore the badge
next to the local model, reading as "the rule picked this" when it hadn't
(the rule only defines math@high). matched now means the head of the chain
came from the rule's tier-specific or any-tier list; verified live: the same
low math request now reports route_source "router".
2026-08-08 09:02:25 +03:00
infosave2007 06813b3e97 feat(routing)!: task rules become per-complexity model maps
The single per-task «Complexity» pin was the wrong shape — what you actually
decide is WHICH model serves a task at WHICH complexity. A task rule is now a
per-tier map, edited in the console as rows of “model ▾ + complexity ▾” pairs
(plus ×/add) inside each task:

  [routing.tasks.code]
  low  = ["cheap-coder"]
  high = ["best-coder"]
  any  = ["generic-coder"]   # leads at every tier after the tier pick

Chain order: the task's tier-specific list → (at high: the global high chain,
so escalation still beats any-tier pins) → the task's any-tier list → the
global tier chain → default as the last resort; deduplicated. Plain-array
rules keep meaning "any tier" unchanged, and the short-lived {models, tier}
form from the previous commit is normalized into the map on load.

Verified live: legacy config migrated visibly in the editor, pairs survive
save+reload, a math prompt at low ignored the math@high pair, per-tier leads
confirmed by unit tests (19 passing).
2026-08-08 08:54:27 +03:00
infosave2007 95e560dc72 feat: per-key spend accounting and per-task complexity pinning
API keys page: the table now answers "which department spends what" — per-key
requests, tokens in/out and cost for a selectable window (24h/7d/30d) plus a
cumulative all-time cost column, joined from stats by the key's account.

The stats log stays bounded while the accounting survives: once stats.jsonl
exceeds max_file_mb (default 16, 0 disables), records older than the
retention window fold into cumulative per-account totals in a small sidecar
file (tmp+rename on both) and leave the log. alltime = folded + in-file, a
record is never counted twice, verified end-to-end on synthetic data
(1.2 MB → 301 KB, totals identical before/after compaction and restarts).

Routing: a task rule can now PIN the complexity tier —
`[routing.tasks.legal] models=[...] tier="high"` (old plain-array rules parse
unchanged). The pinned tier replaces the router's verdict before the
escalation check and reports itself as route_source "router+task-rule@high".
The Routing page grows a per-task «Complexity» selector (Auto/low/medium/
high); verified live: a code prompt with a pinned high went
router+task-rule@high → the expensive model.

13 new strings in all 7 languages; docs/ROUTING(.ru) updated.
2026-08-08 08:42:11 +03:00
infosave2007 ef28a6633d fix(benchmark): hard probes survive their own popularity
The router deliberately scores familiar text as less novel, so every
benchmark run slightly burns the hard-prompt pool — with 3 templates per
test a variant would drift below the high band within a handful of runs
and flip the test red for no real reason.

Two-part fix: pools doubled to 6 interdisciplinary families each (science:
+ quantum error correction, soil-carbon MRV, cryo-EM; ops: + ED triage +
staffing, port berth/tides/rail, agri-drone fleets — all probed high against
the live router), and a route-judged test that lands non-high gets ONE retry
with a freshly generated template before failing. A single burnt wording no
longer fails the assertion; a genuinely unreachable high tier still does.
2026-08-07 22:47:28 +03:00
infosave2007 b4d36334bf chore: release 0.2.39
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-07 22:25:52 +03:00
infosave2007 dd9ccaea94 fix(shell): stale tabs now know the gateway was updated
Recurring confusion all day: the SPA lives in an open tab until reload, so
after a gateway rebuild/update users kept seeing (and benchmarking with!) an
old interface — "where is the feature", "why are the test ids old". The meta
endpoint now carries the process start timestamp; the existing 20s health
poll compares it, and when the gateway has restarted the tab shows a fixed
banner: "The gateway was updated — this page is running an older interface"
with a one-click Reload. 2 strings in all 7 languages.
2026-08-07 22:14:08 +03:00
infosave2007 ff8415d7d4 feat(routing): high tier escalates over task rules; benchmark generates hard prompts that stay hard
Routing semantics fix observed live: a task rule ("code → cheap model") was
pinning even high-complexity prompts to the cheap model — the tier system
never got a say. At tier=high the tier chain now leads and the task rule
demotes to fallback; low/medium keep task-rule-first. Unit-tested both ways.

The benchmark's hard probes are now GENERATED per run from two pools
(science: protein folding / telescope scheduling / materials discovery;
ops: epidemic logistics / grid markets / climate risk) with rotating domains
and numbers. Two findings against the live router drove this: ambiguity
carries the biggest weight in the complexity score, so interdisciplinary
prompts rate high reliably (0.52-0.66) while single-domain "hard" prompts
classify too confidently and sink to medium; and any fixed wording loses
novelty on every repeat. Their labels are null — straddling task labels is
precisely what makes them high, so classification is not judged.

Verified end-to-end: full run 100% answers, 13/13 classification, both hard
probes high → routed to the expensive model (one of them over a code task
rule), at $0.0018 each thanks to the tiny route-judge token budget.
2026-08-07 21:34:24 +03:00
infosave2007 7479cc51c2 feat(benchmark): hard tests that light up the high tier — judged on routing, not essays
Two "hard" probes (full suite only): a compiler-backend design and a security
protocol design, each dense enough for the router's complexity model to rate
them high. They are judged on the ROUTER's decision (judge: "route") — pass
means tier=high was reached and the request went to the big model; the answer
itself is not scored (open-ended essays aren't keyword-judgeable, and
reasoning models happily burn 3000+ thinking tokens on them — measured). With
max_tokens kept tiny the probes cost cents even on the priciest model, while
the cost column tells the whole story: $0.0148 for the high-tier request next
to $0.00005 for the easy ones.

Each run picks a random variant of the prompt: the router deliberately scores
familiar text as less novel, so a fixed wording would decay to medium over
repeated runs. In pinned-models mode route-judged tests are skipped.

Also: creative-writing budget 96→512 (deepseek was spending the whole cap
thinking and returning an empty poem), and when a model does burn its entire
budget inside <think> the table now says so ("budget spent on reasoning")
instead of the misleading "empty answer".

Server-side (cortiq-router prod, both replicas rolling-restarted): the high
band started at 0.66 while the complexity model tops out ~0.65 on the hardest
single prompts — high was unreachable. Band boundary moved to 0.50 based on
measured distribution (easy 0.25-0.36, medium 0.37-0.49, hard 0.50-0.65);
config backed up as config.prod.toml.bak-tiers.
2026-08-07 20:15:46 +03:00
infosave2007 b1e0dc76ec fix(benchmark): fair to reasoning models, fair to router degradation, real sub-cent costs
Three false signals in benchmark results, all seen on a live run:

- Reasoning models were scored ❌ because the per-test max_tokens cap cut
  them mid-<think> before the actual answer. Every run now gets +256 tokens
  of headroom; non-thinking models stop at EOS as before, thinking ones get
  to finish (math-mult on nanbeige: false 0/1 → honest 1/1).
- When the router degraded (timeout → fallback chain) the table showed
  "✗ degraded" and counted it against classification accuracy. There was no
  classification to judge — such rows now show a muted "⚠ router degraded"
  with an explanatory tooltip and are excluded from the accuracy KPI.
- Costs below $0.0001 (cheap flash models) collapsed to "$0.0000". money
  formatting now grows precision until two significant digits survive
  ($0.000035 instead of $0.0000).

2 new strings in all 7 languages.
2026-08-07 19:48:03 +03:00
infosave2007 c3dd251cce fix(keys): drop the allowed-models picker from the new-key dialog
Creating a key now asks only account, rate limit and optional custom value —
allow_models is sent empty (= all models). Per-key model restrictions stay
editable in Settings for whoever actually needs them.
2026-08-07 19:41:56 +03:00
infosave2007 9af51b679e fix(bench): newer cortiq pretty-prints --json — parse it, and report honest tok/s
The one-click model benchmark showed "error" even though cortiq bench ran
fine: the parser expected the JSON result on a single line, while newer
cortiq releases pretty-print it across many. Extraction now finds the last
balanced JSON object anywhere in the mixed CLI output (both formats work).

Also hardened tok/s selection: steady decode speed (decode_tok_s_steady)
is preferred over incl-prefill, and the loose "contains tok+per" fallback —
which would happily report allocs_per_token=769 as tok/s — now only matches
*tok_s* / *tokens_per* fields.
2026-08-07 19:32:47 +03:00
infosave2007 1bba427042 feat(keys): one-paste harness setup — Claude Code, Codex, OpenCode, Continue, Cline
The "connect your app" card in the API-keys view now leads with auto-setup
scripts for popular coding harnesses, each pre-filled with this gateway's URL
and the freshly created key:

  Claude Code — env exports (the gateway speaks Anthropic /v1/messages
      natively; the protocol is now ON by default, and when a config has it
      off the tab shows a warning with a one-click Enable button)
  Codex — appends a [model_providers.cortiq] block to ~/.codex/config.toml
  OpenCode — writes the provider into opencode.json
  Continue — appends the model to ~/.continue/config.yaml
  Cline — the exact values to paste into the extension settings

The gateway also detects which harnesses are already present on its machine
(binaries across PATH + common install dirs, config files under home) via
GET /admin/api/harnesses: installed tabs get a ✓, and the status line under
the snippet says "installed" or "already configured for this gateway" when
the harness config mentions cortiq. Works on Linux/macOS/Windows.

curl / python / javascript SDK snippets stay as before. 16 new strings in
all 7 languages.
2026-08-07 19:26:54 +03:00
infosave2007 54b73f1365 fix: honest costs in streams, oracle-sized timeouts, history & log cleanup
Cost showed $0 for every streamed request: the X-Cortiq-Cost-USD header is
written before generation, when the cost cannot be known. The gateway now
ships the final number in a closing SSE event (`cortiq_cost_usd`, alongside
the token estimate when the provider sent no usage), and the console prefers
it over the header. Money renders with 4 decimals, the last two in a smaller
font — $0.0006 stays readable instead of collapsing to $0.

Timeouts, both ends: the router default was 800 ms, silently degrading every
prompt the hosted router escalates to its oracle (~10 s) — now 15 s. Provider
clients had NO timeout at all (a dead OpenRouter host hung requests forever) —
now a 10 s connect timeout, with no overall cap so long generations and
streams are never cut.

Benchmark history: per-run × and a Clear-all button. Logs view: a clear
button backed by DELETE /admin/api/logs.
2026-08-07 19:15:27 +03:00
infosave2007 ec92bdddb7 feat(benchmark): smart-routing mode is the point — live results, per-model cost
The benchmark now leads with the product's actual feature. Two explicit modes:

  🧭 Smart routing (default) — every probe goes through cortiq-auto; each row
     shows the router's task label judged against the expected one (✓/✗ with
     "expected: …"), the complexity tier, the chosen model with what that
     request cost, and a task-rule badge when a routing rule fired. KPIs:
     answer quality, classification accuracy, TTFT, tok/s and money spent,
     plus a "requests went to" distribution with per-model cost.
  🎯 Individual models — the old head-to-head comparison, now as the clearly
     separate option (model checkboxes appear only here).

The suite covers all 8 taxonomy labels (added summarization, creative-writing
and chitchat probes; expected labels annotated per test). Results are LIVE:
the table grows a row after every test (with a small fade-in), the progress
card shows the prompt being asked, and a 🏁 verdict toast lands at the end
with the score and the total spend.

12 new strings in all 7 languages.
2026-08-07 19:04:41 +03:00
infosave2007 1120d8f11a feat(benchmark): the built-in benchmark, redesigned into the console
The old standalone benchmark.html (Russian-only, Tailwind+Chart.js from CDNs,
never shipped in the crate) becomes a proper sidebar view. Same scoring ideas,
reimplemented: keyword checks with |-synonyms, think-tag stripping, tail-loop
detection, pass/partial/fail — plus what the old page never had: runs go
through the LIVE pipeline (pinned per model, streamed), so every test also
measures tok/s and time-to-first-token with the gateway's real token counts.

A built-in suite of 12 language-neutral probes (math, logic, code, QA,
instruction-following, extraction, translation, JSON formatting); the ⚡ quick
mode runs the 6 short ones — about a minute on a local model. Results: score
bars per model sorted best-first, a per-test grid (click a cell for the full
answer), previous runs and custom tests kept in the browser.

22 new strings in all 7 languages.
2026-08-07 18:56:47 +03:00
infosave2007 914ff42fc5 release: v0.2.38 — media generation, task-based routing, ops batch
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-07 18:34:00 +03:00
infosave2007 2ede0ee452 feat(routing): per-task rules on top of complexity tiers; hosted router is the default
The router has always returned two signals — task label and complexity tier —
but only the tier picked models. [routing.tasks] adds `label → ordered models`:
a matching rule's models lead the candidate chain and the tier chain follows as
fallback, so pinning translations to a local model never weakens failover.
Requests whose chain was headed by a task rule report
route_source "router+task-rule".

The Routing page grows a per-task editor built from the account's LIVE
taxonomy (GET /v1/taxonomies via the new /admin/api/routing/labels proxy;
built-in label set when the router is unreachable), with localized task names,
each task's share of 7-day traffic, "by complexity (no rule)" as the explicit
empty state, and custom labels for other taxonomies. Deleting a model scrubs
it from task rules like it always did from tiers.

Also fixes the default config pointing at a placeholder localhost:8080 router
with a nonexistent taxonomy — a fresh install now targets the hosted allaigate
router (IP endpoint, verify_tls off, data-assistant, 12 s oracle timeout) and
degrades gracefully until a key is set. Verified end-to-end against the live
router: translation → task-rule model, math → tier path.

16 new strings in all 7 languages.
2026-08-07 17:39:38 +03:00
infosave2007 15de36be3b feat: ops & UX batch — jobs survive restarts, logs/system in the console, media queue, gallery, chat, exact stream tokens
Fifteen changes aimed at the first day of real use:

- Import and media jobs persist to <data>/jobs-*.json and reload on start
  (running ones settle as errors); the Register button no longer vanishes
  because the gateway restarted.
- Docker ships the cortiq binary (built in the builder stage) — local .cmf
  chat models and image/video generation now work in containers, where
  auto-install never could (no cargo in the runtime image). libvulkan1 added
  for optional GPU offload.
- Open admin console on 0.0.0.0 without a token now warns in the log and
  shows a dismissible banner — the VPS footgun is at least labeled.
- "Restart gateway" button in Settings: stops managed model servers, spawns
  a fresh copy (bind retried while the port frees), Docker relies on the
  container restart policy instead.
- Dashboard "System" panel: CPU sparkline + load, RAM (with the gateway's
  own RSS), free disk at the models dir — sampled by sysinfo every 2 s.
- Featured model cards warn "not enough disk" / "may not fit in RAM" before
  the download, from the same live snapshot.
- Log viewer in the sidebar: a 500-line in-memory ring fed by a second
  ANSI-free tracing layer; filter, level, pause, copy.
- Update check: crates.io polled twice a day, the console shows an
  "update available" notice.
- Media generations queue up (with position shown) instead of being
  rejected; progress gained an honest ETA from measured throughput.
- Generation gallery under the image/video tabs: lazy PPM thumbnails,
  click to reopen with seed/steps, delete.
- Playground chat is a real dialogue now: history is resent, bubbles,
  Clear, Cmd/Ctrl+Enter.
- "Connect your app" snippets (curl / Python / JS) on the keys page, with
  the real key inlined right after creation.
- One-click benchmark on the Models page via `cortiq bench --json`.
- Config export now downloads the actual gateway.toml (comments intact);
  import accepts TOML (validated, hot-reloaded) and the legacy JSON.
- Streaming token counts are exact when the model server provides usage
  (cortiq-server patch, shipped separately); the gateway fallback now
  counts content chunks (1/token on cortiq) instead of chars/4.

Wizard's Finish lands on the Playground. 37 new strings in all 7 languages.
2026-08-07 17:07:48 +03:00
infosave2007 a77beeabab feat(playground): image & video generation tabs + real chat metrics
The infosave collection now ships media models (Lumina-Image-2.0, MiniMax-H3
Turbo) that cortiq runs via `imagine`/`animate` CLI, not `cortiq serve`. The
gateway grows a media API: local .cmf files are classified by `cortiq info`
(chat/image/video, cached), generations run as tracked jobs with step progress
parsed from stdout, one at a time, cancellable, outputs under <data>/media.

The playground becomes three tabs. Image: width/height/steps/CFG/seed, live
progress, result rendered from P6 PPM to a canvas in the browser (PNG export
via toBlob). Video: frames/steps/quality/seed plus optional first/last
keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded in JS
and played on a canvas synced to the wav — no ffmpeg on either side (mp4 remux
happens opportunistically when ffmpeg exists). A metrics card shows resolution,
steps, seed, total time, time per step/frame and file size.

Chat metrics: the local cortiq server sends no usage chunk in streams, so
token counts showed 0/0. tap_stream now injects one OpenAI-shaped usage event
ahead of [DONE] with the gateway's ~4-chars/token estimate, flagged
`cortiq_estimated`; the UI renders it as "≈ N / M" and adds tok/s and
time-to-first-token. Registration skips media packs (a `cortiq serve` entry
for a diffusion model would only fail) in both Register and the wizard's
Finish.

36 new UI strings translated across all 7 languages.
2026-08-07 15:55:06 +03:00
63 changed files with 8186 additions and 302 deletions
+253
View File
@@ -3,6 +3,259 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.46] - 2026-08-21
Verified end to end against a real Open WebUI container talking to the gateway.
### Fixed
- **Offered `tools` no longer make a local model unroutable.** Open WebUI attaches its ~26
built-in tools (knowledge bases, notes, calendar, …) to *every* message, and the gateway
treated that offer as a hard capability requirement — so a plain "hello" against a pool of
tools-less local models failed with «No healthy model available that satisfies required
capabilities». In the OpenAI contract `tools` is an offer: only `tool_choice: "required"`
(or a named function) makes the call impossible without them. The gateway now still
prefers a tool-capable model, but falls back to serving the request with the tools
stripped (along with `tool_choice`/`parallel_tool_calls`, which a provider rejects on
their own). `tool_choice: "required"` still fails loudly, as it must.
- **A pinned model that merely lacks tool support is honoured** instead of being reported as
«not available — it is not in the pool (deleted?)». An unknown pin still fails loudly.
### Added
- `X-Cortiq-Tools-Dropped: true` (and `cortiq.tools_dropped` in the echo block) whenever the
gateway answers without the tools the client offered, so the downgrade is never silent.
## [0.2.45] - 2026-08-21
### Fixed
- **Open WebUI (and every strict OpenAI client) can finally chat with the gateway.**
`POST /v1/chat/completions` rejected perfectly ordinary payloads with a bare HTTP 422 —
a plain-text body no OpenAI client can parse, which is why the chat window just spun with
no answer and no error. The request parser now accepts every shape real clients send:
`content` as a string, `null`, or an array of content parts (text parts concatenated,
images skipped), `tools: null`, `stream: null`, a string `reasoning_effort`
(`low`/`medium`/`high`) as well as a numeric budget, `max_completion_tokens` as an alias
for `max_tokens`, `messages[].name` and `messages[].tool_call_id` proxied through, and a
missing `Content-Type`. A body that genuinely cannot be read now returns an OpenAI error
envelope with HTTP 400.
- **Strict OpenAI response JSON, no optional `null`s.** Absent tool calls no longer emit
`"tool_calls": null` (the key is omitted entirely); `finish_reason` is mapped to the
spec's closed set, so `cortiq serve`'s `max_tokens` reaches clients as `length`; the
response always carries a non-empty `id`.
- **`model` echoes the id the client requested** (`cortiq-auto` stays `cortiq-auto`, not the
upstream's `cortiq`), matching OpenAI's own behaviour. The model that actually answered
stays in `cortiq.selected_model` and `X-Cortiq-Selected-Model`. Applies to
`/v1/chat/completions`, `/v1/completions` and `/v1/messages`.
- **Streaming is re-normalised instead of forwarded verbatim.** Every chunk is reissued as a
strict `chat.completion.chunk`: the stream opens with a `role` delta, `message`-shaped
chunks are converted to `delta`, empty deltas are dropped, and it always closes with one
terminal `finish_reason` and `data: [DONE]` — even when the upstream stops early or dies
mid-flight, in which case the failure arrives as an SSE `error` event rather than a client
waiting forever. SSE responses also carry `X-Accel-Buffering: no` so nginx/Traefik cannot
buffer the stream into a hang.
- **Client bookkeeping keys are no longer proxied upstream.** `chat_id`, `session_id`,
`metadata`, `features`, `background_tasks` and friends (plus a non-string `user`) are
stripped before the provider call, so strict backends stop rejecting the request.
### Added
- **`GET /v1/models` lists managed local CMF models.** Every `[[cmf.servers]]` entry now
appears alongside `cortiq-auto` and the `[[models]]` pool, with live `running`/`healthy`
state in a non-standard `cortiq` field, so Open WebUI and other clients can pin a local
`.cmf` directly. Entries also carry `created`.
### Changed
- Dropped leftover `[to_wire]` debug prints from the OpenAI provider — they wrote to stderr
on every single request.
## [0.2.44] - 2026-08-08
### Added
- **Power controls in Settings**: «Stop everything & power off» stops every managed local
model server and exits the gateway (the panel shows a clear "gateway is off" screen);
«Restart everything» (relabeled) re-execs the gateway with its local servers. Both behind
confirm dialogs, translated in all seven languages.
### Fixed
- `cortiq-gateway` typed in any directory picks up your configuration. Resolution order:
`--config` → a readable `./config/gateway.toml` → `$CORTIQ_GATEWAY_HOME` →
`~/.cortiq-gateway/gateway.toml`. An unrelated project that merely contains `Cargo.toml`
and a `config/` folder no longer hijacks the config path ("cannot read config
config/gateway.toml").
## [0.2.43] - 2026-08-08
### Added
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
configured = open mode, as before.
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
when the server becomes healthy, so a local model whose chat template can drive tool
calling serves agent (tools) traffic with zero manual configuration. Manual override:
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
remaining credit is a KPI card, red under $5.
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
per harness, an install command chip when the harness is missing, and a paste-your-key
field that substitutes the real key into every snippet.
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
Caddy TLS fronting, security checklist, backup list.
### Fixed
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
route's chain has no model with the required capabilities (tools), the gateway escalates
through medium→high chains and then the rest of the pool instead of returning
"No healthy model available". Explicit pins still fail loudly.
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
no history-expansion `!`), and the install chips are readable in the light theme.
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
into the sidecar totals file — per-account all-time and month spend survive compaction
and restarts.
### Breaking
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
will now be rejected with 401 — either remove the keys from the config or start sending
`Authorization: Bearer <key>`.
## [0.2.42] - 2026-08-08
### Added
- **The continuous complexity score is visible**: the benchmark's routing table and the
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
see what different prompts actually weigh and tune the boundary sliders from real traffic.
### Changed
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
«Рабочий стол».
## [0.2.41] - 2026-08-08
### Added
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
(the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
other, the numbers ride on the handles and the legend updates live — no hand-typed score
ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
(`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
stats; unchecked = the router's own bands, untouched behavior.
### Fixed
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
fresh template — difficulty is judged in the background, so the second occurrence is the
honest steady-state answer.
## [0.2.40] - 2026-08-08
### Added
- **Per-key spend accounting** on the API-keys page: requests, tokens in/out and cost per key
for a selectable window (24h / 7d / 30d) plus a cumulative all-time cost column — "which
department spends what" at a glance.
- **Bounded stats on disk**: once `stats.jsonl` outgrows `stats.max_file_mb` (default 16, 0
disables), records older than the retention window fold into cumulative per-account totals in
a small sidecar file (tmp+rename on both) and leave the log. All-time accounting survives
compaction and restarts; nothing is ever counted twice.
- **Per-complexity models inside a task rule**: each task is edited as rows of
"model + complexity" pairs — `[routing.tasks.code] low=["cheap"] high=["best"] any=[...]`.
The tier-specific pick leads its own tier, `any` (and legacy plain arrays) leads everywhere,
the global tier chain and `default` always remain as fallback, and high verdicts still
escalate over any-tier pins.
### Changed
- Benchmark hard probes are generated from wider interdisciplinary pools (6 families each) and
retry once with a fresh template — the router's novelty decay no longer flips them red.
### Fixed
- The `task-rule` badge is honest: it appears only when the rule actually chose the head of the
candidate chain, not whenever a rule merely exists for the task.
## [0.2.39] - 2026-08-07
### Added
- **Benchmark, redesigned into the console** (⏱ sidebar entry): two explicit modes. **Smart
routing** (default) sends every probe through `cortiq-auto` and judges the ROUTER — task label
vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge —
with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a
"requests went to" per-model distribution. **Individual models** compares pinned models
head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run ×
and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
- **Hard probes that exercise the high tier**: two route-judged tests (science / ops pools)
generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings
decay with the router's novelty signal. Pass = the router said `high` and the request escalated;
the tiny token budget keeps them at cents even on the priciest model.
- **One-paste harness setup** on the API-keys page: Claude Code (native Anthropic `/v1/messages`,
now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline —
each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses
already installed on its machine (✓ on the tab, "already configured for this gateway" status).
- **Stale-tab detection**: open consoles notice a gateway restart via the meta start marker and
show a reload banner instead of silently running an old interface.
- Log viewer Clear button (`DELETE /admin/api/logs`).
### Changed
- **High complexity escalates over task rules**: at tier `high` the tier chain leads and the task
rule demotes to fallback — a `code → cheap-model` rule no longer pins the hardest prompts to
the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
- **Honest stream costs**: the final cost rides a closing SSE event (`cortiq_cost_usd`) since the
initial header is written before generation; money renders with 4 decimals (last two smaller)
and grows precision below $0.0001 instead of collapsing to $0.0000.
- **Oracle-sized timeouts**: router default 800 ms → 15 s (hosted escalations take ~10 s);
provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall
cap so long generations and streams are never cut.
- Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-`<think>`),
router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on
reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
- New-key dialog simplified to account / rate limit / optional custom value.
### Fixed
- `cortiq bench` results showed "error" with newer cortiq releases that pretty-print `--json`:
the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers
steady decode speed (the old heuristic could report `allocs_per_token` as tok/s).
## [0.2.38] - 2026-08-07
### Added
- **Image & video generation in the Playground**: three tabs now — Chat, Image, Video. Image runs
`cortiq imagine` over a local text-to-image `.cmf` (Lumina) with width/height/steps/CFG/seed,
live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser
with one-click PNG export. Video runs `cortiq animate` (MiniMax-H3) with frames/steps/quality
and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded
in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens
opportunistically when ffmpeg exists). Local `.cmf` files are classified chat/image/video via
`cortiq info`; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56.
- **Task-based routing**: `[routing.tasks]` maps a router task label to an ordered model list.
A matching rule's models lead the candidate chain and the complexity chain follows as fallback,
so a task pin never weakens failover. The Routing page builds the editor from the account's
live taxonomy (`GET /v1/taxonomies`), shows each task's share of 7-day traffic, and supports
custom labels. Responses whose chain was headed by a rule report `route_source "router+task-rule"`.
- **Generation gallery** under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete),
and generations **queue up** (with position shown) instead of being rejected.
- **Chat is a real dialogue**: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live
metrics: token counts, tok/s and time-to-first-token.
- **Exact stream token counts** when the model server provides usage; the gateway fallback counts
content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate.
- **Jobs survive restarts**: import and media job lists persist to `<data>/jobs-*.json` and are
reconciled on start — the Register button no longer vanishes because the gateway restarted.
- **Admin console, ops batch**: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy);
a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at
the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download;
a Restart-gateway button; an update-available notice (crates.io checked twice a day); config
export as the real `gateway.toml` and TOML import (validated, hot-reloaded); "Connect your app"
snippets (curl/Python/JS) on the keys page; one-click `cortiq bench --json` on the Models page.
- **Docker ships the cortiq binary** (built in the builder stage, plus `libvulkan1`): local `.cmf`
chat models and media generation now work in containers, where auto-install never could.
- **Security**: an open admin console listening on 0.0.0.0 outside Docker warns in the log and
shows a dismissible banner.
### Fixed
- The default config pointed at a placeholder `localhost:8080` router with a nonexistent taxonomy,
leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now
targets the hosted allaigate router and degrades gracefully until a key is set.
- The wizard's Finish lands on the Playground instead of the Models list.
## [0.2.37] - 2026-08-06
### Added
Generated
+219 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "cortiq-gateway"
version = "0.2.37"
version = "0.2.46"
dependencies = [
"anyhow",
"arc-swap",
@@ -302,6 +302,7 @@ dependencies = [
"rust-embed",
"serde",
"serde_json",
"sysinfo",
"thiserror 1.0.69",
"tokio",
"tokio-stream",
@@ -341,6 +342,16 @@ dependencies = [
"crypto-common",
]
[[package]]
name = "dispatch2"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
dependencies = [
"bitflags",
"objc2",
]
[[package]]
name = "displaydoc"
version = "0.2.6"
@@ -873,6 +884,15 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "ntapi"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae"
dependencies = [
"winapi",
]
[[package]]
name = "nu-ansi-term"
version = "0.50.3"
@@ -882,6 +902,63 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "objc2"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a12a8ed07aefc768292f076dc3ac8c48f3781c8f2d5851dd3d98950e8c5a89f"
dependencies = [
"objc2-encode",
]
[[package]]
name = "objc2-core-foundation"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
dependencies = [
"bitflags",
"dispatch2",
"objc2",
]
[[package]]
name = "objc2-encode"
version = "4.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33"
[[package]]
name = "objc2-foundation"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
dependencies = [
"bitflags",
"objc2",
]
[[package]]
name = "objc2-io-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15"
dependencies = [
"libc",
"objc2-core-foundation",
]
[[package]]
name = "objc2-open-directory"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb82bed227edf5201dfedf072bba4015a33d3d4a98519837295a90f0a23f676d"
dependencies = [
"objc2",
"objc2-core-foundation",
"objc2-foundation",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -1430,6 +1507,21 @@ dependencies = [
"syn",
]
[[package]]
name = "sysinfo"
version = "0.39.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2071df9448915b71c4fe6d25deaf1c22f12bd234f01540b77312bb8e41361e6"
dependencies = [
"libc",
"memchr",
"ntapi",
"objc2-core-foundation",
"objc2-io-kit",
"objc2-open-directory",
"windows",
]
[[package]]
name = "thiserror"
version = "1.0.69"
@@ -1914,6 +2006,22 @@ dependencies = [
"rustls-pki-types",
]
[[package]]
name = "winapi"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-util"
version = "0.1.11"
@@ -1923,12 +2031,113 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580"
dependencies = [
"windows-collections",
"windows-core",
"windows-future",
"windows-numerics",
]
[[package]]
name = "windows-collections"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610"
dependencies = [
"windows-core",
]
[[package]]
name = "windows-core"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
dependencies = [
"windows-implement",
"windows-interface",
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-future"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb"
dependencies = [
"windows-core",
"windows-link",
"windows-threading",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-interface"
version = "0.59.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-numerics"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26"
dependencies = [
"windows-core",
"windows-link",
]
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-strings"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-sys"
version = "0.52.0"
@@ -1989,6 +2198,15 @@ dependencies = [
"windows_x86_64_msvc 0.53.1",
]
[[package]]
name = "windows-threading"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37"
dependencies = [
"windows-link",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.52.6"
+4 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "cortiq-gateway"
version = "0.2.37"
version = "0.2.46"
edition = "2021"
rust-version = "1.75"
license = "Apache-2.0"
@@ -61,6 +61,9 @@ clap = { version = "4", features = ["derive"] }
# open browser on first run (optional)
open = { version = "5", optional = true }
# host metrics for the admin dashboard (CPU / RAM / disk)
sysinfo = "0.39"
# errors / logging / metrics
thiserror = "1"
anyhow = "1"
+8 -2
View File
@@ -3,6 +3,10 @@
# drifts to a newer glibc than the runtime image and the binary refuses to start
FROM rust:1-slim-trixie AS builder
WORKDIR /app
# the cortiq runtime CANNOT be auto-installed inside the container (no cargo in
# the runtime image), so it is compiled here and shipped alongside the gateway —
# local .cmf chat models and image/video generation work in Docker out of the box
RUN cargo install cortiq-cli --locked
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
COPY Cargo.toml Cargo.lock ./
COPY src ./src
@@ -11,13 +15,15 @@ RUN cargo build --release
# ---- runtime stage ----------------------------------------------------------
FROM debian:trixie-slim
# curl is required at runtime — model imports download ready .cmf files with it
# curl — model imports download ready .cmf files with it;
# libvulkan1 — optional GPU offload for cortiq (falls back to CPU without a GPU)
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& apt-get install -y --no-install-recommends ca-certificates curl libvulkan1 \
&& rm -rf /var/lib/apt/lists/* \
&& useradd -r -u 10001 cortiq \
&& mkdir -p /app/config /app/data/models
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
COPY --from=builder /usr/local/cargo/bin/cortiq /usr/local/bin/cortiq
# ship the example as the default config so `docker run` works out of the box
COPY config/gateway.example.toml /app/config/gateway.toml
RUN chown -R 10001:10001 /app
+5 -2
View File
@@ -40,7 +40,8 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down.
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. Plus **per-task rules** (`[routing.tasks]`): pin translations to a local model and code to your strongest coder — the complexity chain stays as fallback ([docs](docs/ROUTING.md)).
- **Image & video generation.** The Playground drives local text-to-image (Lumina) and text-to-video (MiniMax-H3) `.cmf` models via `cortiq imagine`/`animate`: live progress with ETA, a generation gallery, PNG export, in-browser AVI playback — no ffmpeg anywhere.
- **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard).
- **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes.
- **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint.
@@ -86,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
---
## 🖥️ Admin console
@@ -137,7 +140,7 @@ The wizard takes you from nothing to a working local model:
| **Language** | 7 languages; changeable later from the header |
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, and `routing.default` points at it |
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, `routing.default` points at it, and you land in the Playground |
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
+5 -2
View File
@@ -39,7 +39,8 @@
- **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели.
- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. Плюс **правила по задачам** (`[routing.tasks]`): переводы — в локальную модель, код — сильнейшему кодеру; цепочка по сложности остаётся резервом ([доки](docs/ROUTING.ru.md)).
- **Генерация картинок и видео.** Песочница запускает локальные text-to-image (Lumina) и text-to-video (MiniMax-H3) `.cmf` модели через `cortiq imagine`/`animate`: живой прогресс с ETA, галерея генераций, экспорт в PNG, проигрывание AVI прямо в браузере — без ffmpeg вообще.
- **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
- **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
- **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга.
@@ -81,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
---
## 🖥️ Панель управления
@@ -129,7 +132,7 @@ cortiq-gateway
| **Язык** | 7 языков; потом меняется в шапке |
| **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки |
| **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` |
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё |
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё — и вы попадаете в Песочницу |
Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели.
Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг
+9 -5
View File
@@ -14,11 +14,11 @@ listen = "0.0.0.0:9000"
# NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the
# hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default.
[router]
url = "http://localhost:8080"
url = "https://138.226.222.209" # hosted allaigate router (or your own cortiq-router URL)
api_key_env = "CORTIQ_ROUTER_KEY" # name of the env var holding the router key
timeout_ms = 800 # timeout for the /v1/route call (12000+ for oracle)
taxonomy_id = "general.task-type" # for allaigate: "data-assistant"
verify_tls = true # false if the router uses an IP / self-signed cert
timeout_ms = 12000 # hard prompts escalate to an oracle (~10 s)
taxonomy_id = "data-assistant" # the hosted router's task taxonomy
verify_tls = false # the hosted endpoint is IP-bound; true for your own domain
# --- how to pick the text to classify from the conversation -----------------
[route]
@@ -71,7 +71,7 @@ openai_chat = true
openai_completions = true
openai_embeddings = true
openai_models = true
anthropic_messages = false
anthropic_messages = true
mcp = false
native_passthrough = true
@@ -119,6 +119,10 @@ enabled = true
file = "config/stats.jsonl" # empty = in-memory only
ring_size = 500 # how many recent requests to keep in the feed
retention = "7d" # aggregate/series depth (replayed on startup)
# stats.jsonl is compacted once it exceeds this size: old records fold into
# cumulative per-account totals (sidecar file), so per-key accounting survives
# while the disk stays bounded. 0 disables.
max_file_mb = 16
# ============================================================================
# Semantic cache — returns a cached answer for prompts semantically near a
+122
View File
@@ -0,0 +1,122 @@
# Deploying cortiq-gateway for a team / office
One gateway on a small server, API keys per department, local models for the
cheap tiers, cloud models for the hard ones. This page covers the ops side.
## 1. Run as a service
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Task Scheduler → Create Task → "Run whether user is logged on or not",
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS and the outside world
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
and bind the gateway to localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE streaming
proxy_read_timeout 3600s; # long generations
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
same with automatic certificates.)
## 3. Security checklist
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
the console warns that it is open to the network.
- **API keys**: create one per department/developer on the keys page. With at
least one key configured the model APIs require Bearer / x-api-key —
requests without a valid key get 401.
- **Rate limits**: per-key requests/min → 429 with Retry-After.
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
account's month-to-date spend crosses it.
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
## 4. Backups
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
- `secrets.toml` — provider API keys (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
cumulative per-key totals and month-to-date budget spend.
Copy those three and you can rebuild the instance anywhere.
## 5. Observability
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
OpenRouter balance (red under $5).
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
+123
View File
@@ -0,0 +1,123 @@
# Развёртывание cortiq-gateway для команды / офиса
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
## 1. Запуск как сервис
### Linux (systemd)
```ini
# /etc/systemd/system/cortiq-gateway.service
[Unit]
Description=Cortiq Gateway
After=network-online.target
[Service]
User=cortiq
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl enable --now cortiq-gateway
```
### macOS (launchd)
```xml
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>com.cortiq.gateway</string>
<key>ProgramArguments</key><array>
<string>/usr/local/bin/cortiq-gateway</string>
<string>--no-port-search</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
</dict></plist>
```
```bash
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
```
### Windows
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
--no-port-search`. Либо [NSSM](https://nssm.cc):
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
### Docker
```bash
docker run -d --restart unless-stopped -p 9000:9000 \
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
```
## 2. TLS и внешний мир
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
привяжите к localhost:
```toml
# gateway.toml
listen = "127.0.0.1:9000"
```
```nginx
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:9000;
proxy_http_version 1.1;
proxy_buffering off; # SSE-стриминг
proxy_read_timeout 3600s; # долгие генерации
}
}
```
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
с автоматическими сертификатами.)
## 3. Чек-лист безопасности
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
него консоль предупреждает, что открыта сети.
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
запросы без валидного ключа получают 401.
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
траты аккаунта за месяц превысят лимит.
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
## 4. Бэкапы
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
- `secrets.toml` — ключи провайдеров (chmod 600);
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
тоталы по ключам и месячные траты для бюджетов.
Скопируйте эти три файла — и инстанс поднимается где угодно.
## 5. Наблюдаемость
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
сложности, баланс OpenRouter (красный ниже $5).
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
+47 -9
View File
@@ -53,7 +53,24 @@ Routing rule common to all protocols:
```
Standard OpenAI fields are supported; unknown fields are proxied to the provider unchanged
(forward-compatible).
(forward-compatible), minus the bookkeeping keys chat UIs attach to the payload
(`chat_id`, `session_id`, `metadata`, `features`, …), which are dropped so strict upstreams
do not reject the call.
The parser is deliberately lenient about the shapes real clients send:
`content` may be a string, `null`, or an array of content parts (text parts are
concatenated, images and other non-text parts are skipped); `tools` may be `null`;
`reasoning_effort` may be a string (`low` | `medium` | `high`) or a token budget;
`max_completion_tokens` is accepted as an alias for `max_tokens`. A body that still
cannot be read comes back as an OpenAI error envelope with HTTP 400 — never a bare 422,
which OpenAI clients cannot parse (they show no answer at all).
`tools` is treated as an **offer**, not a requirement. The gateway prefers a tool-capable
model, but when none is available it answers without the tools rather than failing, and says
so via `X-Cortiq-Tools-Dropped: true` (chat UIs attach their built-in tools to every message,
so the strict reading made ordinary chat impossible on a tools-less local pool).
`tool_choice: "required"` — or a named function — is a real requirement and still fails with
`502 no_healthy_model` when no model can honour it.
### Response (non-streaming)
@@ -62,12 +79,12 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
"id": "chatcmpl-9f...",
"object": "chat.completion",
"created": 1718800000,
"model": "local-qwen", // the ACTUALLY selected model
"model": "cortiq-auto", // the model the CLIENT asked for, echoed back
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "x = 2 or x = 3" },
"finish_reason": "stop"
"finish_reason": "stop" // always one of stop|length|tool_calls|content_filter
}
],
"usage": {
@@ -86,10 +103,24 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
}
```
> `model` echoes the id the client requested, exactly as OpenAI does — clients match the
> reply against the id they sent. The model that actually answered is in
> `cortiq.selected_model` and in the `X-Cortiq-Selected-Model` header.
> Optional fields are omitted rather than sent as `null`: absent tool calls mean no
> `tool_calls` key at all. `"tool_calls": null` is not in the OpenAI schema and trips
> strict clients.
> The `cortiq` field is an optional extension. OpenAI SDKs ignore it. If the client does
> not want extra fields, disable `cortiq.echo` — the metadata will remain in the response
> headers.
Streaming (`stream: true`) emits strict `chat.completion.chunk` events: the stream always
opens with a `role` delta, carries exactly one terminal `finish_reason`, and ends with
`data: [DONE]` — even when the upstream stops early or dies mid-flight, in which case the
failure arrives as an SSE `error` event. A client can therefore never be left waiting on a
half-formed stream.
### Metadata in response headers (always present)
```
@@ -164,20 +195,27 @@ single user message and follows the same pipeline. Response `object` is `"text_c
## 4. OpenAI Models
`GET /v1/models` — auto-discovery for clients. Returns the gateway's virtual models
(`cortiq-auto`, `cortiq-embed`) and all real models in the pool.
(`cortiq-auto`, `cortiq-embed`), every model in the `[[models]]` pool, and every managed
local `[[cmf.servers]]` model, so a client can pin a local `.cmf` directly.
```jsonc
{
"object": "list",
"data": [
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
{ "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
{ "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
{ "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
{ "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
"cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
]
}
```
Managed CMF models are listed as soon as they are configured, with their live state in the
non-standard `cortiq` field (clients ignore it). They are not filtered by health: a large
`.cmf` takes minutes to load, and a model that disappears from the list while it warms up
is worse than one marked "not ready yet".
---
## 5. Anthropic Messages
@@ -198,7 +236,7 @@ single user message and follows the same pipeline. Response `object` is `"text_c
"id": "msg_...",
"type": "message",
"role": "assistant",
"model": "local-qwen",
"model": "cortiq-auto",
"content": [ { "type": "text", "text": "x = 2 or x = 3" } ],
"stop_reason": "end_turn",
"usage": { "input_tokens": 24, "output_tokens": 11 }
+45 -8
View File
@@ -53,7 +53,24 @@
```
Поддерживаются стандартные поля OpenAI; неизвестные поля проксируются к провайдеру
без изменений (forward-compatible).
без изменений (forward-compatible) — кроме служебных ключей, которые чат-интерфейсы
добавляют в payload для собственных нужд (`chat_id`, `session_id`, `metadata`,
`features`, …): они отбрасываются, чтобы строгий upstream не отклонил вызов.
Парсер намеренно терпим к тем формам, которые реально шлют клиенты: `content` может быть
строкой, `null` или массивом content-частей (текстовые части склеиваются, изображения и
прочие нетекстовые пропускаются); `tools` может быть `null`; `reasoning_effort` — строкой
(`low` | `medium` | `high`) или бюджетом токенов; `max_completion_tokens` принимается как
синоним `max_tokens`. Если тело всё-таки не читается, возвращается OpenAI-конверт ошибки с
HTTP 400 — но не «голый» 422, который OpenAI-клиенты не умеют разбирать (для пользователя
это выглядит как отсутствие ответа).
`tools` трактуется как **предложение**, а не требование. Шлюз предпочитает модель с
поддержкой tools, но если такой нет — отвечает без них, а не падает, и сообщает об этом
заголовком `X-Cortiq-Tools-Dropped: true` (чат-интерфейсы добавляют свои встроенные tools в
каждое сообщение, поэтому строгое прочтение делало обычный чат на локальном пуле без tools
невозможным). `tool_choice: "required"` — или именованная функция — это уже настоящее
требование, и оно по-прежнему завершается `502 no_healthy_model`, если его некому выполнить.
### Ответ (без стриминга)
@@ -62,12 +79,12 @@
"id": "chatcmpl-9f...",
"object": "chat.completion",
"created": 1718800000,
"model": "local-qwen", // РЕАЛЬНО выбранная модель
"model": "cortiq-auto", // модель, которую ЗАПРОСИЛ клиент — возвращается как есть
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "x = 2 или x = 3" },
"finish_reason": "stop"
"finish_reason": "stop" // всегда одно из stop|length|tool_calls|content_filter
}
],
"usage": {
@@ -86,9 +103,22 @@
}
```
> `model` возвращает тот id, который запросил клиент, — ровно как это делает OpenAI:
> клиенты сопоставляют ответ с отправленным id. Модель, которая реально ответила, — в
> `cortiq.selected_model` и в заголовке `X-Cortiq-Selected-Model`.
> Необязательные поля не сериализуются вместо того, чтобы приходить как `null`: если
> tool-вызовов нет, ключа `tool_calls` в ответе нет вообще. `"tool_calls": null` не входит
> в схему OpenAI и ломает строгие клиенты.
> Поле `cortiq` — необязательное расширение. OpenAI-SDK его игнорируют. Если клиент
> не хочет «лишних» полей — выключите `cortiq.echo`, метаданные останутся в заголовках.
Стриминг (`stream: true`) отдаёт строгие события `chat.completion.chunk`: поток всегда
открывается дельтой с `role`, содержит ровно один финальный `finish_reason` и завершается
`data: [DONE]` — даже если upstream оборвался или упал на середине (тогда ошибка приходит
SSE-событием `error`). Клиент никогда не остаётся ждать недосказанный поток.
### Метаданные в заголовках (всегда)
```
@@ -163,20 +193,27 @@ data: [DONE]
## 4. OpenAI Models
`GET /v1/models` — автодискавери для клиентов. Возвращает виртуальные модели шлюза
(`cortiq-auto`, `cortiq-embed`) и все реальные модели пула.
(`cortiq-auto`, `cortiq-embed`), все модели пула `[[models]]` и все управляемые локальные
модели `[[cmf.servers]]` — чтобы клиент мог напрямую закрепить локальный `.cmf`.
```jsonc
{
"object": "list",
"data": [
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
{ "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
{ "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
{ "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
{ "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
"cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
]
}
```
Управляемые CMF-модели попадают в список сразу, как только настроены; их живое состояние —
в нестандартном поле `cortiq` (клиенты его игнорируют). По здоровью они не фильтруются:
крупный `.cmf` грузится минутами, и модель, исчезающая из списка на время прогрева, хуже
модели, помеченной «ещё не готова».
---
## 5. Anthropic Messages
+41
View File
@@ -162,3 +162,44 @@ fine-tune the task mask. Over time this makes routing fit your specific pool mor
enabled = false
# the ground-truth source is set by the integrator (e.g. the X-Cortiq-Correct-Label header)
```
## Task-based routing
The router classifies every request twice: a **task label** (what kind of work it
is) and a **complexity tier** (how hard it is). Tiers pick *how much model* you
need; task rules pick *which* model is best at the job:
```toml
[routing.tasks]
translation = ["local-qwen"] # translations are cheap — keep them local
code = ["claude-sonnet", "gpt"] # code goes to the strongest coder first
```
A matching rule's models go **first** in the candidate chain; the complexity
chain follows as fallback, so a task pin never weakens failover. Tasks without
a rule route by complexity alone.
**Per-complexity models for a task.** A task rule also accepts a per-tier
map — "which model serves this task at each complexity":
```toml
[routing.tasks.code]
low = ["cheap-coder"] # routine code stays cheap
high = ["best-coder"] # hard code goes to the strongest coder
any = ["generic-coder"] # leads at every tier after the tier-specific pick
```
The tier-specific list leads its own tier; `any` (and the legacy plain-array
form) leads at every tier; the global tier chain and `default` always remain
as fallback, so a task rule never weakens failover. The admin console's
Routing page edits this as rows of “model + complexity” pairs per task.
**Exception — high complexity escalates.** When the router rates a request
`high` (and a `high` chain is configured), the tier chain leads and the task
rule demotes to fallback. Otherwise a rule like `code → cheap-model` would pin
even the hardest prompts to the cheap model and the tier system would never
get a say. Low/medium keep task-rule-first semantics. The available labels come from the router
(`GET /v1/taxonomies`) — the admin console's Routing page builds the editor
from your account's live taxonomy and shows each task's share of recent
traffic. When a rule supplied the head of the chain the response reports
`route_source: "router+task-rule"`.
+40
View File
@@ -160,3 +160,43 @@ cooldown = "30s"
enabled = false
# источник истины задаёт интегратор (например, заголовок X-Cortiq-Correct-Label)
```
## Распределение по задачам
Роутер классифицирует каждый запрос дважды: **тип задачи** (что это за работа)
и **уровень сложности** (насколько она трудна). Уровни выбирают, *сколько
модели* нужно; правила по задачам — *какая* модель лучше в этом деле:
```toml
[routing.tasks]
translation = ["local-qwen"] # переводы дёшевы — оставляем локально
code = ["claude-sonnet", "gpt"] # код — сильнейшему кодеру
```
Модели подходящего правила идут **первыми** в цепочке кандидатов; цепочка по
сложности следует за ними как резерв — правило по задаче не ослабляет failover.
Задачи без правила маршрутизируются только по сложности.
**Модели по сложности внутри задачи.** Правило задачи принимает и карту по
уровням — «какая модель обслуживает задачу на какой сложности»:
```toml
[routing.tasks.code]
low = ["cheap-coder"] # рутинный код — дёшево
high = ["best-coder"] # сложный код — сильнейшему кодеру
any = ["generic-coder"] # ведёт на всех уровнях после уровневого выбора
```
Список конкретного уровня ведёт на своём уровне; `any` (и старая форма-массив)
ведёт на всех; глобальная цепочка уровня и `default` всегда остаются резервом —
правило задачи не ослабляет failover. На вкладке Роутинг это редактируется
строками пар «модель + сложность» у каждой задачи.
**Исключение — высокая сложность эскалирует.** Когда роутер оценил запрос как
`high` (и цепочка `high` настроена), цепочка уровня идёт первой, а правило
задачи становится резервом. Иначе правило вида `code → дешёвая-модель`
приковало бы к дешёвой модели даже самые сложные запросы, и система уровней
не имела бы права голоса. Для low/medium правило задачи ведёт, как и раньше. Список меток приходит
с роутера (`GET /v1/taxonomies`) — вкладка Роутинг строит редактор по живой
таксономии аккаунта и показывает долю каждой задачи в трафике. Когда цепочку
возглавило правило задачи, ответ сообщает `route_source: "router+task-rule"`.
+453 -9
View File
@@ -116,13 +116,7 @@ fn tiers_to_map(routing: &RoutingCfg) -> HashMap<String, Vec<String>> {
routing
.tiers
.iter()
.map(|(k, v)| {
let list = match v {
TierTargets::List(l) => l.clone(),
TierTargets::One(s) => vec![s.clone()],
};
(k.clone(), list)
})
.map(|(k, v)| (k.clone(), v.models()))
.collect()
}
@@ -155,10 +149,13 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
)
.route("/admin/api/models/:id/probe", post(probe_model))
.route("/admin/api/routing", get(get_routing).put(put_routing))
.route("/admin/api/routing/labels", get(get_routing_labels))
.route(
"/admin/api/protocols",
get(get_protocols).put(put_protocols),
)
.route("/admin/api/harnesses", get(get_harnesses))
.route("/admin/api/balance", get(get_balance))
.route("/admin/api/settings", get(get_settings).put(put_settings))
.route("/admin/api/keys", get(list_keys).post(create_key))
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
@@ -184,6 +181,21 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
.route("/admin/api/requests", get(get_requests))
.route("/admin/api/test", post(run_test))
.route("/admin/api/test/stream", post(run_test_stream))
.route("/admin/api/logs", get(get_logs).delete(clear_logs))
.route("/admin/api/system", get(get_system))
.route("/admin/api/restart", post(restart_gateway))
.route("/admin/api/shutdown", post(shutdown_gateway))
.route("/admin/api/bench", get(bench_list).post(bench_start))
.route("/admin/api/config/export", get(config_export))
.route("/admin/api/config/import", post(config_import))
.route("/admin/api/media/models", get(media_models))
.route("/admin/api/media", get(media_jobs).post(media_generate))
.route(
"/admin/api/media/:job",
get(media_job_status).delete(media_delete),
)
.route("/admin/api/media/:job/cancel", post(media_cancel))
.route("/admin/api/media/:job/file/:kind", get(media_file))
.route("/admin/api/setup/status", get(setup_status))
.route("/admin/api/setup/complete", post(setup_complete))
.route("/admin/api/setup/featured", get(setup_featured))
@@ -583,9 +595,23 @@ async fn provider_models(
ok(resp)
}
async fn meta() -> ApiResult {
async fn meta(State(state): State<SharedState>) -> ApiResult {
// process start marker: open tabs poll this and offer a reload when the
// gateway (and thus the embedded SPA) has been restarted/updated
static STARTED: std::sync::OnceLock<u64> = std::sync::OnceLock::new();
let started = *STARTED.get_or_init(|| {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
});
ok(json!({
"version": env!("CARGO_PKG_VERSION"),
"started": started,
"update_available": state.gateway_update.lock().unwrap().clone(),
"admin_open_public": state
.admin_open_public
.load(std::sync::atomic::Ordering::Relaxed),
"providers": ["openai", "anthropic", "openrouter", "lmstudio", "ollama", "http"],
"provider_impl": { "openai": true, "anthropic": true, "openrouter": true, "lmstudio": true, "ollama": true, "http": true },
"cost_tiers": ["cheap", "mid", "expensive", "local"],
@@ -726,8 +752,15 @@ fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
if routing.default == id {
routing.default = String::new();
}
for tt in routing.tiers.values_mut() {
for tt in routing.tiers.values_mut().chain(routing.tasks.values_mut()) {
match tt {
crate::config::TierTargets::Map(m) => {
for v in m.values_mut() {
if let crate::config::TaskTierVal::List(l) = v {
l.retain(|x| x != id);
}
}
}
crate::config::TierTargets::List(v) => v.retain(|x| x != id),
crate::config::TierTargets::One(s) if s == id => {
*tt = crate::config::TierTargets::List(Vec::new());
@@ -735,6 +768,8 @@ fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
crate::config::TierTargets::One(_) => {}
}
}
// an emptied task rule means "no rule", not "no candidates"
routing.tasks.retain(|_, tt| !tt.models().is_empty());
}
/// Whether any managed server or static model in `cfg` still uses `path` as its
@@ -835,6 +870,7 @@ async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -
role: "user".into(),
content: "ping".into(),
tool_calls: vec![],
..Default::default()
}],
tools: vec![],
params: GenParams {
@@ -875,17 +911,79 @@ async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -
async fn get_routing(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let tasks: HashMap<String, serde_json::Value> = live
.cfg
.routing
.tasks
.iter()
.map(|(k, v)| {
(k.clone(), {
let (any, per) = v.task_rule();
let mut o = serde_json::Map::new();
if !any.is_empty() {
o.insert("any".into(), json!(any));
}
for (tier, models) in per {
if !models.is_empty() {
o.insert(tier, json!(models));
}
}
serde_json::Value::Object(o)
})
})
.collect();
ok(json!({
"tiers": tiers_to_map(&live.cfg.routing),
"bands": live.cfg.routing.bands,
"tasks": tasks,
"default": live.cfg.routing.default,
"policy": live.cfg.routing.policy,
}))
}
/// Task labels of the active taxonomy, straight from the router (`GET
/// /v1/taxonomies`) — the UI builds the per-task rule editor from this, so a
/// custom taxonomy shows its own labels without a gateway update.
async fn get_routing_labels(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
match live.router.taxonomies().await {
Ok(Some(v)) => {
let want = live.cfg.router.taxonomy_id.clone().unwrap_or_default();
let mut labels: Vec<String> = Vec::new();
if let Some(arr) = v["taxonomies"].as_array() {
for tx in arr {
let id = tx["taxonomy_id"].as_str().unwrap_or("");
if want.is_empty() || id == want || labels.is_empty() {
if let Some(ls) = tx["labels"].as_array() {
let this: Vec<String> = ls
.iter()
.filter_map(|l| l.as_str().map(String::from))
.collect();
if id == want || labels.is_empty() {
labels = this;
}
}
}
}
}
if labels.is_empty() {
ok(json!({ "labels": Value::Null, "source": "router-empty" }))
} else {
ok(json!({ "labels": labels, "source": "router" }))
}
}
_ => ok(json!({ "labels": Value::Null, "source": "unavailable" })),
}
}
#[derive(Deserialize)]
struct RoutingBody {
#[serde(default)]
tiers: HashMap<String, Vec<String>>,
#[serde(default)]
tasks: HashMap<String, TierTargets>,
#[serde(default)]
bands: Option<crate::config::TierBands>,
default: String,
#[serde(default)]
policy: RoutingPolicy,
@@ -899,6 +997,13 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
.into_iter()
.map(|(k, v)| (k, TierTargets::List(v)))
.collect(),
tasks: body
.tasks
.into_iter()
// empty rule (no models AND no forced tier) = "по сложности"
.filter(|(_, v)| !v.models().is_empty())
.collect(),
bands: body.bands,
default: body.default,
policy: body.policy,
};
@@ -906,6 +1011,129 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
ok(json!({ "ok": true }))
}
// ─────────────────────────────── balance ───────────────────────────────────
// OpenRouter credit balance for the dashboard. Cached for 5 minutes; absent
// (null) when no OpenRouter key is configured — the card hides itself.
async fn get_balance(State(state): State<SharedState>) -> ApiResult {
static CACHE: std::sync::OnceLock<std::sync::Mutex<Option<(std::time::Instant, f64)>>> =
std::sync::OnceLock::new();
let cache = CACHE.get_or_init(|| std::sync::Mutex::new(None));
if let Some((at, v)) = *cache.lock().unwrap() {
if at.elapsed() < std::time::Duration::from_secs(300) {
return ok(json!({ "openrouter_usd": v }));
}
}
let key = state
.secrets
.resolve("OPENROUTER_API_KEY")
.or_else(|| std::env::var("OPENROUTER_API_KEY").ok());
let Some(key) = key.filter(|k| !k.is_empty()) else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(8))
.build()
.unwrap_or_default();
let resp = client
.get("https://openrouter.ai/api/v1/credits")
.bearer_auth(&key)
.send()
.await;
let balance = match resp {
Ok(r) if r.status().is_success() => {
let v: serde_json::Value = r.json().await.unwrap_or_default();
let total = v["data"]["total_credits"].as_f64().unwrap_or(0.0);
let used = v["data"]["total_usage"].as_f64().unwrap_or(0.0);
Some(total - used)
}
_ => None,
};
let Some(balance) = balance else {
return ok(json!({ "openrouter_usd": serde_json::Value::Null }));
};
*cache.lock().unwrap() = Some((std::time::Instant::now(), balance));
ok(json!({ "openrouter_usd": balance }))
}
// ──────────────────────────── harness detection ────────────────────────────
// Which coding harnesses are present on the machine the gateway runs on, so the
// API-keys view can mark tabs "installed" / "already configured for cortiq".
// Detection is best-effort: binaries in PATH + common install dirs, config
// files under the user's home. Works on Linux/macOS/Windows.
fn harness_home() -> Option<std::path::PathBuf> {
std::env::var_os("HOME")
.or_else(|| std::env::var_os("USERPROFILE"))
.map(std::path::PathBuf::from)
}
fn find_bin(name: &str) -> bool {
let exts: &[&str] = if cfg!(windows) {
&[".exe", ".cmd", ".bat", ""]
} else {
&[""]
};
let mut dirs: Vec<std::path::PathBuf> = std::env::var_os("PATH")
.map(|p| std::env::split_paths(&p).collect())
.unwrap_or_default();
if let Some(home) = harness_home() {
// GUI-launched processes often miss the shell PATH — add usual suspects
for extra in [
".local/bin",
".cargo/bin",
"bin",
".bun/bin",
".npm-global/bin",
"AppData/Roaming/npm",
] {
dirs.push(home.join(extra));
}
}
dirs.extend(["/usr/local/bin", "/opt/homebrew/bin"].map(std::path::PathBuf::from));
dirs.iter()
.any(|d| exts.iter().any(|e| d.join(format!("{name}{e}")).is_file()))
}
/// Does the harness config mention cortiq (i.e. was it pointed at this gateway)?
fn config_mentions_cortiq(rel: &str) -> bool {
harness_home()
.map(|h| h.join(rel))
.and_then(|p| std::fs::read_to_string(p).ok())
.map(|c| c.to_lowercase().contains("cortiq"))
.unwrap_or(false)
}
async fn get_harnesses() -> ApiResult {
let harnesses = tokio::task::spawn_blocking(|| {
let vscode_ext_glob = |marker: &str| -> bool {
let Some(home) = harness_home() else {
return false;
};
for extdir in [".vscode/extensions", ".vscode-server/extensions"] {
if let Ok(rd) = std::fs::read_dir(home.join(extdir)) {
for e in rd.flatten() {
if e.file_name().to_string_lossy().starts_with(marker) {
return true;
}
}
}
}
false
};
json!({
"Claude Code": { "installed": find_bin("claude"), "configured": serde_json::Value::Null },
"Codex": { "installed": find_bin("codex"), "configured": config_mentions_cortiq(".codex/config.toml") },
"OpenCode": { "installed": find_bin("opencode"), "configured": config_mentions_cortiq(".config/opencode/opencode.json") },
"Continue": { "installed": harness_home().map(|h| h.join(".continue").is_dir()).unwrap_or(false), "configured": config_mentions_cortiq(".continue/config.yaml") },
"Cline": { "installed": vscode_ext_glob("saoudrizwan.claude-dev"), "configured": serde_json::Value::Null },
})
})
.await
.unwrap_or_else(|_| json!({}));
ok(harnesses)
}
// ───────────────────────────── protocols ───────────────────────────────────
async fn get_protocols(State(state): State<SharedState>) -> ApiResult {
@@ -1177,6 +1405,7 @@ async fn list_keys(State(state): State<SharedState>) -> ApiResult {
"key_masked": mask_key(&k.key),
"account": k.account,
"rate_per_min": k.rate_per_min,
"monthly_budget_usd": k.monthly_budget_usd,
"allow_models": k.allow_models,
})
})
@@ -1192,6 +1421,8 @@ struct NewKey {
#[serde(default)]
rate_per_min: u32,
#[serde(default)]
monthly_budget_usd: Option<f64>,
#[serde(default)]
allow_models: Vec<String>,
}
@@ -1210,6 +1441,7 @@ async fn create_key(State(state): State<SharedState>, Json(b): Json<NewKey>) ->
account: b.account,
rate_per_min: b.rate_per_min,
allow_models: b.allow_models,
monthly_budget_usd: b.monthly_budget_usd,
});
state.reload(cfg)?;
// show the full key only once
@@ -1420,8 +1652,16 @@ async fn setup_complete(
.map(|j| j.output)
.collect();
if !finished.is_empty() {
let cortiq_bin = state.live().cfg.cmf.cortiq_bin.clone();
let mut cfg = current_cfg(&state);
for output in &finished {
// media packs (image/video) are used by the media API directly,
// not served — connecting them as chat servers would only fail
let modality =
crate::media::classify_file(std::path::Path::new(output), &cortiq_bin).await;
if modality != crate::media::Modality::Chat {
continue;
}
if let Some(id) = register_cmf_output(&mut cfg, output) {
registered.push(id);
}
@@ -1959,6 +2199,7 @@ fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<
o1_window: None,
o1_sink: None,
skip_mtp: false,
caps: Vec::new(),
});
Some(id)
}
@@ -1972,6 +2213,15 @@ async fn register_import(State(state): State<SharedState>, Path(job): Path<Strin
if j.state != "done" {
return Err(ApiError::bad("conversion not finished"));
}
let live = state.live();
// Image/video packs are not served by `cortiq serve` — they are used
// directly by the media API. Nothing to register; tell the UI what it is.
let modality =
crate::media::classify_file(std::path::Path::new(&j.output), &live.cfg.cmf.cortiq_bin)
.await;
if modality != crate::media::Modality::Chat {
return ok(json!({ "ok": true, "media_kind": modality, "model_id": null }));
}
let mut cfg = current_cfg(&state);
let id = register_cmf_output(&mut cfg, &j.output).ok_or_else(|| {
ApiError::bad(format!(
@@ -2004,6 +2254,200 @@ async fn get_requests(State(state): State<SharedState>, Query(q): Query<RecentQu
ok(json!({ "requests": items }))
}
// ───────────────────────── system / logs / lifecycle ─────────────────────────
/// Recent gateway log lines (in-memory ring) — for users who never see the
/// terminal: Docker, autostart, a double-clicked exe.
async fn get_logs(Query(q): Query<RecentQuery>) -> ApiResult {
let limit = q.limit.unwrap_or(300).min(500);
ok(json!({ "lines": crate::logbuf::snapshot(limit) }))
}
async fn clear_logs() -> ApiResult {
crate::logbuf::clear();
ok(json!({ "ok": true }))
}
/// Host metrics snapshot: CPU, RAM, free disk at the models dir, gateway RSS.
async fn get_system(State(state): State<SharedState>) -> ApiResult {
let snap = state.sysmon.snapshot();
ok(serde_json::to_value(snap).unwrap_or_else(|_| json!({})))
}
/// Stop every managed local model server and power the gateway off. The 200
/// goes out first so the SPA can show its "gateway is off" screen; nothing is
/// respawned — the operator starts the gateway again by hand (or the Docker
/// restart policy does, in which case this behaves like a restart).
async fn shutdown_gateway(State(state): State<SharedState>) -> ApiResult {
state.cmf.stop().await;
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
tracing::info!("shutdown requested from the admin console");
std::process::exit(0);
});
ok(json!({ "ok": true }))
}
/// Restart the gateway in place: stop managed model servers, spawn a fresh
/// copy of ourselves (it retries the bind while our port frees), then exit.
/// In Docker we only exit — we are PID 1, a spawned child would die with the
/// container anyway; the container's restart policy brings the gateway back.
async fn restart_gateway(State(state): State<SharedState>) -> ApiResult {
state.cmf.stop().await;
let in_docker = std::path::Path::new("/.dockerenv").exists();
let exe = std::env::current_exe().map_err(|e| ApiError::bad(e.to_string()))?;
let args: Vec<String> = std::env::args().skip(1).collect();
tokio::spawn(async move {
// let the 200 reach the browser first
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
if !in_docker {
let _ = std::process::Command::new(exe)
.args(args)
.env("CORTIQ_BIND_RETRY", "1")
.spawn();
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
}
std::process::exit(0);
});
ok(json!({ "ok": true }))
}
#[derive(Deserialize)]
struct BenchBody {
model: String,
}
/// Benchmark a local .cmf with `cortiq bench --json` — real tok/s on this box.
async fn bench_start(State(state): State<SharedState>, Json(b): Json<BenchBody>) -> ApiResult {
let live = state.live();
crate::bench::start(state.bench.clone(), &live.cfg.cmf, b.model).map_err(ApiError::bad)?;
ok(json!({ "ok": true }))
}
async fn bench_list(State(state): State<SharedState>) -> ApiResult {
ok(json!({ "runs": state.bench.list() }))
}
/// The config file as-is (comments included) for download. Secrets live in
/// secrets.toml and are deliberately NOT part of the export.
async fn config_export(State(state): State<SharedState>) -> Response {
match tokio::fs::read_to_string(&state.config_path).await {
Ok(text) => (
[
(header::CONTENT_TYPE, "application/toml".to_string()),
(
header::CONTENT_DISPOSITION,
"attachment; filename=\"gateway.toml\"".to_string(),
),
],
text,
)
.into_response(),
Err(e) => ApiError::bad(format!("cannot read config: {e}")).into_response(),
}
}
/// Replace the whole config from an uploaded TOML. Validated and hot-reloaded
/// exactly like a panel edit; on any error the current config stays untouched.
async fn config_import(State(state): State<SharedState>, body: String) -> ApiResult {
let cfg: crate::config::Config =
toml::from_str(&body).map_err(|e| ApiError::bad(format!("invalid TOML: {e}")))?;
state.reload(cfg)?;
ok(json!({ "ok": true }))
}
// ───────────────────────────── media generation ─────────────────────────────
/// Local `.cmf` files with their modality (chat / image / video) — the
/// playground's media tabs read this to know what can generate what.
async fn media_models(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let models =
crate::media::list_media_models(&live.cfg.cmf.models_dir, &live.cfg.cmf.cortiq_bin).await;
ok(json!({ "models": models }))
}
async fn media_jobs(State(state): State<SharedState>) -> ApiResult {
ok(json!({ "jobs": state.media.list() }))
}
async fn media_generate(
State(state): State<SharedState>,
Json(p): Json<crate::media::MediaParams>,
) -> ApiResult {
let live = state.live();
let id = crate::media::start_generation(state.media.clone(), &live.cfg.cmf, p)
.map_err(ApiError::bad)?;
ok(json!({ "job": id }))
}
async fn media_job_status(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
match state.media.get(&job) {
Some(j) => {
let mut v = serde_json::to_value(&j).unwrap_or_else(|_| json!({}));
if j.state == "queued" {
v["queue_position"] = json!(state.media.queue_position(&job));
}
ok(v)
}
None => Err(ApiError::not_found(format!("media job '{job}' not found"))),
}
}
async fn media_cancel(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
if state.media.get(&job).is_none() {
return Err(ApiError::not_found(format!("media job '{job}' not found")));
}
ok(json!({ "ok": state.media.cancel(&job, &crate::media::media_dir()) }))
}
async fn media_delete(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
if state.media.get(&job).is_none() {
return Err(ApiError::not_found(format!("media job '{job}' not found")));
}
let removed = state
.media
.delete(&job, &crate::media::media_dir())
.map_err(ApiError::bad)?;
ok(json!({ "ok": removed }))
}
/// Serve a finished job's output bytes (ppm / avi / wav / mp4). The job id is
/// server-generated hex and the file name is fixed per kind, so no path from the
/// client ever reaches the filesystem.
async fn media_file(
State(state): State<SharedState>,
Path((job, kind)): Path<(String, String)>,
) -> Response {
let Some(j) = state.media.get(&job) else {
return ApiError::not_found(format!("media job '{job}' not found")).into_response();
};
if j.state != "done" {
return ApiError::bad("generation not finished").into_response();
}
let Some((file, mime)) = crate::media::output_file(&kind) else {
return ApiError::bad(format!("unknown output kind '{kind}'")).into_response();
};
let path = crate::media::media_dir().join(&job).join(file);
match tokio::fs::read(&path).await {
Ok(bytes) => (
[
(header::CONTENT_TYPE, mime.to_string()),
(
header::CONTENT_DISPOSITION,
format!(
"inline; filename=\"cortiq-{job}.{}\"",
file.rsplit('.').next().unwrap_or("bin")
),
),
],
bytes,
)
.into_response(),
Err(_) => ApiError::not_found("output file missing on disk").into_response(),
}
}
// ───────────────────────────── playground (test) ───────────────────────────
#[derive(Deserialize)]
+225
View File
@@ -0,0 +1,225 @@
//! One-click local model benchmark: run `cortiq bench <model> --json` and keep
//! the parsed result per model, so the Models page can show real tok/s numbers
//! measured on this machine.
use serde::Serialize;
use std::collections::HashMap;
use std::process::Stdio;
use std::sync::{Arc, Mutex};
#[derive(Clone, Serialize)]
pub struct BenchRun {
pub model: String, // file stem under models_dir
pub state: String, // running | done | error
pub log: Vec<String>,
pub tok_s: Option<f64>,
/// The full JSON object the benchmark printed, passed through untouched.
pub raw: Option<serde_json::Value>,
pub started: u64,
pub finished: Option<u64>,
}
#[derive(Default)]
pub struct BenchStore {
runs: Mutex<HashMap<String, BenchRun>>,
busy: Mutex<bool>,
}
fn now() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// tok/s out of the benchmark's JSON: known field names first (steady decode
/// speed is the honest number), then any `*tok_s*` field as a fallback. The
/// old loose "contains tok + per" heuristic could grab `allocs_per_token`.
fn extract_tok_s(v: &serde_json::Value) -> Option<f64> {
let obj = v.as_object()?;
for want in [
"decode_tok_s_steady",
"decode_tok_s_incl_prefill",
"decode_tok_s",
"tok_s",
"tok_per_s",
"tokens_per_sec",
] {
if let Some(n) = obj.get(want).and_then(|x| x.as_f64()) {
return Some(n);
}
}
obj.iter()
.find(|(k, val)| {
let k = k.to_ascii_lowercase();
(k.contains("tok_s") || k.contains("tokens_per")) && val.as_f64().is_some()
})
.and_then(|(_, val)| val.as_f64())
}
/// Last parseable JSON object in mixed CLI output. Newer cortiq releases
/// pretty-print the `--json` result over many lines, older ones emit a single
/// line — this handles both, skipping any log noise around it.
fn last_json_object(text: &str) -> Option<serde_json::Value> {
let mut end = text.len();
while let Some(close) = text[..end].rfind('}') {
let mut depth = 0i32;
let mut start = None;
for (i, c) in text[..=close].char_indices().rev() {
match c {
'}' => depth += 1,
'{' => {
depth -= 1;
if depth == 0 {
start = Some(i);
break;
}
}
_ => {}
}
}
if let Some(st) = start {
if let Ok(v) = serde_json::from_str::<serde_json::Value>(&text[st..=close]) {
if v.is_object() {
return Some(v);
}
}
}
end = close;
}
None
}
impl BenchStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
pub fn get(&self, model: &str) -> Option<BenchRun> {
self.runs.lock().unwrap().get(model).cloned()
}
pub fn list(&self) -> Vec<BenchRun> {
self.runs.lock().unwrap().values().cloned().collect()
}
}
/// Kick off a benchmark for `<models_dir>/<model>.cmf`. One at a time — a
/// benchmark loads the whole model and owns the machine while it runs.
pub fn start(
store: Arc<BenchStore>,
cmf: &crate::config::CmfCfg,
model: String,
) -> Result<(), String> {
let path = std::path::Path::new(&cmf.models_dir).join(format!("{model}.cmf"));
if !path.exists() {
return Err(format!("model file not found: {}", path.display()));
}
{
let mut busy = store.busy.lock().unwrap();
if *busy {
return Err("a benchmark is already running — wait for it to finish".into());
}
*busy = true;
}
store.runs.lock().unwrap().insert(
model.clone(),
BenchRun {
model: model.clone(),
state: "running".into(),
log: Vec::new(),
tok_s: None,
raw: None,
started: now(),
finished: None,
},
);
let bin = cmf.cortiq_bin.clone();
let gpu = cmf.gpu;
tokio::spawn(async move {
let out = tokio::process::Command::new(&bin)
.arg("bench")
.arg(&path)
.arg("--json")
.env("CMF_GPU", if gpu { "1" } else { "0" })
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.await;
let mut g = store.runs.lock().unwrap();
let run = g.get_mut(&model).unwrap();
run.finished = Some(now());
match out {
Ok(o) => {
let text = format!(
"{}\n{}",
String::from_utf8_lossy(&o.stdout),
String::from_utf8_lossy(&o.stderr)
);
// keep a readable tail; the JSON line is parsed separately
run.log = text
.lines()
.map(|l| l.trim())
.filter(|l| !l.is_empty() && !l.contains("Metal GPU path"))
.map(String::from)
.collect::<Vec<_>>();
let n = run.log.len();
if n > 30 {
run.log.drain(0..n - 30);
}
let parsed = last_json_object(&text);
match parsed {
Some(v) if o.status.success() => {
run.tok_s = extract_tok_s(&v);
run.raw = Some(v);
run.state = "done".into();
}
_ => run.state = "error".into(),
}
}
Err(e) => {
run.log.push(format!("✗ failed to start cortiq bench: {e}"));
run.state = "error".into();
}
}
drop(g);
*store.busy.lock().unwrap() = false;
});
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tok_s_from_common_keys() {
let v = serde_json::json!({"tok_s": 42.5, "allocs_per_token": 0});
assert_eq!(extract_tok_s(&v), Some(42.5));
let v = serde_json::json!({"decode_tokens_per_sec": 31.0});
assert_eq!(extract_tok_s(&v), Some(31.0));
}
#[test]
fn steady_decode_wins_and_allocs_never_matches() {
let v = serde_json::json!({
"allocs_per_token": 769.4,
"decode_tok_s_incl_prefill": 9.98,
"decode_tok_s_steady": 11.86,
"prefill_tok_s": 25.5,
});
assert_eq!(extract_tok_s(&v), Some(11.86));
let v = serde_json::json!({"allocs_per_token": 769.4, "seq_len": 140});
assert_eq!(extract_tok_s(&v), None);
}
#[test]
fn json_found_single_line_and_pretty() {
let single = "noise\n{\"tok_s\": 5.0}\nmore noise";
assert_eq!(last_json_object(single).unwrap()["tok_s"], 5.0);
let pretty = "INFO loading\n{\n \"decode_tok_s_steady\": 11.86,\n \"seq_len\": 140\n}\nINFO done {not json";
assert_eq!(
last_json_object(pretty).unwrap()["decode_tok_s_steady"],
11.86
);
assert!(last_json_object("no braces here").is_none());
}
}
+1 -1
View File
@@ -55,7 +55,7 @@ impl SemanticCache {
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
Arc::new(Self {
enabled: cfg.enabled,
threshold: cfg.threshold,
threshold: cfg.threshold as f32,
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
max_entries: cfg.max_entries.max(1),
embed_model: cfg.embed_model.clone(),
+40 -8
View File
@@ -56,6 +56,9 @@ struct Managed {
pub struct CmfRuntime {
status: Mutex<CmfStatus>,
children: Mutex<Vec<Managed>>,
/// Server ids whose healthz handshake advertised tool support — lets the
/// pipeline route tools traffic to local models with no manual caps.
pub detected_tools: Mutex<std::collections::HashSet<String>>,
}
impl CmfRuntime {
@@ -63,6 +66,7 @@ impl CmfRuntime {
Arc::new(Self {
status: Mutex::new(CmfStatus::default()),
children: Mutex::new(Vec::new()),
detected_tools: Mutex::new(std::collections::HashSet::new()),
})
}
@@ -193,18 +197,19 @@ impl Default for CmfRuntime {
Self {
status: Mutex::new(CmfStatus::default()),
children: Mutex::new(Vec::new()),
detected_tools: Mutex::new(std::collections::HashSet::new()),
}
}
}
/// crates.io `newest_version` for `cortiq-cli`.
async fn latest_crates_version() -> Option<String> {
/// crates.io `newest_version` for any crate.
async fn latest_version_of(krate: &str) -> Option<String> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.build()
.ok()?;
let body = client
.get("https://crates.io/api/v1/crates/cortiq-cli")
.get(format!("https://crates.io/api/v1/crates/{krate}"))
.header("User-Agent", "cortiq-gateway")
.send()
.await
@@ -216,6 +221,16 @@ async fn latest_crates_version() -> Option<String> {
v["crate"]["newest_version"].as_str().map(|s| s.to_string())
}
/// crates.io `newest_version` for `cortiq-cli`.
async fn latest_crates_version() -> Option<String> {
latest_version_of("cortiq-cli").await
}
/// Newest published gateway version — the "update available" footer check.
pub(crate) async fn latest_gateway_version() -> Option<String> {
latest_version_of("cortiq-gateway").await
}
/// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2").
pub(crate) fn installed_version(bin: &str) -> Option<String> {
let out = std::process::Command::new(bin)
@@ -322,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
.map_err(|e| e.to_string())
}
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
/// Poll `/healthz` until the local server answers 200 (or the deadline
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
/// caller can register them; older servers return `None` there.
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
let url = format!("http://{host}:{port}/healthz");
let client = reqwest::Client::new();
for _ in 0..tries {
@@ -334,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
.await
{
if r.status().is_success() {
return true;
let tools = r
.json::<serde_json::Value>()
.await
.ok()
.and_then(|v| v["capabilities"]["tools"].as_bool());
return (true, tools);
}
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
false
(false, None)
}
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
@@ -422,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
if !rt.is_running(&server.id) {
continue; // failed to spawn / file missing
}
if wait_healthy(&cfg.local_host, server.port, 240).await {
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
if ok {
rt.set_healthy(&server.id, true);
// capability handshake: the server said whether its model's chat
// template can drive tool calling — no manual caps needed
if tools == Some(true) {
rt.detected_tools.lock().unwrap().insert(server.id.clone());
tracing::info!(id = %server.id, "local server advertises tool support");
} else {
rt.detected_tools.lock().unwrap().remove(&server.id);
}
rt.log(format!(
"local CMF server ready — '{}' at http://{}:{}/v1",
server.id, cfg.local_host, server.port
+211 -13
View File
@@ -130,6 +130,10 @@ pub struct CmfServer {
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
#[serde(default)]
pub skip_mtp: bool,
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
/// gateway auto-detects tool support from the server's healthz handshake.
#[serde(default)]
pub caps: Vec<String>,
}
/// CMF-format model factory: import a HuggingFace model, convert it to a
@@ -229,6 +233,7 @@ impl CmfCfg {
o1_window: None,
o1_sink: None,
skip_mtp: false,
caps: Vec::new(),
}];
}
Vec::new()
@@ -305,7 +310,9 @@ fn default_router_enabled() -> bool {
true
}
fn default_router_timeout() -> u64 {
800
// the hosted router escalates hard prompts to an oracle (~10 s);
// an 800 ms budget silently degraded every one of those to the fallback
15_000
}
fn default_verify_tls() -> bool {
true
@@ -396,17 +403,149 @@ pub struct RoutingCfg {
pub default: String,
#[serde(default)]
pub policy: RoutingPolicy,
/// Per-task rules: `task_label → ordered model list`. A matching rule's
/// models go FIRST; the complexity-tier chain follows as fallback, so a
/// task pin never weakens failover. Named field, so a tier cannot be
/// called "tasks" — serde extracts it before the flattened tier map.
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
pub tasks: HashMap<String, TierTargets>,
/// Local complexity bands: re-map the router's continuous score into
/// low/medium/high with YOUR OWN boundaries. The hosted router's bands
/// are global for all customers; these are per-gateway. `None` = keep
/// the router's tier verdict as delivered.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bands: Option<TierBands>,
}
/// Score boundaries: `score ≤ low` → low, `≤ medium` → medium, else high.
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
pub struct TierBands {
pub low: f32,
pub medium: f32,
}
impl TierBands {
pub fn tier_of(&self, score: f32) -> &'static str {
if score <= self.low {
"low"
} else if score <= self.medium {
"medium"
} else {
"high"
}
}
}
/// A routing tier is either a list of model_ids or the string `default`.
/// (`default` is handled by a separate field; only tier lists appear here.)
///
/// TASK rules additionally accept a per-complexity map — "which model serves
/// this task at each tier":
///
/// ```toml
/// [routing.tasks.code]
/// low = ["cheap-coder"]
/// high = ["big-model"]
/// any = ["fallback"] # leads at every tier after the tier-specific list
/// ```
///
/// The plain-array form stays valid and means "leads at any tier". The
/// transitional `{ models = [...], tier = "high" }` form is normalized into
/// the map on load. Untagged: map is tried first, arrays keep parsing as is.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum TierTargets {
Map(HashMap<String, TaskTierVal>),
List(Vec<String>),
One(String),
}
/// A value inside a task's per-tier map: a model list, a single model, or the
/// transitional string value of the legacy `tier = "high"` key.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum TaskTierVal {
List(Vec<String>),
Str(String),
}
impl TaskTierVal {
fn list(&self) -> Vec<String> {
match self {
TaskTierVal::List(v) => v.clone(),
TaskTierVal::Str(s) => vec![s.clone()],
}
}
}
pub const TASK_TIER_KEYS: [&str; 4] = ["any", "low", "medium", "high"];
impl TierTargets {
/// Every referenced model id, whatever the syntactic form (validation).
pub fn models(&self) -> Vec<String> {
match self {
TierTargets::Map(m) => {
let mut out = Vec::new();
for (k, v) in m {
if k == "tier" {
continue; // legacy marker, not a model list
}
out.extend(v.list());
}
out
}
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
}
}
/// Normalized task rule: `(any_tier_models, per_tier_models)`.
/// Legacy `{models, tier}` becomes `per[tier] = models`; a plain array
/// becomes `any`.
pub fn task_rule(&self) -> (Vec<String>, HashMap<String, Vec<String>>) {
match self {
TierTargets::Map(m) => {
let mut any: Vec<String> = Vec::new();
let mut per: HashMap<String, Vec<String>> = HashMap::new();
let legacy_tier = m.get("tier").and_then(|v| match v {
TaskTierVal::Str(s) => Some(s.clone()),
_ => None,
});
for (k, v) in m {
match k.as_str() {
"tier" => {}
"any" | "models" => any.extend(v.list()),
"low" | "medium" | "high" => {
per.entry(k.clone()).or_default().extend(v.list())
}
_ => {}
}
}
if let Some(tier) = legacy_tier {
// legacy pin: the models were meant for that tier
let moved = std::mem::take(&mut any);
per.entry(tier).or_default().extend(moved);
}
(any, per)
}
TierTargets::List(v) => (v.clone(), HashMap::new()),
TierTargets::One(s) => (vec![s.clone()], HashMap::new()),
}
}
/// Unknown keys inside a task map (validation).
pub fn invalid_task_keys(&self) -> Vec<String> {
match self {
TierTargets::Map(m) => m
.keys()
.filter(|k| !TASK_TIER_KEYS.contains(&k.as_str()) && *k != "models" && *k != "tier")
.cloned()
.collect(),
_ => Vec::new(),
}
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RoutingPolicy {
#[serde(default = "default_mode")]
@@ -486,6 +625,9 @@ pub struct ApiKeyCfg {
pub account: String,
#[serde(default)]
pub rate_per_min: u32,
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub monthly_budget_usd: Option<f64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub allow_models: Vec<String>,
}
@@ -579,6 +721,12 @@ pub struct StatsCfg {
/// Retention window for aggregates/time-series replayed on startup, e.g. `7d`.
#[serde(default = "default_stats_retention")]
pub retention: String,
/// Compact the JSONL file once it exceeds this size: records older than
/// `retention` fold into cumulative per-account totals (a small sidecar
/// file) and are removed, so per-key accounting survives while the disk
/// stays bounded. 0 disables compaction.
#[serde(default = "default_stats_max_file_mb")]
pub max_file_mb: u64,
}
impl Default for StatsCfg {
fn default() -> Self {
@@ -587,6 +735,7 @@ impl Default for StatsCfg {
file: default_stats_file(),
ring_size: default_stats_ring(),
retention: default_stats_retention(),
max_file_mb: default_stats_max_file_mb(),
}
}
}
@@ -602,6 +751,9 @@ fn default_stats_ring() -> usize {
fn default_stats_retention() -> String {
"7d".into()
}
fn default_stats_max_file_mb() -> u64 {
16
}
/// Self-warming shadow loop (docs/SELF_WARMING_GATEWAY.md). While a task
/// label is not yet promoted, the client is served the CLOUD answer; in
@@ -685,9 +837,10 @@ fn default_shadow_soak() -> usize {
pub struct CacheCfg {
#[serde(default)]
pub enabled: bool,
/// Cosine-similarity threshold for a hit (0..1).
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
/// the number you typed (an f32 round-trips as 0.9200000166893005).
#[serde(default = "default_semcache_threshold")]
pub threshold: f32,
pub threshold: f64,
/// Entry time-to-live, e.g. `1h`.
#[serde(default = "default_semcache_ttl")]
pub ttl: String,
@@ -709,7 +862,7 @@ impl Default for CacheCfg {
}
}
}
fn default_semcache_threshold() -> f32 {
fn default_semcache_threshold() -> f64 {
0.92
}
fn default_semcache_ttl() -> String {
@@ -724,13 +877,16 @@ impl Default for Config {
fn default() -> Self {
Self {
listen: "0.0.0.0:9000".to_string(),
// The hosted allaigate router — the thing this gateway is built
// around. Without a key the pipeline degrades gracefully to local
// models and the console asks for one; nothing breaks.
router: RouterCfg {
enabled: true,
url: "http://localhost:8080".to_string(),
timeout_ms: default_router_timeout(),
verify_tls: true,
url: "https://138.226.222.209".to_string(),
timeout_ms: 15_000, // hard prompts escalate to an oracle (~10 s)
verify_tls: false, // the hosted endpoint is IP-bound
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
taxonomy_id: Some("general.task-type".to_string()),
taxonomy_id: Some("data-assistant".to_string()),
},
route: RouteCfg::default(),
models: Vec::new(),
@@ -738,6 +894,8 @@ impl Default for Config {
tiers: HashMap::new(),
default: String::new(),
policy: RoutingPolicy::default(),
tasks: HashMap::new(),
bands: None,
},
breaker: BreakerCfg::default(),
protocols: ProtocolsCfg {
@@ -745,7 +903,9 @@ impl Default for Config {
openai_completions: true,
openai_embeddings: true,
openai_models: true,
anthropic_messages: false,
// on by default: the API-keys view ships a Claude Code
// auto-setup snippet that targets /v1/messages
anthropic_messages: true,
mcp: false,
native_passthrough: true,
},
@@ -764,14 +924,23 @@ impl Default for Config {
}
/// Where the config lives when `--config` is not given.
/// Repo checkout (`config/gateway.toml` exists or `config/` dir exists) → keep it there;
/// installed binary (no repo) → `$data_dir/gateway.toml`.
/// A readable `./config/gateway.toml` (a real gateway checkout / install dir)
/// wins; otherwise the per-user home (`$CORTIQ_GATEWAY_HOME` or
/// `~/.cortiq-gateway`) — so `cortiq-gateway` typed in any directory picks up
/// the same configuration it was set up with. A foreign project that merely
/// has `Cargo.toml` and a `config/` folder must NOT capture the config
/// (that used to shadow the user's real config with a broken path).
pub fn default_config_path() -> String {
let repo_path = "config/gateway.toml";
if std::path::Path::new(repo_path).exists() {
// the file itself, readable — not just an entry that exists()
if std::fs::read_to_string(repo_path).is_ok() {
return repo_path.to_string();
}
if std::path::Path::new("Cargo.toml").exists() && std::path::Path::new("config").exists() {
// our own repo checkout before first run: config/ exists, file not yet
let ours = std::fs::read_to_string("Cargo.toml")
.map(|t| t.contains("name = \"cortiq-gateway\""))
.unwrap_or(false);
if ours && std::path::Path::new("config").is_dir() {
return repo_path.to_string();
}
data_dir()
@@ -849,6 +1018,7 @@ impl Config {
}
for (tier, targets) in &self.routing.tiers {
let list = match targets {
TierTargets::Map(_) => targets.models(),
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
@@ -862,6 +1032,33 @@ impl Config {
}
}
}
if let Some(b) = &self.routing.bands {
if !(b.low > 0.0 && b.low < b.medium && b.medium < 1.0) {
anyhow::bail!(
"config: routing.bands must satisfy 0 < low < medium < 1 (got {} / {})",
b.low,
b.medium
);
}
}
for (task, targets) in &self.routing.tasks {
for id in &targets.models() {
if !ids.contains(id.as_str()) {
anyhow::bail!(
"config: routing.tasks.{} references unknown model id '{}'",
task,
id
);
}
}
if let Some(bad) = targets.invalid_task_keys().first() {
anyhow::bail!(
"config: routing.tasks.{}.{} is not a tier (use any/low/medium/high)",
task,
bad
);
}
}
Ok(())
}
@@ -952,6 +1149,7 @@ min_class = { low = "cheap", medium = "mid" }
let low = back.routing.tiers.get("low").expect("low tier present");
match low {
TierTargets::Map(_) => assert_eq!(low.models(), vec!["local-qwen".to_string()]),
TierTargets::List(v) => assert_eq!(v, &vec!["local-qwen".to_string()]),
TierTargets::One(s) => assert_eq!(s, "local-qwen"),
}
+7
View File
@@ -13,6 +13,8 @@ pub enum GatewayError {
Unauthorized(String),
#[error("rate limit exceeded")]
RateLimited { retry_after_secs: u64 },
#[error("monthly budget exhausted (limit ${budget_usd})")]
BudgetExhausted { budget_usd: f64 },
#[error("upstream unavailable: {0}")]
UpstreamUnavailable(String),
#[error("internal error: {0}")]
@@ -37,6 +39,11 @@ impl GatewayError {
"rate_limit_error",
"rate_limited",
),
Self::BudgetExhausted { .. } => (
StatusCode::TOO_MANY_REQUESTS,
"insufficient_quota",
"insufficient_quota",
),
Self::UpstreamUnavailable(_) => (
StatusCode::BAD_GATEWAY,
"upstream_unavailable",
+51 -1
View File
@@ -132,7 +132,7 @@ fn default_quant() -> String {
}
/// Live state of one conversion job.
#[derive(Clone, Debug, Serialize)]
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Job {
pub id: String,
pub repo: String,
@@ -162,14 +162,60 @@ pub struct JobStore {
/// Cancel handles, keyed by job id (not serialized). Notifying aborts the
/// converter process and cleans up its partial output.
cancels: Mutex<HashMap<String, Arc<Notify>>>,
/// When set, every state change lands in this JSON file, so the job list —
/// and its Register buttons — survives a gateway restart.
persist: Mutex<Option<std::path::PathBuf>>,
last_save: Mutex<Option<std::time::Instant>>,
}
impl JobStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
/// Load previously saved jobs and start persisting to `path`. A job that
/// was `running` when the gateway died is settled as an error — its process
/// is gone; the file it produced (if complete) is still on disk.
pub fn attach_persistence(&self, path: std::path::PathBuf) {
if let Ok(text) = std::fs::read_to_string(&path) {
if let Ok(mut jobs) = serde_json::from_str::<Vec<Job>>(&text) {
let mut g = self.jobs.lock().unwrap();
for j in jobs.drain(..) {
let mut j = j;
if j.state == "running" {
j.state = "error".into();
j.log.push("✗ interrupted by a gateway restart".into());
j.finished = Some(now());
}
g.insert(j.id.clone(), j);
}
}
}
*self.persist.lock().unwrap() = Some(path);
}
fn save(&self, force: bool) {
let Some(path) = self.persist.lock().unwrap().clone() else {
return;
};
if !force {
let mut last = self.last_save.lock().unwrap();
if let Some(t) = *last {
if t.elapsed() < std::time::Duration::from_secs(2) {
return;
}
}
*last = Some(std::time::Instant::now());
}
let jobs: Vec<Job> = self.jobs.lock().unwrap().values().cloned().collect();
if let Ok(json) = serde_json::to_vec(&jobs) {
let tmp = path.with_extension("json.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &path);
}
}
}
fn insert(&self, job: Job) {
self.jobs.lock().unwrap().insert(job.id.clone(), job);
self.save(true);
}
fn set_cancel(&self, id: &str, n: Arc<Notify>) {
self.cancels.lock().unwrap().insert(id.to_string(), n);
@@ -215,6 +261,7 @@ impl JobStore {
j.phase = Some(phase);
}
}
self.save(false);
}
/// Expected total download size, once the HF tree is known.
fn set_total(&self, id: &str, total: u64) {
@@ -263,6 +310,7 @@ impl JobStore {
j.state = state.into();
j.finished = Some(now());
}
self.save(true);
}
fn finish(&self, id: &str, ok: bool) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
@@ -280,6 +328,7 @@ impl JobStore {
j.total_bytes = Some(j.total_bytes.unwrap_or(on_disk).max(on_disk));
}
}
self.save(true);
}
pub fn get(&self, id: &str) -> Option<Job> {
self.jobs.lock().unwrap().get(id).cloned()
@@ -295,6 +344,7 @@ impl JobStore {
cleanup_output(&job.output);
self.cancels.lock().unwrap().remove(id);
self.jobs.lock().unwrap().remove(id);
self.save(true);
Ok(true)
}
pub fn list(&self) -> Vec<Job> {
+78
View File
@@ -0,0 +1,78 @@
//! In-memory ring of recent log lines, so the admin console can show logs to
//! users who never see the terminal (Docker, autostart, double-clicked exe).
//! Fed by a second `tracing` layer writing plain text (no ANSI) into the ring
//! while the normal stdout layer stays colored.
use std::collections::VecDeque;
use std::io;
use std::sync::{Arc, Mutex, OnceLock};
const CAP: usize = 500;
fn ring() -> &'static Arc<Mutex<VecDeque<String>>> {
static RING: OnceLock<Arc<Mutex<VecDeque<String>>>> = OnceLock::new();
RING.get_or_init(|| Arc::new(Mutex::new(VecDeque::with_capacity(CAP))))
}
/// Last `limit` lines, oldest first.
pub fn snapshot(limit: usize) -> Vec<String> {
let g = ring().lock().unwrap();
g.iter()
.skip(g.len().saturating_sub(limit))
.cloned()
.collect()
}
/// Drop everything in the ring (the admin console's "clear" button).
pub fn clear() {
ring().lock().unwrap().clear();
}
pub fn push_line(line: &str) {
let line = line.trim_end();
if line.is_empty() {
return;
}
let mut g = ring().lock().unwrap();
if g.len() >= CAP {
g.pop_front();
}
g.push_back(line.to_string());
}
/// `MakeWriter` for the ring layer: buffers partial writes until a newline.
#[derive(Clone, Default)]
pub struct RingMakeWriter;
pub struct RingWriter {
buf: String,
}
impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for RingMakeWriter {
type Writer = RingWriter;
fn make_writer(&'a self) -> Self::Writer {
RingWriter { buf: String::new() }
}
}
impl io::Write for RingWriter {
fn write(&mut self, data: &[u8]) -> io::Result<usize> {
self.buf.push_str(&String::from_utf8_lossy(data));
while let Some(idx) = self.buf.find('\n') {
let line: String = self.buf.drain(..=idx).collect();
push_line(&line);
}
Ok(data.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl Drop for RingWriter {
fn drop(&mut self) {
if !self.buf.is_empty() {
push_line(&self.buf.clone());
}
}
}
+67 -10
View File
@@ -9,11 +9,14 @@
#![allow(dead_code)]
mod admin;
mod bench;
mod cache;
mod cmf_runtime;
mod config;
mod error;
mod import;
mod logbuf;
mod media;
mod model;
mod pipeline;
mod promotion;
@@ -25,6 +28,7 @@ mod routing;
mod secrets;
mod state;
mod stats;
mod sysmon;
use axum::{routing::get, Router};
use clap::Parser;
@@ -203,12 +207,23 @@ async fn main() -> anyhow::Result<()> {
if listen != original_listen {
let _ = cfg.save(&config_path);
}
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| cfg.log.level.clone().into()),
)
.init();
{
use tracing_subscriber::layer::SubscriberExt;
use tracing_subscriber::util::SubscriberInitExt;
let filter = tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| cfg.log.level.clone().into());
tracing_subscriber::registry()
.with(filter)
.with(tracing_subscriber::fmt::layer())
// second copy of every line, without ANSI, into the in-memory ring
// that backs the admin console's Логи view
.with(
tracing_subscriber::fmt::layer()
.with_ansi(false)
.with_writer(logbuf::RingMakeWriter),
)
.init();
}
let admin_enabled = cfg.admin.enabled;
let admin_token = resolve_admin_token(&cfg, &args);
@@ -216,6 +231,18 @@ async fn main() -> anyhow::Result<()> {
let state = SharedState::build(cfg, config_path.clone())?;
// Open admin console reachable from other machines: fine on a laptop,
// a footgun on a VPS. Flag it for the UI banner and say it in the log.
let host_public = listen.starts_with("0.0.0.0") || listen.starts_with("[::]");
if admin_enabled && admin_token.is_empty() && host_public && !is_docker() {
state
.admin_open_public
.store(true, std::sync::atomic::Ordering::Relaxed);
tracing::warn!(
"admin console is OPEN (no token) and listening on {listen} — anyone on your network can manage this gateway; set a token via --admin-token or [admin].token if this machine is reachable from outside"
);
}
// Close the loop with the CMF format (github.com/infosave2007/cmf): install
// / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
// the background so startup is never blocked by a cargo install or a model
@@ -231,7 +258,7 @@ async fn main() -> anyhow::Result<()> {
.route("/health", get(|| async { "ok" }))
.route("/readyz", get(|| async { "ready" }))
.route("/metrics", get(admin::metrics))
.merge(protocols::build_router());
.merge(protocols::build_router(state.clone()));
if admin_enabled {
app = app
@@ -244,9 +271,34 @@ async fn main() -> anyhow::Result<()> {
.allow_methods(tower_http::cors::Any)
.allow_headers(tower_http::cors::Any);
let app = app.with_state(state).layer(cors);
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
// small enough that a hostile client can't balloon memory
let app =
app.with_state(state)
.layer(cors)
.layer(tower_http::limit::RequestBodyLimitLayer::new(
32 * 1024 * 1024,
));
let listener = tokio::net::TcpListener::bind(&listen).await?;
let listener = {
let mut attempt = 0u32;
loop {
match tokio::net::TcpListener::bind(&listen).await {
Ok(l) => break l,
Err(e) => {
// during self-restart the old process is still letting go of
// the port — retry for a few seconds before giving up
let retrying = std::env::var("CORTIQ_BIND_RETRY").is_ok();
if retrying && attempt < 50 {
attempt += 1;
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
continue;
}
return Err(e.into());
}
}
}
};
tracing::info!("cortiq-gateway listening on {listen}");
if admin_enabled {
let host_for_url = if listen.starts_with("0.0.0.0:") {
@@ -295,6 +347,11 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("open this URL to finish setup: {url}");
}
}
axum::serve(listener, app).await?;
axum::serve(listener, app)
.with_graceful_shutdown(async {
let _ = tokio::signal::ctrl_c().await;
tracing::info!("shutdown signal received — draining connections");
})
.await?;
Ok(())
}
+861
View File
@@ -0,0 +1,861 @@
//! Media generation: run `cortiq imagine` (text→image, Lumina) and
//! `cortiq animate` (text→video+audio, MiniMax-H3) over local `.cmf` files as
//! tracked background jobs, the same shape as import jobs so the admin UI can
//! poll progress and fetch the result.
//!
//! These models are NOT served by `cortiq serve` — each generation is one CLI
//! run. The gateway owns the process, parses its progress, and keeps the output
//! under `<data>/media/<job>/`.
use serde::Serialize;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::process::Stdio;
use std::sync::{Arc, Mutex};
use std::time::{SystemTime, UNIX_EPOCH};
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::sync::Notify;
fn now() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// Random job id — media jobs are never deduplicated (every run is a new work).
fn gen_media_id() -> String {
let mut b = [0u8; 8];
let _ = getrandom::getrandom(&mut b);
b.iter().map(|x| format!("{x:02x}")).collect()
}
/// What a `.cmf` under models_dir can do, decided by `cortiq info`.
#[derive(Clone, Copy, PartialEq, Eq, Debug, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum Modality {
Chat,
Image,
Video,
}
/// `cortiq info` output → modality. The arch line names the family; the media
/// packs also carry their pipeline components (DiT/VAE/vocoder) in the text.
fn classify_info(info: &str) -> Modality {
let lower = info.to_ascii_lowercase();
let arch = lower
.lines()
.find_map(|l| l.trim().strip_prefix("arch:"))
.unwrap_or("")
.trim()
.to_string();
// generic arch suffixes first (e.g. "lumina2-image", "mmh3-video"), then
// known family names as a fallback for packs without the suffix
if arch.contains("video") || arch.contains("mmh3") || arch.contains("minimax") {
return Modality::Video;
}
if arch.contains("image") || arch.contains("lumina") || arch.contains("diffusion") {
return Modality::Image;
}
if lower.contains("text-to-video") {
return Modality::Video;
}
if lower.contains("text-to-image") {
return Modality::Image;
}
Modality::Chat
}
/// One local `.cmf` with its detected modality.
#[derive(Clone, Serialize)]
pub struct MediaModel {
pub name: String, // file stem
pub path: String,
pub size_bytes: u64,
pub modality: Modality,
}
#[derive(Default)]
struct InfoCache(Mutex<HashMap<String, (u64, u64, Modality)>>); // path → (mtime, size, modality)
/// Modality of one `.cmf`, via `cortiq info` — cached by (path, mtime, size),
/// so the process runs once per file version.
pub async fn classify_file(path: &Path, cortiq_bin: &str) -> Modality {
static CACHE: std::sync::OnceLock<InfoCache> = std::sync::OnceLock::new();
let cache = CACHE.get_or_init(InfoCache::default);
let size = crate::import::path_size(path);
let mtime = std::fs::metadata(path)
.ok()
.and_then(|m| m.modified().ok())
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
.map(|d| d.as_secs())
.unwrap_or(0);
let key = path.to_string_lossy().to_string();
if let Some(k) = cache
.0
.lock()
.unwrap()
.get(&key)
.filter(|(m, s, _)| *m == mtime && *s == size)
.map(|(_, _, k)| *k)
{
return k;
}
let output = tokio::process::Command::new(cortiq_bin)
.arg("info")
.arg(path)
.output()
.await;
let text = output
.map(|o| {
format!(
"{}{}",
String::from_utf8_lossy(&o.stdout),
String::from_utf8_lossy(&o.stderr)
)
})
.unwrap_or_default();
let k = classify_info(&text);
cache.0.lock().unwrap().insert(key, (mtime, size, k));
k
}
/// Scan models_dir and classify every `.cmf` (file or sharded dir).
pub async fn list_media_models(models_dir: &str, cortiq_bin: &str) -> Vec<MediaModel> {
let mut out = Vec::new();
let Ok(rd) = std::fs::read_dir(models_dir) else {
return out;
};
for entry in rd.filter_map(|e| e.ok()) {
let path = entry.path();
let name = path
.file_name()
.unwrap_or_default()
.to_string_lossy()
.to_string();
if !name.to_ascii_lowercase().ends_with(".cmf") {
continue;
}
let size = crate::import::path_size(&path);
if size == 0 {
continue; // empty or still being written
}
let modality = classify_file(&path, cortiq_bin).await;
out.push(MediaModel {
name: name
.trim_end_matches(".cmf")
.trim_end_matches(".CMF")
.to_string(),
path: path.to_string_lossy().to_string(),
size_bytes: size,
modality,
});
}
out.sort_by(|a, b| a.name.cmp(&b.name));
out
}
/// Parameters for one generation, straight from the playground form.
#[derive(Clone, Debug, serde::Deserialize)]
pub struct MediaParams {
pub model: String, // file stem under models_dir
pub kind: String, // image | video
pub prompt: String,
#[serde(default)]
pub width: Option<u32>,
#[serde(default)]
pub height: Option<u32>,
#[serde(default)]
pub steps: Option<u32>,
#[serde(default)]
pub cfg: Option<f32>, // image only
#[serde(default)]
pub seed: Option<u64>,
#[serde(default)]
pub frames: Option<u32>, // video only
#[serde(default)]
pub quality: Option<u32>, // video only: AVI JPEG quality
/// Optional first/last keyframe for video, base64-encoded binary P6 PPM.
#[serde(default)]
pub first_frame_b64: Option<String>,
#[serde(default)]
pub last_frame_b64: Option<String>,
}
/// Live state of one generation.
#[derive(Clone, Serialize, serde::Deserialize)]
pub struct MediaJob {
pub id: String,
pub kind: String, // image | video
pub model: String,
pub prompt: String,
pub width: u32,
pub height: u32,
pub steps: u32,
pub seed: u64,
pub frames: Option<u32>,
pub state: String, // queued | running | done | error | cancelled
pub progress: Option<f32>,
pub phase: Option<String>,
pub log: Vec<String>,
pub started: u64,
pub finished: Option<u64>,
/// Files this job produced: name → relative kind ("image", "video", "audio", "mp4").
pub outputs: Vec<String>,
pub out_bytes: Option<u64>,
}
/// Everything needed to actually launch a queued job when its turn comes.
struct Prepared {
bin: String,
args: Vec<String>,
gpu: bool,
dir: PathBuf,
out_path: PathBuf,
kind: String,
}
#[derive(Default)]
pub struct MediaStore {
jobs: Mutex<HashMap<String, MediaJob>>,
cancels: Mutex<HashMap<String, Arc<Notify>>>,
/// One generation runs at a time — these models take the whole GPU/RAM
/// budget — but new requests queue up instead of being rejected.
queue: Mutex<std::collections::VecDeque<String>>,
current: Mutex<Option<String>>,
prepared: Mutex<HashMap<String, Prepared>>,
persist: Mutex<Option<PathBuf>>,
}
impl MediaStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
/// Load saved generations (their files are still under media_dir) and start
/// persisting. Jobs that were queued/running when the gateway died settle
/// as errors — the process is gone.
pub fn attach_persistence(&self, path: PathBuf) {
if let Ok(text) = std::fs::read_to_string(&path) {
if let Ok(mut jobs) = serde_json::from_str::<Vec<MediaJob>>(&text) {
let mut g = self.jobs.lock().unwrap();
for mut j in jobs.drain(..) {
if j.state == "running" || j.state == "queued" {
j.state = "error".into();
j.log.push("✗ interrupted by a gateway restart".into());
j.finished = Some(now());
}
g.insert(j.id.clone(), j);
}
}
}
*self.persist.lock().unwrap() = Some(path);
}
fn save(&self) {
let Some(path) = self.persist.lock().unwrap().clone() else {
return;
};
let jobs: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
if let Ok(json) = serde_json::to_vec(&jobs) {
let tmp = path.with_extension("json.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &path);
}
}
}
/// How many jobs sit in the queue ahead of `id` (0 = next up).
pub fn queue_position(&self, id: &str) -> Option<usize> {
self.queue.lock().unwrap().iter().position(|x| x == id)
}
pub fn list(&self) -> Vec<MediaJob> {
let mut v: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
v.sort_by_key(|j| std::cmp::Reverse(j.started));
v.truncate(20);
v
}
pub fn get(&self, id: &str) -> Option<MediaJob> {
self.jobs.lock().unwrap().get(id).cloned()
}
pub fn cancel(self: &Arc<Self>, id: &str, media_dir: &Path) -> bool {
let state = self
.jobs
.lock()
.unwrap()
.get(id)
.map(|j| j.state.clone())
.unwrap_or_default();
match state.as_str() {
"queued" => {
self.queue.lock().unwrap().retain(|x| x != id);
self.prepared.lock().unwrap().remove(id);
let _ = std::fs::remove_dir_all(media_dir.join(id));
self.set_cancelled(id);
true
}
"running" => {
if let Some(n) = self.cancels.lock().unwrap().get(id) {
n.notify_one();
true
} else {
false
}
}
_ => false,
}
}
/// Remove a settled job and its output directory. Err = still running.
pub fn delete(&self, id: &str, media_dir: &Path) -> Result<bool, String> {
let job = self.jobs.lock().unwrap().get(id).cloned();
let Some(job) = job else { return Ok(false) };
if job.state == "running" || job.state == "queued" {
return Err("cancel the running generation first".into());
}
let _ = std::fs::remove_dir_all(media_dir.join(id));
self.jobs.lock().unwrap().remove(id);
self.save();
Ok(true)
}
fn push_line(&self, id: &str, line: String) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
j.log.push(line);
let n = j.log.len();
if n > 120 {
j.log.drain(0..n - 120);
}
}
}
fn set_progress(&self, id: &str, frac: f32, phase: Option<String>) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
let f = frac.clamp(0.0, 1.0);
if j.progress.is_none() || f > j.progress.unwrap() {
j.progress = Some(f);
}
if let Some(p) = phase {
j.phase = Some(p);
}
}
}
fn finish(self: &Arc<Self>, id: &str, ok: bool, outputs: Vec<String>, out_bytes: u64) {
let mut g = self.jobs.lock().unwrap();
if let Some(j) = g.get_mut(id) {
if j.state == "cancelled" {
drop(g);
self.after_slot_freed(id);
return;
}
j.finished = Some(now());
j.state = if ok { "done" } else { "error" }.into();
if ok {
j.progress = Some(1.0);
j.outputs = outputs;
j.out_bytes = Some(out_bytes);
}
}
drop(g);
self.after_slot_freed(id);
}
fn set_cancelled(self: &Arc<Self>, id: &str) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
j.state = "cancelled".into();
j.finished = Some(now());
}
self.after_slot_freed(id);
}
fn after_slot_freed(self: &Arc<Self>, id: &str) {
{
let mut cur = self.current.lock().unwrap();
if cur.as_deref() == Some(id) {
*cur = None;
}
}
self.cancels.lock().unwrap().remove(id);
self.save();
promote_next(self.clone());
}
}
/// Start the next queued generation, if the slot is free.
fn promote_next(store: Arc<MediaStore>) {
let next = {
let mut cur = store.current.lock().unwrap();
if cur.is_some() {
return;
}
let Some(id) = store.queue.lock().unwrap().pop_front() else {
return;
};
*cur = Some(id.clone());
id
};
let Some(p) = store.prepared.lock().unwrap().remove(&next) else {
// job was cancelled while queued and slipped through — free the slot
*store.current.lock().unwrap() = None;
return;
};
if let Some(j) = store.jobs.lock().unwrap().get_mut(&next) {
j.state = "running".into();
j.phase = Some("starting".into());
j.started = now();
}
store.save();
let cancel = Arc::new(Notify::new());
store
.cancels
.lock()
.unwrap()
.insert(next.clone(), cancel.clone());
let s = store.clone();
tokio::spawn(async move {
run_generation(
s, next, p.bin, p.args, p.gpu, p.dir, p.out_path, p.kind, cancel,
)
.await;
});
}
/// Progress markers in `cortiq imagine`/`animate` stdout. The exact wording has
/// varied between releases, so accept anything that carries an X/Y pair near a
/// step/frame word, a bare percentage, or the converter-style `@PROGRESS`.
fn parse_media_progress(line: &str) -> Option<(f32, Option<String>)> {
let l = line.trim();
if let Some(rest) = l.strip_prefix("@PROGRESS ") {
let mut it = rest.splitn(2, ' ');
let frac: f32 = it.next()?.trim().parse().ok()?;
let phase = it
.next()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty());
return Some((frac.clamp(0.0, 1.0), phase));
}
let lower = l.to_ascii_lowercase();
// "step 12/30", "denoise 3/4", "frame 7/39" — any word followed by x/y
for word in ["step", "frame", "sigma", "denois"] {
if let Some(pos) = lower.find(word) {
let tail = &lower[pos..];
if let Some((x, y)) = extract_ratio(tail) {
return Some((x / y, Some(l.to_string())));
}
}
}
// bare "42%" (avoid matching sizes like "42%faster")
if let Some(idx) = lower.find('%') {
let head: String = lower[..idx]
.chars()
.rev()
.take_while(|c| c.is_ascii_digit() || *c == '.')
.collect();
let num: String = head.chars().rev().collect();
if let Ok(p) = num.parse::<f32>() {
if (0.0..=100.0).contains(&p) {
return Some((p / 100.0, None));
}
}
}
None
}
/// First "N/M" pair in the string with M > 0.
fn extract_ratio(s: &str) -> Option<(f32, f32)> {
let bytes = s.as_bytes();
let mut i = 0;
while i < bytes.len() {
if bytes[i].is_ascii_digit() {
let start = i;
while i < bytes.len() && bytes[i].is_ascii_digit() {
i += 1;
}
if i < bytes.len() && bytes[i] == b'/' {
let x: f32 = s[start..i].parse().ok()?;
let ystart = i + 1;
let mut j = ystart;
while j < bytes.len() && bytes[j].is_ascii_digit() {
j += 1;
}
if j > ystart {
if let Ok(y) = s[ystart..j].parse::<f32>() {
if y > 0.0 && x <= y {
return Some((x, y));
}
}
}
}
} else {
i += 1;
}
}
None
}
/// Root for generated files: `<data>/media`.
pub fn media_dir() -> PathBuf {
crate::config::data_dir().join("media")
}
const B64: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
/// Minimal base64 decode (standard alphabet, padding optional) — the keyframes
/// are the only base64 input the gateway takes; not worth a dependency.
fn b64_decode(s: &str) -> Option<Vec<u8>> {
let mut lut = [255u8; 256];
for (i, c) in B64.iter().enumerate() {
lut[*c as usize] = i as u8;
}
let mut out = Vec::with_capacity(s.len() * 3 / 4);
let mut buf = 0u32;
let mut bits = 0u32;
for c in s.bytes() {
if c == b'=' || c == b'\n' || c == b'\r' {
continue;
}
let v = lut[c as usize];
if v == 255 {
return None;
}
buf = (buf << 6) | v as u32;
bits += 6;
if bits >= 8 {
bits -= 8;
out.push((buf >> bits) as u8);
}
}
Some(out)
}
/// Kick off a generation. Returns the job id, or an error when another one is
/// already running / the model file is missing.
pub fn start_generation(
store: Arc<MediaStore>,
cmf: &crate::config::CmfCfg,
p: MediaParams,
) -> Result<String, String> {
if p.prompt.trim().is_empty() {
return Err("empty prompt".into());
}
let kind = p.kind.as_str();
if kind != "image" && kind != "video" {
return Err(format!("unknown kind '{kind}'"));
}
let model_path = Path::new(&cmf.models_dir).join(format!("{}.cmf", p.model));
if !model_path.exists() {
return Err(format!("model file not found: {}", model_path.display()));
}
let id = gen_media_id();
let dir = media_dir().join(&id);
if let Err(e) = std::fs::create_dir_all(&dir) {
return Err(format!("cannot create media dir {}: {e}", dir.display()));
}
// Optional keyframes (video): decode to PPM files next to the output.
let mut first_frame: Option<PathBuf> = None;
let mut last_frame: Option<PathBuf> = None;
for (b64, slot, name) in [
(&p.first_frame_b64, &mut first_frame, "first.ppm"),
(&p.last_frame_b64, &mut last_frame, "last.ppm"),
] {
if let Some(data) = b64.as_deref().filter(|s| !s.is_empty()) {
let bytes = b64_decode(data).ok_or("invalid keyframe base64")?;
if !bytes.starts_with(b"P6") {
return Err("keyframe must be a binary P6 PPM".into());
}
let path = dir.join(name);
std::fs::write(&path, bytes).map_err(|e| format!("write keyframe: {e}"))?;
*slot = Some(path);
}
}
let width = p.width.unwrap_or(512).clamp(64, 2048);
let height = p
.height
.unwrap_or(if kind == "video" { 288 } else { 512 })
.clamp(64, 2048);
let steps = p
.steps
.unwrap_or(if kind == "video" { 4 } else { 30 })
.clamp(1, 200);
let seed = p.seed.unwrap_or(42);
let frames = p.frames.unwrap_or(39).clamp(1, 1000);
let out_name = if kind == "image" {
"out.ppm"
} else {
"out.avi"
};
let out_path = dir.join(out_name);
let mut args: Vec<String> = if kind == "image" {
vec![
"imagine".into(),
model_path.to_string_lossy().to_string(),
"--prompt".into(),
p.prompt.clone(),
"--width".into(),
width.to_string(),
"--height".into(),
height.to_string(),
"--steps".into(),
steps.to_string(),
"--cfg".into(),
format!("{}", p.cfg.unwrap_or(4.0)),
"--seed".into(),
seed.to_string(),
"--out".into(),
out_path.to_string_lossy().to_string(),
]
} else {
let mut a = vec![
"animate".into(),
model_path.to_string_lossy().to_string(),
"--prompt".into(),
p.prompt.clone(),
"--width".into(),
width.to_string(),
"--height".into(),
height.to_string(),
"--frames".into(),
frames.to_string(),
"--steps".into(),
steps.to_string(),
"--seed".into(),
seed.to_string(),
"--quality".into(),
p.quality.unwrap_or(92).clamp(10, 100).to_string(),
"--out".into(),
out_path.to_string_lossy().to_string(),
];
if let Some(f) = &first_frame {
a.push("--first-frame".into());
a.push(f.to_string_lossy().to_string());
}
if let Some(f) = &last_frame {
a.push("--last-frame".into());
a.push(f.to_string_lossy().to_string());
}
a
};
// keep the model path first for readability in logs
let _ = &mut args;
store.jobs.lock().unwrap().insert(
id.clone(),
MediaJob {
id: id.clone(),
kind: kind.to_string(),
model: p.model.clone(),
prompt: p.prompt.clone(),
width,
height,
steps,
seed,
frames: (kind == "video").then_some(frames),
state: "queued".into(),
progress: None,
phase: None,
log: vec![format!("→ cortiq {}", args.join(" "))],
started: now(),
finished: None,
outputs: Vec::new(),
out_bytes: None,
},
);
store.prepared.lock().unwrap().insert(
id.clone(),
Prepared {
bin: cmf.cortiq_bin.clone(),
args,
gpu: cmf.gpu,
dir,
out_path,
kind: kind.to_string(),
},
);
store.queue.lock().unwrap().push_back(id.clone());
store.save();
promote_next(store.clone());
Ok(id)
}
#[allow(clippy::too_many_arguments)]
async fn run_generation(
store: Arc<MediaStore>,
id: String,
bin: String,
args: Vec<String>,
gpu: bool,
dir: PathBuf,
out_path: PathBuf,
kind: String,
cancel: Arc<Notify>,
) {
let mut cmd = tokio::process::Command::new(&bin);
cmd.args(&args)
.env("CMF_GPU", if gpu { "1" } else { "0" })
.stdout(Stdio::piped())
.stderr(Stdio::piped());
let mut child = match cmd.spawn() {
Ok(c) => c,
Err(e) => {
store.push_line(&id, format!("✗ failed to start cortiq: {e}"));
store.finish(&id, false, Vec::new(), 0);
return;
}
};
let mut readers = Vec::new();
if let Some(o) = child.stdout.take() {
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), o)));
}
if let Some(e) = child.stderr.take() {
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), e)));
}
let status = tokio::select! {
s = child.wait() => s,
_ = cancel.notified() => {
let _ = child.start_kill();
let _ = child.wait().await;
for r in readers { r.abort(); }
let _ = std::fs::remove_dir_all(&dir);
store.push_line(&id, "✗ cancelled".into());
store.set_cancelled(&id);
return;
}
};
for r in readers {
let _ = r.await;
}
let ok = status.map(|s| s.success()).unwrap_or(false);
if !ok || !out_path.exists() {
store.push_line(&id, "✗ generation failed — see the log above".into());
store.finish(&id, false, Vec::new(), 0);
return;
}
let mut outputs = vec![if kind == "image" { "image" } else { "video" }.to_string()];
if kind == "video" {
// `animate` writes a .wav next to the .avi
if dir.join("out.wav").exists() {
outputs.push("audio".into());
}
// opportunistic browser-friendly remux when ffmpeg is present
if let Ok(p) = which_ffmpeg() {
store.push_line(&id, "→ remuxing to mp4 (ffmpeg found)".into());
let mp4 = dir.join("out.mp4");
let mut f = tokio::process::Command::new(p);
f.arg("-y").arg("-i").arg(dir.join("out.avi"));
if dir.join("out.wav").exists() {
f.arg("-i")
.arg(dir.join("out.wav"))
.args(["-map", "0:v", "-map", "1:a", "-c:a", "aac"]);
}
f.args(["-c:v", "libx264", "-pix_fmt", "yuv420p"]).arg(&mp4);
f.stdout(Stdio::null()).stderr(Stdio::null());
if matches!(f.status().await, Ok(s) if s.success()) && mp4.exists() {
outputs.push("mp4".into());
}
}
}
let bytes = crate::import::path_size(&out_path);
store.push_line(
&id,
format!("✓ done — {}", crate::import::format_bytes(bytes)),
);
store.finish(&id, true, outputs, bytes);
}
/// Strip ANSI color escapes that `tracing` writes even into a pipe.
fn strip_ansi(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut chars = s.chars();
while let Some(c) = chars.next() {
if c == '\u{1b}' {
for d in chars.by_ref() {
if d.is_ascii_alphabetic() {
break;
}
}
} else {
out.push(c);
}
}
out
}
async fn read_lines(store: Arc<MediaStore>, id: String, stream: impl tokio::io::AsyncRead + Unpin) {
let mut lines = BufReader::new(stream).lines();
while let Ok(Some(l)) = lines.next_line().await {
let l = strip_ansi(l.trim());
if l.is_empty() {
continue;
}
// the Metal backend logs one line per pipeline — dozens per run, all identical
if l.contains("Metal GPU path") {
continue;
}
if let Some((frac, phase)) = parse_media_progress(&l) {
store.set_progress(&id, frac, phase);
} else {
store.push_line(&id, l);
}
}
}
fn which_ffmpeg() -> Result<String, ()> {
for p in [
"ffmpeg",
"/usr/local/bin/ffmpeg",
"/opt/homebrew/bin/ffmpeg",
] {
if std::process::Command::new(p)
.arg("-version")
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
{
return Ok(p.to_string());
}
}
Err(())
}
/// (relative file, content-type) for an output kind of a finished job.
pub fn output_file(kind: &str) -> Option<(&'static str, &'static str)> {
match kind {
"image" => Some(("out.ppm", "image/x-portable-pixmap")),
"video" => Some(("out.avi", "video/x-msvideo")),
"audio" => Some(("out.wav", "audio/wav")),
"mp4" => Some(("out.mp4", "video/mp4")),
_ => None,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn progress_parses_common_shapes() {
assert_eq!(parse_media_progress("step 15/30").unwrap().0, 0.5);
assert_eq!(
parse_media_progress("Denoising step 3/4 ...").unwrap().0,
0.75
);
assert_eq!(parse_media_progress("frame 39/39 decoded").unwrap().0, 1.0);
assert_eq!(parse_media_progress("@PROGRESS 0.25 vae").unwrap().0, 0.25);
assert!(parse_media_progress("loaded 2361 tensors").is_none());
}
#[test]
fn classify_by_arch() {
assert_eq!(classify_info(" Arch: lumina2\n"), Modality::Image);
assert_eq!(classify_info(" Arch: mmh3\n"), Modality::Video);
assert_eq!(classify_info(" Arch: qwen3\n"), Modality::Chat);
}
#[test]
fn b64_roundtrip() {
assert_eq!(b64_decode("UDYgd2g=").unwrap(), b"P6 wh");
assert!(b64_decode("!!").is_none());
}
}
+47 -2
View File
@@ -15,13 +15,53 @@ pub enum RoutingDirective {
Pinned { model_id: String },
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
pub struct Message {
pub role: String, // system | user | assistant | tool
#[serde(default)]
/// Always a plain string internally. On the wire OpenAI clients legitimately
/// send `null` (assistant turns that only carry tool calls) or an array of
/// content parts (Open WebUI and every other multimodal UI), so
/// [`de_content`] flattens both shapes instead of rejecting the request.
#[serde(default, deserialize_with = "de_content")]
pub content: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub tool_calls: Vec<serde_json::Value>,
/// Speaker name (`messages[].name`) — proxied through when the client sends it.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
/// Set on `role = "tool"` replies; dropping it breaks multi-turn tool calling.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
}
/// Deserialize `messages[].content` from any shape an OpenAI-compatible client
/// may send: a string, `null`, or an array of content parts. Text parts are
/// concatenated; non-text parts (images, audio, files) are skipped — this
/// gateway routes text, and a picture must not turn into a 422.
fn de_content<'de, D>(d: D) -> std::result::Result<String, D::Error>
where
D: serde::Deserializer<'de>,
{
use serde::Deserialize as _;
Ok(match serde_json::Value::deserialize(d)? {
serde_json::Value::Null => String::new(),
serde_json::Value::String(s) => s,
serde_json::Value::Array(parts) => {
let mut out = String::new();
for part in parts {
// {"type":"text","text":"..."} / {"type":"input_text","text":"..."}
let text = part
.get("text")
.and_then(|t| t.as_str())
.or_else(|| part.as_str());
if let Some(t) = text {
out.push_str(t);
}
}
out
}
other => other.to_string(),
})
}
#[derive(Clone, Debug, Default)]
@@ -73,6 +113,11 @@ pub struct RouteInfo {
pub cost_usd: f64,
#[serde(default)]
pub failover: bool,
/// The client offered `tools` but the model that answered cannot use them,
/// so they were dropped from the upstream call (see `Pipeline::resolve`).
/// Surfaced as `X-Cortiq-Tools-Dropped` so a silent downgrade is debuggable.
#[serde(default)]
pub tools_dropped: bool,
}
#[derive(Clone, Debug, Serialize)]
+271 -18
View File
@@ -183,7 +183,10 @@ impl Pipeline {
&self,
req: &ChatRequest,
live: &crate::state::Live,
) -> Result<(RouteDecision, String, Vec<String>)> {
// Tool support detected via the local servers' healthz handshake —
// treated as equivalent to declared caps when filtering candidates.
detected_tools: &std::collections::HashSet<String>,
) -> Result<(RouteDecision, String, Vec<String>, bool)> {
// 1. Extract text for classification
let text = self.extract_text(
&req.messages,
@@ -231,9 +234,30 @@ impl Pipeline {
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
match live.router.route(&text, prof).await {
Ok(Some(dec)) => {
let mut dec = dec;
// per-gateway complexity bands: re-map the
// router's continuous score with the user's
// own boundaries (sliders on the Routing page)
if let Some(b) = &live.cfg.routing.bands {
if matches!(
dec.complexity_tier.as_str(),
"low" | "medium" | "high"
) {
dec.complexity_tier =
b.tier_of(dec.complexity_score).to_string();
}
}
let src = dec.source.clone();
let tier = dec.complexity_tier.clone();
(dec, src, live.routing.candidates(&tier))
let (cands, task_rule) = live
.routing
.candidates_for(&dec.task_label, &dec.complexity_tier);
// surface which rule chose the head of the chain
let src = if task_rule {
format!("{src}+task-rule")
} else {
src
};
(dec, src, cands)
}
Ok(None) | Err(_) => {
tracing::warn!(
@@ -253,10 +277,14 @@ impl Pipeline {
// Filter candidates by capabilities (e.g. tools support)
let needs_tools = !req.tools.is_empty();
let tools_mandatory = needs_tools && tool_choice_is_mandatory(&req.params.passthrough);
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
prov.caps().tools || detected_tools.contains(id)
};
let mut final_candidates = Vec::new();
for model_id in candidates {
if let Some(prov) = live.registry.get(&model_id) {
if needs_tools && !prov.caps().tools {
if needs_tools && !has_tools(&model_id, &prov) {
tracing::debug!(
"Candidate '{}' skipped: lacks support for requested tools",
model_id
@@ -267,6 +295,31 @@ impl Pipeline {
}
}
// `tools` is an OFFER in the OpenAI contract, not a demand: only
// `tool_choice: "required"` (or a named function) makes the call
// impossible without them. Chat UIs lean on that — Open WebUI attaches
// its ~26 built-in tools to every single message — so treating the offer
// as a hard requirement made an ordinary "hello" unanswerable on a pool
// of tools-less local models. Serve the request without the tools and
// flag the downgrade (`X-Cortiq-Tools-Dropped`) instead of failing.
let mut tools_dropped = false;
// A pin stays a pin: when the pinned model exists and merely lacks tool
// support, honour the pin and drop the offer rather than reporting the
// model as missing.
if final_candidates.is_empty() && needs_tools && !tools_mandatory {
if let RoutingDirective::Pinned { model_id } = &req.routing {
if live.registry.get(model_id).is_some() {
final_candidates.push(model_id.clone());
tools_dropped = true;
tracing::info!(
model = %model_id,
"pinned model does not support the offered tools — serving without them"
);
}
}
}
// An explicit pin must never silently answer from a different model: if
// the pinned id is unknown or was filtered out, fail loudly so the caller
// knows their choice was not honoured (instead of getting a stray reply
@@ -285,25 +338,86 @@ impl Pipeline {
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
for id in live.local_candidates() {
if let Some(prov) = live.registry.get(&id) {
if needs_tools && !prov.caps().tools {
if needs_tools && !has_tools(&id, &prov) {
continue;
}
final_candidates.push(id);
}
}
}
// Capability widening: a tier chain built of tools-less local models
// must not kill an agent request. When the filtered chain is empty on
// an AUTO route, widen to any pool model that satisfies the required
// capabilities — preferring the routing table's own medium→high order
// (an agentic request is worth a capable model), then the rest of the
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
// this every low-tier verdict was a hard failure.
let widened_order = || -> Vec<String> {
let mut widened: Vec<String> = Vec::new();
for tier in ["medium", "high"] {
for id in live.routing.candidates(tier) {
if !widened.contains(&id) {
widened.push(id);
}
}
}
for id in live.registry.ids() {
if !widened.contains(id) {
widened.push(id.clone());
}
}
widened
};
if final_candidates.is_empty() {
for id in widened_order() {
if let Some(prov) = live.registry.get(&id) {
if needs_tools && !has_tools(&id, &prov) {
continue;
}
final_candidates.push(id);
}
}
if !final_candidates.is_empty() {
tracing::info!(
chosen = %final_candidates[0],
"capability widening: tier chain had no model with the required capabilities"
);
}
}
// Same relaxation for an AUTO route once every narrower pass came up empty.
if final_candidates.is_empty() && needs_tools && !tools_mandatory {
for id in widened_order() {
if live.registry.get(&id).is_some() {
final_candidates.push(id);
}
}
if !final_candidates.is_empty() {
tools_dropped = true;
tracing::info!(
chosen = %final_candidates[0],
"no model in the pool supports the offered tools — serving without them"
);
}
}
if final_candidates.is_empty() {
return Err(GatewayError::UpstreamUnavailable(
"No healthy model available that satisfies required capabilities (e.g. tools)"
.to_string(),
));
}
Ok((decision, route_source, final_candidates))
Ok((decision, route_source, final_candidates, tools_dropped))
}
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
async fn run_inner(&self, mut req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
let live = state.live();
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
let (decision, route_source, mut final_candidates, tools_dropped) =
self.resolve(&req, &live, &detected_tools).await?;
if tools_dropped {
strip_tools(&mut req);
}
// Serving-when-promoted (gated): a promoted task-label is served by the
// local model FIRST (failover to cloud on error). Per-request VETO uses
@@ -365,6 +479,7 @@ impl Pipeline {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
..Default::default()
},
);
}
@@ -392,6 +507,7 @@ impl Pipeline {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
..Default::default()
},
);
}
@@ -411,6 +527,7 @@ impl Pipeline {
router_request_id: decision.router_request_id.clone(),
cost_usd: cost,
failover: attempt > 0,
tools_dropped,
};
return Ok(resp);
}
@@ -437,7 +554,7 @@ impl Pipeline {
/// recorded when the stream completes (tokens taken from the final usage chunk).
pub async fn run_stream(
&self,
req: ChatRequest,
mut req: ChatRequest,
state: &SharedState,
) -> Result<(RouteInfo, crate::providers::ChatStream)> {
let started = Instant::now();
@@ -450,13 +567,18 @@ impl Pipeline {
.to_string();
let live = state.live();
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
Ok(t) => t,
Err(e) => {
record_error(state, &account, &protocol, &directive, started, &e);
return Err(e);
}
};
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
let (decision, route_source, final_candidates, tools_dropped) =
match self.resolve(&req, &live, &detected_tools).await {
Ok(t) => t,
Err(e) => {
record_error(state, &account, &protocol, &directive, started, &e);
return Err(e);
}
};
if tools_dropped {
strip_tools(&mut req);
}
let mut last_err = None;
let mut attempt = 0usize;
@@ -497,6 +619,7 @@ impl Pipeline {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
..Default::default()
},
);
}
@@ -524,6 +647,7 @@ impl Pipeline {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
..Default::default()
},
);
}
@@ -541,7 +665,13 @@ impl Pipeline {
router_request_id: decision.router_request_id.clone(),
cost_usd: 0.0,
failover: attempt > 0,
tools_dropped,
};
// for the client-facing usage estimate when the provider
// sends none: the prompt size is known here, the reply is
// measured inside the tap
let prompt_chars: usize =
req.messages.iter().map(|m| m.content.chars().count()).sum();
let tapped = tap_stream(
stream,
state.stats.clone(),
@@ -551,6 +681,7 @@ impl Pipeline {
protocol,
directive,
started,
prompt_chars,
);
return Ok((info, Box::pin(tapped)));
}
@@ -612,6 +743,26 @@ impl Pipeline {
}
}
/// Remove the tool offer from a request the chosen model cannot honour. The
/// companion fields must go too: an upstream given `tool_choice` with no
/// `tools` answers 400.
fn strip_tools(req: &mut ChatRequest) {
req.tools.clear();
req.params.passthrough.remove("tool_choice");
req.params.passthrough.remove("parallel_tool_calls");
}
/// Is the client's `tool_choice` a demand rather than a preference?
/// `"required"` and `{"type":"function", …}` mean the caller cannot be served
/// without tools; `"auto"`, `"none"` and an absent field are just an offer.
fn tool_choice_is_mandatory(passthrough: &serde_json::Map<String, serde_json::Value>) -> bool {
match passthrough.get("tool_choice") {
Some(serde_json::Value::String(s)) => s.eq_ignore_ascii_case("required"),
Some(serde_json::Value::Object(_)) => true,
_ => false,
}
}
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
@@ -661,6 +812,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
role: "assistant".into(),
content: hit.content.clone(),
tool_calls: vec![],
..Default::default()
},
finish_reason: "stop".into(),
}],
@@ -678,6 +830,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
router_request_id: None,
cost_usd: 0.0,
failover: false,
tools_dropped: false,
},
}
}
@@ -722,6 +875,7 @@ fn tap_stream(
protocol: String,
directive: String,
started: Instant,
prompt_chars: usize,
) -> impl futures::Stream<Item = Result<bytes::Bytes>> {
use futures::StreamExt;
async_stream::stream! {
@@ -731,12 +885,15 @@ fn tap_stream(
let mut completion_tokens = 0u32;
let mut got_usage = false;
let mut delta_chars = 0usize;
let mut delta_chunks = 0u32;
let mut errored: Option<String> = None;
while let Some(item) = inner.next().await {
let mut done_marker = false;
match &item {
Ok(bytes) => {
if let Ok(s) = std::str::from_utf8(bytes) {
done_marker = s.contains("data: [DONE]");
buf.push_str(s);
while let Some(idx) = buf.find("\n\n") {
let evt: String = buf.drain(..idx + 2).collect();
@@ -765,6 +922,7 @@ fn tap_stream(
v["choices"][0]["delta"]["content"].as_str()
{
delta_chars += c.chars().count();
delta_chunks += 1;
}
}
}
@@ -774,12 +932,52 @@ fn tap_stream(
}
Err(e) => errored = Some(e.to_string()),
}
// The local cortiq server (and some providers) never send a usage
// chunk. Clients still deserve token counts, so ahead of [DONE] we
// inject one final OpenAI-shaped usage event with the gateway's
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
// mistakes it for the tokenizer's truth.
if done_marker {
// The initial X-Cortiq-Cost-USD header is always 0 — cost is
// only known after generation. Ship the final number (and the
// token estimate when the provider sent none) ahead of [DONE].
if !got_usage {
prompt_tokens = (prompt_chars / 4) as u32;
completion_tokens = if delta_chunks > 0 {
delta_chunks
} else {
(delta_chars / 4) as u32
};
}
let cost = provider.price(prompt_tokens, completion_tokens);
let mut evt = serde_json::json!({
"object": "chat.completion.chunk",
"model": info.selected_model,
"choices": [],
"cortiq_cost_usd": cost,
});
if !got_usage {
evt["usage"] = serde_json::json!({
"prompt_tokens": prompt_tokens,
"completion_tokens": completion_tokens,
"total_tokens": prompt_tokens + completion_tokens,
});
evt["cortiq_estimated"] = serde_json::json!(true);
got_usage = true;
}
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
}
yield item;
}
if !got_usage && completion_tokens == 0 {
// estimate when the server didn't return usage (~4 chars per token)
completion_tokens = (delta_chars / 4) as u32;
// stream ended without a [DONE] marker — estimate for the stats
completion_tokens = if delta_chunks > 0 {
delta_chunks
} else {
(delta_chars / 4) as u32
};
prompt_tokens = (prompt_chars / 4) as u32;
}
let cost = provider.price(prompt_tokens, completion_tokens);
stats.record(RequestRecord {
@@ -838,6 +1036,7 @@ fn shadow_req(model_id: &str, content: String, max_tokens: u32) -> ChatRequest {
role: "user".to_string(),
content,
tool_calls: Vec::new(),
..Default::default()
}],
tools: Vec::new(),
params: GenParams {
@@ -943,3 +1142,57 @@ async fn shadow_capture(
judge_tokens,
});
}
#[cfg(test)]
mod tests {
use super::*;
fn map(json: serde_json::Value) -> serde_json::Map<String, serde_json::Value> {
json.as_object().cloned().unwrap_or_default()
}
#[test]
fn an_offered_tool_list_is_not_a_demand() {
// Open WebUI attaches its built-in tools with no tool_choice at all.
assert!(!tool_choice_is_mandatory(&map(serde_json::json!({}))));
assert!(!tool_choice_is_mandatory(&map(
serde_json::json!({"tool_choice": "auto"})
)));
assert!(!tool_choice_is_mandatory(&map(
serde_json::json!({"tool_choice": "none"})
)));
}
#[test]
fn required_and_named_tool_choice_are_demands() {
assert!(tool_choice_is_mandatory(&map(
serde_json::json!({"tool_choice": "required"})
)));
assert!(tool_choice_is_mandatory(&map(
serde_json::json!({"tool_choice": {"type": "function", "function": {"name": "f"}}})
)));
}
#[test]
fn stripping_tools_also_drops_their_companion_fields() {
let mut req = ChatRequest {
routing: RoutingDirective::Auto { profile: None },
messages: vec![],
tools: vec![serde_json::json!({"type": "function"})],
params: GenParams {
passthrough: map(
serde_json::json!({"tool_choice": "auto", "parallel_tool_calls": true, "top_k": 40}),
),
..Default::default()
},
stream: false,
meta: RequestMeta::default(),
};
strip_tools(&mut req);
assert!(req.tools.is_empty());
assert!(!req.params.passthrough.contains_key("tool_choice"));
assert!(!req.params.passthrough.contains_key("parallel_tool_calls"));
// unrelated passthrough must survive
assert!(req.params.passthrough.contains_key("top_k"));
}
}
+16 -10
View File
@@ -5,7 +5,7 @@
use crate::error::{GatewayError, Result};
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
use crate::protocols::openai_chat::cortiq_headers;
use crate::protocols::openai_chat::{cortiq_headers, sse_headers};
use crate::state::SharedState;
use axum::response::{IntoResponse, Response};
use axum::{extract::State, routing::post, Json, Router};
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<MessagesRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.anthropic_messages {
@@ -83,6 +84,7 @@ async fn handler(
role: "system".into(),
content: sys,
tool_calls: vec![],
..Default::default()
});
}
for m in &req.messages {
@@ -90,9 +92,12 @@ async fn handler(
role: m["role"].as_str().unwrap_or("user").to_string(),
content: content_to_string(&m["content"]),
tool_calls: vec![],
..Default::default()
});
}
// Echoed back as-is: clients match the reply against the id they asked for.
let requested_model = req.model.clone();
let canonical = ChatRequest {
routing: parse_routing(&req.model),
messages,
@@ -108,19 +113,19 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "anthropic_messages".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
if req.stream {
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
let mut headers = cortiq_headers(&info);
headers.insert(
axum::http::header::CONTENT_TYPE,
axum::http::HeaderValue::from_static("text/event-stream"),
);
let translated = openai_to_anthropic_sse(stream);
return Ok((headers, axum::body::Body::from_stream(translated)).into_response());
let translated = openai_to_anthropic_sse(stream, requested_model);
return Ok((
sse_headers(&info),
axum::body::Body::from_stream(translated),
)
.into_response());
}
let resp = state.pipeline.run(canonical, &state).await?;
@@ -134,7 +139,7 @@ async fn handler(
"id": resp.id,
"type": "message",
"role": "assistant",
"model": resp.model_used,
"model": requested_model,
"content": [{ "type": "text", "text": answer }],
"stop_reason": "end_turn",
"stop_sequence": serde_json::Value::Null,
@@ -150,6 +155,7 @@ async fn handler(
/// events, so Anthropic-native clients can stream regardless of the upstream provider.
fn openai_to_anthropic_sse(
stream: crate::providers::ChatStream,
model: String,
) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
use futures::StreamExt;
async_stream::stream! {
@@ -160,7 +166,7 @@ fn openai_to_anthropic_sse(
yield Ok(evt("message_start", serde_json::json!({
"type": "message_start",
"message": {"id": "msg_gw", "type": "message", "role": "assistant",
"content": [], "model": "cortiq-auto", "stop_reason": null,
"content": [], "model": model, "stop_reason": null,
"usage": {"input_tokens": 0, "output_tokens": 0}}
})));
yield Ok(evt("content_block_start", serde_json::json!({
+1
View File
@@ -94,6 +94,7 @@ async fn handle_tool_call(
role: "user".into(),
content: prompt.to_string(),
tool_calls: vec![],
..Default::default()
}],
tools: vec![],
params: GenParams::default(),
+51 -1
View File
@@ -12,10 +12,59 @@ pub mod openai_models;
use crate::state::SharedState;
use axum::Router;
/// Account resolved by the API-key middleware, consumed by protocol handlers
/// for attribution (stats, budgets). Present only when keys are configured.
#[derive(Clone)]
pub struct AccountTag(pub String);
/// API-key gate for the model-facing surface. Open mode (no keys configured)
/// passes everything through — the admin console already warns about it.
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
/// enforced with a fixed one-minute window (429 + Retry-After), and a
/// monthly budget, when set, turns into 429 insufficient_quota once the
/// account's month-to-date spend crosses it.
async fn api_auth(
axum::extract::State(state): axum::extract::State<SharedState>,
mut req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
use axum::response::IntoResponse;
let live = state.live();
if live.cfg.api_keys.is_empty() {
return next.run(req).await;
}
let provided = req
.headers()
.get(axum::http::header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|v| v.strip_prefix("Bearer "))
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
.map(str::trim)
.unwrap_or("");
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
.into_response();
};
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
return crate::error::GatewayError::RateLimited {
retry_after_secs: state.rate.retry_after_secs(),
}
.into_response();
}
if let Some(budget) = key.monthly_budget_usd {
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
.into_response();
}
}
req.extensions_mut().insert(AccountTag(key.account.clone()));
next.run(req).await
}
/// Build the protocols router. Implemented adapters are **always** mounted;
/// each handler checks the live `protocols.*` flag from config itself, so
/// toggles in the admin panel take effect without a restart (hot switching).
pub fn build_router() -> Router<SharedState> {
pub fn build_router(state: SharedState) -> Router<SharedState> {
Router::new()
.merge(openai_chat::routes())
.merge(openai_completions::routes())
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
.merge(anthropic_messages::routes())
.merge(mcp::routes())
.merge(native_passthrough::routes())
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
}
+421 -42
View File
@@ -1,6 +1,17 @@
//! Incoming adapter for OpenAI Chat Completions: `POST /v1/chat/completions`.
//! Translates the OpenAI request body into a canonical [`ChatRequest`], passes it
//! to the pipeline, then converts the canonical [`ChatResponse`] back to OpenAI format.
//!
//! Wire-compatibility rules this adapter enforces (see docs/PROTOCOLS.md §2):
//! * the request body is parsed leniently — every shape a real OpenAI client
//! sends (`content: null`, content-part arrays, `tools: null`, a string
//! `reasoning_effort`, UI-private extras) is accepted, and a body we truly
//! cannot read comes back as an OpenAI error envelope, never a bare 422;
//! * the response is strict OpenAI JSON — no optional `null` fields, a
//! spec-legal `finish_reason`, and the *requested* model id echoed back;
//! * the SSE stream is re-normalised chunk by chunk and always terminates
//! with `data: [DONE]`, so a client can never sit waiting on a half-formed
//! stream from an upstream that improvises.
use crate::error::{GatewayError, Result};
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RouteInfo, RoutingDirective};
@@ -31,35 +42,119 @@ pub(crate) fn cortiq_headers(c: &RouteInfo) -> axum::http::HeaderMap {
if let Some(id) = &c.router_request_id {
put(&mut h, "X-Cortiq-Request-Id", id);
}
if c.tools_dropped {
put(&mut h, "X-Cortiq-Tools-Dropped", "true");
}
h
}
/// Headers every SSE response carries. `X-Accel-Buffering: no` matters when the
/// gateway sits behind nginx/Traefik — without it the proxy buffers the whole
/// stream and the client's chat window just spins.
pub(crate) fn sse_headers(info: &RouteInfo) -> axum::http::HeaderMap {
use axum::http::HeaderValue;
let mut headers = cortiq_headers(info);
headers.insert(
axum::http::header::CONTENT_TYPE,
HeaderValue::from_static("text/event-stream; charset=utf-8"),
);
headers.insert(
axum::http::header::CACHE_CONTROL,
HeaderValue::from_static("no-cache"),
);
headers.insert("X-Accel-Buffering", HeaderValue::from_static("no"));
headers
}
pub fn routes() -> Router<SharedState> {
Router::new().route("/v1/chat/completions", post(handler))
}
/// `reasoning_effort` is a string in the OpenAI spec (`low` | `medium` | `high`)
/// but a token budget in several local runtimes. Accept both.
#[derive(Deserialize)]
#[serde(untagged)]
enum Effort {
Budget(u32),
Level(String),
}
impl Effort {
fn to_budget(&self) -> Option<u32> {
match self {
Self::Budget(n) => Some(*n),
Self::Level(s) => match s.trim().to_ascii_lowercase().as_str() {
"none" | "off" | "minimal" => Some(0),
"low" => Some(512),
"medium" | "default" | "auto" => Some(2048),
"high" | "max" => Some(8192),
_ => None,
},
}
}
}
#[derive(Deserialize)]
struct OpenAiChatRequest {
model: String,
model: Option<String>,
messages: Vec<Message>,
#[serde(default)]
temperature: Option<f32>,
#[serde(default)]
max_tokens: Option<u32>,
/// The modern spelling of `max_tokens`; clients send one or the other.
#[serde(default)]
max_completion_tokens: Option<u32>,
#[serde(default)]
top_p: Option<f32>,
#[serde(default)]
think_budget: Option<u32>,
#[serde(default)]
reasoning_effort: Option<u32>,
reasoning_effort: Option<Effort>,
/// `null` is a legal value here for several clients, hence `Option`.
#[serde(default)]
stream: bool,
stream: Option<bool>,
#[serde(default)]
tools: Vec<serde_json::Value>,
tools: Option<Vec<serde_json::Value>>,
#[serde(flatten)]
rest: serde_json::Map<String, serde_json::Value>,
}
/// Keys chat UIs attach to the payload for their own bookkeeping. They are not
/// part of the OpenAI request schema, and forwarding them to a strict upstream
/// (llama.cpp, vLLM, `cortiq serve`) is a needless way to earn a 400.
const CLIENT_PRIVATE_KEYS: &[&str] = &[
"chat_id",
"session_id",
"id",
"metadata",
"background_tasks",
"features",
"variables",
"model_item",
"tool_ids",
"filter_ids",
"files",
"citations",
"params",
"direct",
];
/// Strip client-private bookkeeping from the passthrough map, and drop a
/// non-string `user` (Open WebUI sends an object there for pipelines, which the
/// OpenAI schema does not allow).
fn sanitize_passthrough(
mut rest: serde_json::Map<String, serde_json::Value>,
) -> serde_json::Map<String, serde_json::Value> {
for k in CLIENT_PRIVATE_KEYS {
rest.remove(*k);
}
if rest.get("user").map(|u| !u.is_string()).unwrap_or(false) {
rest.remove("user");
}
rest
}
/// Parse the `model` field value into a routing directive.
/// `cortiq-auto[:profile]` → Auto; otherwise → Pinned(real id).
fn parse_routing(model: &str) -> RoutingDirective {
@@ -73,9 +168,43 @@ fn parse_routing(model: &str) -> RoutingDirective {
}
}
fn now_secs() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// Map whatever an upstream calls its stop reason onto the closed set the
/// OpenAI schema defines. Clients that validate the field (LiteLLM, the
/// official SDKs, several agent frameworks) reject anything else.
pub(crate) fn normalize_finish_reason(raw: &str) -> &'static str {
match raw.trim().to_ascii_lowercase().as_str() {
"length" | "max_tokens" | "max_new_tokens" | "token_limit" => "length",
"tool_calls" | "tool_use" | "function_call" => "tool_calls",
"content_filter" | "safety" => "content_filter",
_ => "stop",
}
}
/// A response id every client can key on. Upstreams that omit `id` (or send an
/// empty one) would otherwise leave the field blank, which some clients treat
/// as a malformed completion.
fn response_id(upstream: &str) -> String {
let trimmed = upstream.trim();
if !trimmed.is_empty() {
return trimmed.to_string();
}
format!("chatcmpl-{}", crate::admin::random_token(12))
}
async fn handler(
State(state): State<SharedState>,
Json(req): Json<OpenAiChatRequest>,
acct: Option<axum::Extension<super::AccountTag>>,
// Taken as raw bytes rather than `Json<T>`: axum's extractor rejection is a
// plain-text 422 that OpenAI clients cannot parse, and a client that cannot
// parse the failure has no answer to show — it just hangs.
body: axum::body::Bytes,
) -> Result<Response> {
// hot protocol toggle: if the adapter is disabled in config — return 404
if !state.live().cfg.protocols.openai_chat {
@@ -83,68 +212,95 @@ async fn handler(
"openai_chat protocol is disabled".into(),
));
}
let req: OpenAiChatRequest = serde_json::from_slice(&body)
.map_err(|e| GatewayError::InvalidRequest(format!("invalid request body: {e}")))?;
if req.messages.is_empty() {
return Err(GatewayError::InvalidRequest(
"messages must not be empty".into(),
));
}
// Echoed back verbatim: OpenAI returns the model the caller asked for, and
// clients (Open WebUI among them) match the reply against their own id. The
// model that actually answered travels in `cortiq.selected_model` and the
// `X-Cortiq-Selected-Model` header.
let requested_model = req
.model
.as_deref()
.map(str::trim)
.filter(|m| !m.is_empty())
.unwrap_or("cortiq-auto")
.to_string();
let stream = req.stream.unwrap_or(false);
let canonical = ChatRequest {
routing: parse_routing(&req.model),
routing: parse_routing(&requested_model),
messages: req.messages,
tools: req.tools,
tools: req.tools.unwrap_or_default(),
params: GenParams {
temperature: req.temperature,
max_tokens: req.max_tokens,
max_tokens: req.max_tokens.or(req.max_completion_tokens),
top_p: req.top_p,
think_budget: req.think_budget.or(req.reasoning_effort),
think_budget: req
.think_budget
.or_else(|| req.reasoning_effort.as_ref().and_then(Effort::to_budget)),
stop: Vec::new(),
passthrough: req.rest,
passthrough: sanitize_passthrough(req.rest),
},
stream: req.stream,
stream,
meta: RequestMeta {
protocol: "openai_chat".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
// streaming: forward provider SSE chunks verbatim with X-Cortiq-* headers
if req.stream {
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
let mut headers = cortiq_headers(&info);
headers.insert(
axum::http::header::CONTENT_TYPE,
axum::http::HeaderValue::from_static("text/event-stream"),
);
headers.insert(
axum::http::header::CACHE_CONTROL,
axum::http::HeaderValue::from_static("no-cache"),
);
return Ok((headers, axum::body::Body::from_stream(stream)).into_response());
// streaming: provider SSE is re-normalised into strict OpenAI chunks
if stream {
let (info, raw) = state.pipeline.run_stream(canonical, &state).await?;
let headers = sse_headers(&info);
let normalized = normalize_sse(raw, requested_model);
return Ok((headers, axum::body::Body::from_stream(normalized)).into_response());
}
let resp = state.pipeline.run(canonical, &state).await?;
let headers = cortiq_headers(&resp.cortiq);
let mut body = serde_json::json!({
"id": resp.id,
"object": "chat.completion",
"created": std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs(),
"model": resp.model_used,
"choices": resp.choices.iter().map(|c| {
let choices: Vec<serde_json::Value> = resp
.choices
.iter()
.map(|c| {
let mut message = serde_json::Map::new();
message.insert("role".into(), serde_json::json!(c.message.role));
message.insert("content".into(), serde_json::json!(c.message.content));
// Absent tool calls are omitted entirely — `"tool_calls": null` is
// not in the schema and trips strict clients.
if !c.message.tool_calls.is_empty() {
message.insert(
"tool_calls".into(),
serde_json::Value::Array(c.message.tool_calls.clone()),
);
}
let finish = if c.message.tool_calls.is_empty() {
normalize_finish_reason(&c.finish_reason)
} else {
"tool_calls"
};
serde_json::json!({
"index": c.index,
"message": {
"role": c.message.role,
"content": c.message.content,
"tool_calls": if c.message.tool_calls.is_empty() { serde_json::Value::Null } else { serde_json::Value::Array(c.message.tool_calls.clone()) },
},
"finish_reason": c.finish_reason,
"message": serde_json::Value::Object(message),
"finish_reason": finish,
})
}).collect::<Vec<_>>(),
})
.collect();
let mut body = serde_json::json!({
"id": response_id(&resp.id),
"object": "chat.completion",
"created": now_secs(),
"model": requested_model,
"choices": choices,
"usage": {
"prompt_tokens": resp.usage.prompt_tokens,
"completion_tokens": resp.usage.completion_tokens,
@@ -153,7 +309,7 @@ async fn handler(
});
if state.live().cfg.cortiq.echo {
body["cortiq"] = serde_json::json!({
let mut echo = serde_json::json!({
"task_label": resp.cortiq.task_label,
"complexity": {
"score": resp.cortiq.complexity_score,
@@ -161,10 +317,233 @@ async fn handler(
},
"selected_model": resp.cortiq.selected_model,
"route_source": resp.cortiq.route_source,
"router_request_id": resp.cortiq.router_request_id,
"cost_usd": resp.cortiq.cost_usd,
});
if let Some(rid) = &resp.cortiq.router_request_id {
echo["router_request_id"] = serde_json::json!(rid);
}
if resp.cortiq.tools_dropped {
echo["tools_dropped"] = serde_json::json!(true);
}
body["cortiq"] = echo;
}
Ok((headers, Json(body)).into_response())
}
/// Rewrite an upstream SSE stream into strict OpenAI `chat.completion.chunk`
/// events.
///
/// Every chunk is reissued with an id, `created`, and the *requested* model id;
/// `message`-shaped chunks are converted to `delta`; stop reasons are mapped to
/// the spec's closed set; empty deltas are dropped. The stream is guaranteed to
/// open with a `role` delta, to carry exactly one terminal `finish_reason`, and
/// to end with `data: [DONE]` — even when the upstream dies mid-flight, in
/// which case the failure is delivered as an SSE `error` event so the client
/// shows a message instead of waiting forever.
fn normalize_sse(
stream: crate::providers::ChatStream,
model: String,
) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
use futures::StreamExt;
async_stream::stream! {
let fallback_id = response_id("");
let created = now_secs();
let mut buf = String::new();
let mut id: Option<String> = None;
// Which choice indices already carried a `role` delta (n > 1 gets one each).
let mut roled: std::collections::HashSet<u64> = std::collections::HashSet::new();
let mut sent_finish = false;
let mut failure: Option<String> = None;
// Emit one normalised event; `None` means "nothing worth sending".
macro_rules! chunk {
($choices:expr, $extra:expr) => {{
let mut evt = serde_json::json!({
"id": id.clone().unwrap_or_else(|| fallback_id.clone()),
"object": "chat.completion.chunk",
"created": created,
"model": model,
"choices": $choices,
});
for (k, v) in $extra {
evt[k] = v;
}
bytes::Bytes::from(format!("data: {evt}\n\n"))
}};
}
futures::pin_mut!(stream);
'outer: while let Some(item) = stream.next().await {
let bytes = match item {
Ok(b) => b,
Err(e) => {
failure = Some(e.to_string());
break 'outer;
}
};
buf.push_str(&String::from_utf8_lossy(&bytes));
while let Some(idx) = buf.find("\n\n") {
let event: String = buf.drain(..idx + 2).collect();
for line in event.lines() {
let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
let data = data.trim();
if data.is_empty() || data == "[DONE]" {
continue;
}
let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
// An upstream error delivered inside the stream: pass it on
// and stop — the client needs to see it, not time out.
if let Some(err) = v.get("error").filter(|e| !e.is_null()) {
failure = Some(err.to_string());
break 'outer;
}
if id.is_none() {
id = v["id"].as_str().filter(|s| !s.is_empty()).map(str::to_string);
}
let mut choices = Vec::new();
if let Some(arr) = v["choices"].as_array() {
for (i, c) in arr.iter().enumerate() {
// Some upstreams send a full `message` per chunk
// instead of a `delta`; both mean the same thing here.
let src = if c["delta"].is_object() { &c["delta"] } else { &c["message"] };
let index = c["index"].as_u64().unwrap_or(i as u64);
let mut delta = serde_json::Map::new();
if roled.insert(index) {
delta.insert("role".into(), serde_json::json!("assistant"));
}
if let Some(text) = src["content"].as_str() {
if !text.is_empty() {
delta.insert("content".into(), serde_json::json!(text));
}
}
// Reasoning traces and citations are not in the base
// schema but every client that understands them
// ignores them safely, so they are worth keeping.
for key in ["reasoning_content", "reasoning", "thinking", "refusal", "annotations"] {
if let Some(val) = src.get(key).filter(|v| !v.is_null()) {
delta.insert(key.into(), val.clone());
}
}
if let Some(tc) = src["tool_calls"].as_array().filter(|a| !a.is_empty()) {
delta.insert("tool_calls".into(), serde_json::Value::Array(tc.clone()));
}
let finish = c["finish_reason"].as_str().map(normalize_finish_reason);
if finish.is_none() && delta.is_empty() {
continue;
}
let mut choice = serde_json::json!({
"index": index,
"delta": serde_json::Value::Object(delta),
});
if let Some(f) = finish {
choice["finish_reason"] = serde_json::json!(f);
sent_finish = true;
}
choices.push(choice);
}
}
// Terminal metadata rides on a choice-less chunk, exactly
// as OpenAI ships `usage` when `stream_options` asks for it.
let mut extra: Vec<(&str, serde_json::Value)> = Vec::new();
if let Some(u) = v.get("usage").filter(|u| !u.is_null()) {
extra.push(("usage", u.clone()));
}
for key in ["cortiq_cost_usd", "cortiq_estimated"] {
if let Some(val) = v.get(key).filter(|v| !v.is_null()) {
extra.push((key, val.clone()));
}
}
if !choices.is_empty() || !extra.is_empty() {
yield Ok(chunk!(choices, extra));
}
}
}
}
if let Some(message) = failure {
let evt = serde_json::json!({
"error": { "message": message, "type": "upstream_unavailable", "code": "stream_failed" }
});
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
}
// Close the stream properly whatever the upstream did: a client that
// never sees a stop reason and a [DONE] keeps its spinner turning.
if !sent_finish {
let mut delta = serde_json::Map::new();
if roled.insert(0) {
delta.insert("role".into(), serde_json::json!("assistant"));
}
let choices = vec![serde_json::json!({
"index": 0,
"delta": serde_json::Value::Object(delta),
"finish_reason": "stop",
})];
let extra: Vec<(&str, serde_json::Value)> = Vec::new();
yield Ok(chunk!(choices, extra));
}
yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn content_parts_and_null_are_accepted() {
let body = serde_json::json!({
"model": "cortiq-auto",
"messages": [
{"role": "user", "content": [{"type": "text", "text": "hello "}, {"type": "image_url", "image_url": {"url": "x"}}, {"type": "text", "text": "world"}]},
{"role": "assistant", "content": null},
{"role": "tool", "content": "42", "tool_call_id": "call_1"}
],
"tools": null,
"reasoning_effort": "high",
"chat_id": "owui-123"
});
let req: OpenAiChatRequest = serde_json::from_value(body).expect("must parse");
assert_eq!(req.messages[0].content, "hello world");
assert_eq!(req.messages[1].content, "");
assert_eq!(req.messages[2].tool_call_id.as_deref(), Some("call_1"));
assert!(req.tools.unwrap_or_default().is_empty());
assert_eq!(
req.reasoning_effort.as_ref().and_then(Effort::to_budget),
Some(8192)
);
assert!(!sanitize_passthrough(req.rest).contains_key("chat_id"));
}
#[test]
fn finish_reasons_map_to_the_openai_set() {
assert_eq!(normalize_finish_reason("max_tokens"), "length");
assert_eq!(normalize_finish_reason("eos"), "stop");
assert_eq!(normalize_finish_reason("tool_use"), "tool_calls");
}
#[tokio::test]
async fn stream_is_closed_even_when_the_upstream_stops_early() {
use futures::StreamExt;
let upstream = futures::stream::iter(vec![Ok(bytes::Bytes::from(
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"}}]}\n\n",
))]);
let out: Vec<String> = normalize_sse(Box::pin(upstream), "cortiq-auto".into())
.map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
.collect()
.await;
let joined = out.join("");
assert!(joined.contains("\"role\":\"assistant\""));
assert!(joined.contains("\"model\":\"cortiq-auto\""));
assert!(joined.contains("\"finish_reason\":\"stop\""));
assert!(joined.trim_end().ends_with("data: [DONE]"));
}
}
+14 -13
View File
@@ -4,7 +4,7 @@
use crate::error::{GatewayError, Result};
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
use crate::protocols::openai_chat::cortiq_headers;
use crate::protocols::openai_chat::{cortiq_headers, normalize_finish_reason, sse_headers};
use crate::state::SharedState;
use axum::response::{IntoResponse, Response};
use axum::{extract::State, routing::post, Json, Router};
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
async fn handler(
State(state): State<SharedState>,
acct: Option<axum::Extension<super::AccountTag>>,
Json(req): Json<CompletionsRequest>,
) -> Result<Response> {
if !state.live().cfg.protocols.openai_completions {
@@ -70,12 +71,15 @@ async fn handler(
));
}
// Echoed back as-is: clients match the reply against the id they asked for.
let requested_model = req.model.clone();
let canonical = ChatRequest {
routing: parse_routing(&req.model),
messages: vec![Message {
role: "user".into(),
content: prompt,
tool_calls: vec![],
..Default::default()
}],
tools: vec![],
params: GenParams {
@@ -89,19 +93,15 @@ async fn handler(
stream: req.stream,
meta: RequestMeta {
protocol: "openai_completions".into(),
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
..Default::default()
},
};
if req.stream {
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
let mut headers = cortiq_headers(&info);
headers.insert(
axum::http::header::CONTENT_TYPE,
axum::http::HeaderValue::from_static("text/event-stream"),
);
return Ok((
headers,
sse_headers(&info),
axum::body::Body::from_stream(chat_to_completions_sse(stream)),
)
.into_response());
@@ -113,11 +113,12 @@ async fn handler(
.first()
.map(|c| c.message.content.clone())
.unwrap_or_default();
let finish = resp
.choices
.first()
.map(|c| c.finish_reason.clone())
.unwrap_or_else(|| "stop".into());
let finish = normalize_finish_reason(
resp.choices
.first()
.map(|c| c.finish_reason.as_str())
.unwrap_or("stop"),
);
let headers = cortiq_headers(&resp.cortiq);
let body = serde_json::json!({
"id": resp.id,
@@ -126,7 +127,7 @@ async fn handler(
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs(),
"model": resp.model_used,
"model": requested_model,
"choices": [{ "text": answer, "index": 0, "logprobs": serde_json::Value::Null, "finish_reason": finish }],
"usage": {
"prompt_tokens": resp.usage.prompt_tokens,
+47 -1
View File
@@ -1,5 +1,10 @@
//! Inbound OpenAI Models adapter: `GET /v1/models`.
//! Lists the configured pool (plus the virtual `cortiq-auto`) for client discovery.
//! Lists everything a client can address: the virtual `cortiq-auto`, the
//! configured `[[models]]` pool, and the managed local `[[cmf.servers]]`.
//!
//! The managed CMF servers belong here because they are routable model ids like
//! any other — omitting them left OpenAI clients (Open WebUI among them) unable
//! to pin a local model even though the gateway would happily serve it.
use crate::error::{GatewayError, Result};
use crate::state::SharedState;
@@ -9,6 +14,13 @@ pub fn routes() -> Router<SharedState> {
Router::new().route("/v1/models", get(handler))
}
fn now_secs() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse> {
let live = state.live();
if !live.cfg.protocols.openai_models {
@@ -16,17 +28,51 @@ async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse>
"openai_models protocol is disabled".into(),
));
}
let created = now_secs();
let mut data = vec![serde_json::json!({
"id": "cortiq-auto",
"object": "model",
"created": created,
"owned_by": "cortiq",
})];
let mut seen: std::collections::HashSet<String> =
std::collections::HashSet::from(["cortiq-auto".to_string()]);
for m in &live.cfg.models {
if !seen.insert(m.id.clone()) {
continue;
}
data.push(serde_json::json!({
"id": m.id,
"object": "model",
"created": created,
"owned_by": m.provider,
}));
}
// Managed local CMF servers. Health is reported alongside (a non-standard
// field clients ignore) rather than used as a filter: a large `.cmf` takes
// minutes to load, and a model that vanishes from the list while it warms up
// is worse than one marked "not ready yet".
let cmf = state.cmf.status();
for s in live.cfg.cmf.effective_servers() {
if !seen.insert(s.id.clone()) {
continue; // a static [[models]] entry with the same id wins
}
let status = cmf.servers.iter().find(|st| st.id == s.id);
data.push(serde_json::json!({
"id": s.id,
"object": "model",
"created": created,
"owned_by": "cortiq-cmf",
"cortiq": {
"kind": "cmf",
"local": true,
"running": status.map(|st| st.running).unwrap_or(false),
"healthy": status.map(|st| st.healthy).unwrap_or(false),
},
}));
}
Ok(Json(serde_json::json!({ "object": "list", "data": data })))
}
+9 -1
View File
@@ -32,7 +32,13 @@ impl AnthropicProvider {
caps: Caps::from_list(&cfg.caps),
price_in: cfg.price_in,
price_out: cfg.price_out,
http: reqwest::Client::new(),
// connect timeout: a dead host fails over in seconds instead of
// hanging the request; NO overall timeout — that would cut long
// generations and streams (OpenRouter can be slow legitimately)
http: reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_else(|_| reqwest::Client::new()),
})
}
@@ -140,6 +146,7 @@ impl Provider for AnthropicProvider {
role: "assistant".into(),
content,
tool_calls: vec![],
..Default::default()
},
finish_reason: v["stop_reason"].as_str().unwrap_or("stop").to_string(),
}],
@@ -157,6 +164,7 @@ impl Provider for AnthropicProvider {
router_request_id: None,
cost_usd: 0.0,
failover: false,
tools_dropped: false,
},
})
}
+9 -7
View File
@@ -28,7 +28,13 @@ impl OpenAiProvider {
caps: Caps::from_list(&cfg.caps),
price_in: cfg.price_in,
price_out: cfg.price_out,
http: reqwest::Client::new(),
// connect timeout: a dead host fails over in seconds instead of
// hanging the request; NO overall timeout — that would cut long
// generations and streams (OpenRouter can be slow legitimately)
http: reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_else(|_| reqwest::Client::new()),
})
}
@@ -48,13 +54,11 @@ impl OpenAiProvider {
if let Some(m) = req.params.max_tokens {
body["max_tokens"] = m.into();
}
eprintln!("[to_wire] think_budget={:?}", req.params.think_budget);
if let Some(tb) = req.params.think_budget {
body["think_budget"] = tb.into();
body["reasoning_effort"] = tb.into();
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
body["enable_thinking"] = (tb > 0).into();
eprintln!("[to_wire] SET enable_thinking={}", tb > 0);
}
if !req.tools.is_empty() {
body["tools"] = serde_json::Value::Array(req.tools.clone());
@@ -63,10 +67,6 @@ impl OpenAiProvider {
for (k, v) in &req.params.passthrough {
body[k] = v.clone();
}
eprintln!(
"[to_wire] enable_thinking={:?}",
body.get("enable_thinking")
);
body
}
}
@@ -135,6 +135,7 @@ impl Provider for OpenAiProvider {
role,
content,
tool_calls,
..Default::default()
},
finish_reason: c["finish_reason"].as_str().unwrap_or("stop").to_string(),
});
@@ -166,6 +167,7 @@ impl Provider for OpenAiProvider {
router_request_id: None,
cost_usd: 0.0,
failover: false,
tools_dropped: false,
},
})
}
+1 -1
View File
@@ -48,7 +48,7 @@ impl Registry {
price_out: 0.0,
kind: "chat".into(),
api_key_env: None,
caps: Vec::new(),
caps: s.caps.clone(),
temperature: s.temperature,
top_p: s.top_p,
max_tokens: s.max_tokens,
+17
View File
@@ -71,6 +71,23 @@ impl RouterClient {
/// Classify text. Returns `Ok(None)` on network error/timeout —
/// the caller treats this as "router unavailable" and uses the default tier.
/// `GET /v1/taxonomies` — the account's taxonomies with their label sets.
/// `Ok(None)` when the router is disabled or has no key configured.
pub async fn taxonomies(&self) -> anyhow::Result<Option<serde_json::Value>> {
if self.url.is_empty() {
return Ok(None);
}
let mut req = self.http.get(format!("{}/v1/taxonomies", self.url));
if let Some(k) = &self.api_key {
req = req.bearer_auth(k);
}
let resp = req.send().await?;
if !resp.status().is_success() {
anyhow::bail!("router /v1/taxonomies: HTTP {}", resp.status());
}
Ok(Some(resp.json().await?))
}
pub async fn route(&self, text: &str, profile: &str) -> anyhow::Result<Option<RouteDecision>> {
let mut body = serde_json::json!({
"input": { "text": text },
+212 -17
View File
@@ -3,38 +3,117 @@
use crate::config::{Config, TierTargets};
fn to_list(t: &TierTargets) -> Vec<String> {
t.models()
}
/// A task rule, normalized: per-tier model lists ("which model serves this
/// task at each complexity") plus any-tier models that lead regardless.
#[derive(Clone, Default)]
pub struct TaskRule {
pub any: Vec<String>,
pub per: std::collections::HashMap<String, Vec<String>>,
}
pub struct RoutingTable {
tiers: std::collections::HashMap<String, Vec<String>>,
/// Per-task rules — a matching rule's models lead the candidate chain.
tasks: std::collections::HashMap<String, TaskRule>,
default: String,
}
impl RoutingTable {
pub fn from_config(cfg: &Config) -> Self {
let tiers = cfg
.routing
.tiers
.iter()
.map(|(tier, targets)| {
let list = match targets {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
(tier.clone(), list)
})
.collect();
Self {
tiers,
tiers: cfg
.routing
.tiers
.iter()
.map(|(tier, targets)| (tier.clone(), to_list(targets)))
.collect(),
tasks: cfg
.routing
.tasks
.iter()
.map(|(task, targets)| {
let (any, per) = targets.task_rule();
(task.clone(), TaskRule { any, per })
})
.filter(|(_, r)| !r.any.is_empty() || r.per.values().any(|v| !v.is_empty()))
.collect(),
default: cfg.routing.default.clone(),
}
}
/// Candidate chain for `(task, tier)` and whether a task rule shaped it.
///
/// Order of preference:
/// 1. the task's models FOR THIS TIER (the user said exactly which model
/// serves this task at this complexity);
/// 2. at tier=high — the global high chain (escalation: any-tier task
/// models must not undercut it), then the task's any-tier models;
/// otherwise the task's any-tier models, then the global tier chain;
/// 3. `default` as the last resort. Deduplicated, empties dropped.
pub fn candidates_for(&self, task: &str, tier: &str) -> (Vec<String>, bool) {
fn push(out: &mut Vec<String>, id: &str) {
if !id.is_empty() && !out.iter().any(|x| x == id) {
out.push(id.to_string());
}
}
let mut out: Vec<String> = Vec::new();
let rule = self.tasks.get(task);
let tier_chain = self
.tiers
.get(tier)
.cloned()
.unwrap_or_else(|| vec![self.default.clone()]);
if let Some(r) = rule {
if let Some(list) = r.per.get(tier) {
for id in list {
push(&mut out, id);
}
}
}
let escalate = tier == "high" && self.tiers.contains_key("high");
if escalate {
// high verdicts escalate over any-tier task pins
for id in &tier_chain {
push(&mut out, id);
}
if let Some(r) = rule {
for id in &r.any {
push(&mut out, id);
}
}
} else {
if let Some(r) = rule {
for id in &r.any {
push(&mut out, id);
}
}
for id in &tier_chain {
push(&mut out, id);
}
}
let d = self.default.clone();
push(&mut out, &d);
// "task-rule" must mean "the rule chose the head of this chain" — a
// rule that exists but lost to the tier chain (math has a high pick,
// the request came in low) must not claim the credit.
let matched = match (rule, out.first()) {
(Some(r), Some(head)) => {
r.per.get(tier).map(|v| v.contains(head)).unwrap_or(false)
|| r.any.first() == Some(head)
}
_ => false,
};
(out, matched)
}
/// Candidates for the given complexity tier, in preference order.
/// If the tier is not defined, the sole candidate is `default`.
pub fn candidates(&self, tier: &str) -> Vec<String> {
self.tiers
.get(tier)
.cloned()
.unwrap_or_else(|| vec![self.default.clone()])
self.candidates_for("", tier).0
}
/// Default model (used when the router is unavailable).
@@ -46,3 +125,119 @@ impl RoutingTable {
// TODO(v0.2): cost_aware mode and circuit breaker — pick the cheapest available
// model no lower than min_class[tier], respecting max_cost_usd_per_request,
// and skip models whose circuit breaker is open.
#[cfg(test)]
mod tests {
use super::*;
fn table(tasks: &[(&str, &[&str])], tiers: &[(&str, &[&str])], default: &str) -> RoutingTable {
RoutingTable {
tiers: tiers
.iter()
.map(|(k, v)| (k.to_string(), v.iter().map(|s| s.to_string()).collect()))
.collect(),
tasks: tasks
.iter()
.map(|(k, v)| {
(
k.to_string(),
TaskRule {
any: v.iter().map(|s| s.to_string()).collect(),
per: Default::default(),
},
)
})
.collect(),
default: default.to_string(),
}
}
fn with_task_tier_models(
mut t: RoutingTable,
task: &str,
tier: &str,
models: &[&str],
) -> RoutingTable {
t.tasks.entry(task.to_string()).or_default().per.insert(
tier.to_string(),
models.iter().map(|s| s.to_string()).collect(),
);
t
}
#[test]
fn task_rule_leads_tier_chain_follows() {
let t = table(
&[("translation", &["local"])],
&[("low", &["cheap", "local"]), ("high", &["big"])],
"cheap",
);
// dedup: rule model already in the tier chain appears once, first
let (c, matched) = t.candidates_for("translation", "low");
assert!(matched);
assert_eq!(c, vec!["local", "cheap"]);
}
#[test]
fn high_tier_escalates_over_task_rule() {
let t = table(
&[("code", &["cheap-coder"])],
&[("low", &["cheap"]), ("high", &["big"])],
"cheap",
);
// medium (no chain defined → default): task rule still leads
let (c, m) = t.candidates_for("code", "medium");
assert!(m);
assert_eq!(c[0], "cheap-coder");
// high: tier chain leads, task rule follows as fallback
let (c, m) = t.candidates_for("code", "high");
// escalation put the global high chain first → the rule did not pick
// the head, so no task-rule credit
assert!(!m);
// default rides along as the very last fallback
assert_eq!(c, vec!["big", "cheap-coder", "cheap"]);
}
#[test]
fn per_tier_models_lead_their_tier() {
let base = table(
&[("code", &["generic-coder"])],
&[("low", &["cheap"]), ("high", &["big"])],
"cheap",
);
let t = with_task_tier_models(base, "code", "high", &["best-coder"]);
// at high: the task's HIGH pick leads, escalation chain next,
// any-tier task models after it, default last
let (c, m) = t.candidates_for("code", "high");
assert!(m);
assert_eq!(c, vec!["best-coder", "big", "generic-coder", "cheap"]);
// at low: any-tier task models lead as before (and the badge is
// honest — the head came from the rule)
let (c, m) = t.candidates_for("code", "low");
assert!(m);
assert_eq!(c, vec!["generic-coder", "cheap"]);
// a tier with an explicit pick but no any-models
let t2 = with_task_tier_models(
table(&[], &[("low", &["cheap"]), ("high", &["big"])], "cheap"),
"legal",
"high",
&["strong"],
);
let (c, m) = t2.candidates_for("legal", "high");
assert!(m);
assert_eq!(c, vec!["strong", "big", "cheap"]);
// the same rule at LOW did not shape the head → no task-rule credit
let (c, m) = t2.candidates_for("legal", "low");
assert!(!m);
assert_eq!(c, vec!["cheap"]);
}
#[test]
fn no_rule_falls_back_to_tier_then_default() {
let t = table(&[], &[("low", &["a"])], "d");
assert_eq!(t.candidates_for("qa", "low").0, vec!["a", "d"]);
let (c, matched) = t.candidates_for("qa", "unknown-tier");
assert!(!matched);
assert_eq!(c, vec!["d"]);
}
}
+79
View File
@@ -66,14 +66,63 @@ impl Live {
}
}
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
/// fine for office-grade fairness and costs one HashMap lookup per request.
#[derive(Default)]
pub struct RateWindows {
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
}
impl RateWindows {
pub fn allow(&self, key: &str, per_min: u32) -> bool {
let minute = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
/ 60;
let mut g = self.inner.lock().unwrap();
let e = g.entry(key.to_string()).or_insert((minute, 0));
if e.0 != minute {
*e = (minute, 0);
}
if e.1 >= per_min {
return false;
}
e.1 += 1;
true
}
/// Seconds until the current minute window rolls over.
pub fn retry_after_secs(&self) -> u64 {
60 - (std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
% 60)
}
}
pub struct AppState {
pub live: ArcSwap<Live>,
pub stats: Arc<Stats>,
pub promotion: Arc<crate::promotion::Promotion>,
pub imports: Arc<crate::import::JobStore>,
/// Image/video generation jobs (cortiq imagine / animate).
pub media: Arc<crate::media::MediaStore>,
/// One-click local model benchmarks (cortiq bench --json).
pub bench: Arc<crate::bench::BenchStore>,
/// Host metrics sampler (CPU / RAM / disk) for the dashboard.
pub sysmon: Arc<crate::sysmon::SysMon>,
/// Newest cortiq-gateway version on crates.io, when newer than ours.
pub gateway_update: Arc<std::sync::Mutex<Option<String>>>,
/// Admin console reachable from other machines without a token — the
/// footgun banner flag (set at startup, shown by the SPA).
pub admin_open_public: std::sync::atomic::AtomicBool,
/// Managed local CMF model server (install/update/spawn lifecycle + status).
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
pub cache: Arc<SemanticCache>,
/// Per-API-key fixed-window rate limiter.
pub rate: RateWindows,
pub secrets: SecretStore,
pub config_path: String,
pub pipeline: crate::pipeline::Pipeline,
@@ -113,6 +162,30 @@ impl SharedState {
let stats = Stats::new(&cfg.stats);
let promotion = crate::promotion::Promotion::new(promotion_cfg(&cfg.shadow));
let imports = crate::import::JobStore::new();
let media = crate::media::MediaStore::new();
let bench = crate::bench::BenchStore::new();
// jobs survive restarts: files under <data>/, reconciled on load
let data = crate::config::data_dir();
let _ = std::fs::create_dir_all(&data);
imports.attach_persistence(data.join("jobs-import.json"));
media.attach_persistence(data.join("jobs-media.json"));
let sysmon = crate::sysmon::spawn(cfg.cmf.models_dir.clone());
let gateway_update: Arc<std::sync::Mutex<Option<String>>> =
Arc::new(std::sync::Mutex::new(None));
{
// check crates.io for a newer gateway once at start and twice a day
let slot = gateway_update.clone();
tokio::spawn(async move {
loop {
if let Some(latest) = crate::cmf_runtime::latest_gateway_version().await {
let newer =
crate::cmf_runtime::version_lt(env!("CARGO_PKG_VERSION"), &latest);
*slot.lock().unwrap() = newer.then_some(latest);
}
tokio::time::sleep(std::time::Duration::from_secs(12 * 3600)).await;
}
});
}
let cmf = crate::cmf_runtime::CmfRuntime::new();
let cache = SemanticCache::new(&cfg.cache);
let router_status = RouterLastStatus::default();
@@ -122,8 +195,14 @@ impl SharedState {
stats,
promotion,
imports,
media,
bench,
sysmon,
gateway_update,
admin_open_public: std::sync::atomic::AtomicBool::new(false),
cmf,
cache,
rate: RateWindows::default(),
secrets,
config_path,
pipeline: crate::pipeline::Pipeline::new(),
+257 -1
View File
@@ -55,7 +55,7 @@ pub struct RequestRecord {
pub error: Option<String>,
}
#[derive(Clone, Debug, Default, Serialize)]
#[derive(Clone, Debug, Default, Serialize, serde::Deserialize)]
pub struct Totals {
pub requests: u64,
pub ok: u64,
@@ -130,6 +130,20 @@ struct Inner {
/// Per-minute buckets — the single source for the windowed admin snapshot.
buckets: VecDeque<Bucket>,
recent: VecDeque<RequestRecord>,
/// Per-account totals over EVERY record still in the JSONL file (no
/// retention gate) — per-key accounting for the keys page.
by_account_alltime: HashMap<String, Totals>,
/// Per-account totals folded out of the file by compaction. alltime(key)
/// = folded + by_account_alltime, with no double counting: a record is
/// either in the file or folded, never both.
folded: HashMap<String, Totals>,
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
month_spend: HashMap<String, (String, f64)>,
/// Records with ts ≤ this are already reflected in the persisted
/// month_spend (watermark against double counting on replay).
month_upto_ts: u64,
/// Appends since the last compaction size check.
writes_since_check: u32,
}
pub struct Stats {
@@ -137,6 +151,7 @@ pub struct Stats {
file: Option<String>,
ring_size: usize,
retention_secs: u64,
max_file_bytes: u64,
inner: Mutex<Inner>,
}
@@ -152,9 +167,12 @@ impl Stats {
file,
ring_size: cfg.ring_size.max(1),
retention_secs: parse_duration_secs(&cfg.retention).unwrap_or(7 * 86_400),
max_file_bytes: cfg.max_file_mb.saturating_mul(1024 * 1024),
inner: Mutex::new(Inner::default()),
};
s.load_folded();
s.replay();
s.maybe_compact(true);
std::sync::Arc::new(s)
}
@@ -175,6 +193,12 @@ impl Stats {
// push everything to the "recent" ring (it self-limits by size),
// but apply to aggregates/series only entries within the window.
Self::push_recent(&mut inner, &rec, self.ring_size);
inner
.by_account_alltime
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
if rec.ts >= cutoff {
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
}
@@ -182,6 +206,33 @@ impl Stats {
}
}
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
return;
}
let m = month_of(rec.ts);
let e = inner
.month_spend
.entry(rec.account.clone())
.or_insert((m.clone(), 0.0));
if e.0 != m {
*e = (m, 0.0);
}
e.1 += rec.cost_usd;
}
/// Month-to-date spend for an account (budget checks).
pub fn month_spend(&self, account: &str) -> f64 {
let inner = self.inner.lock().unwrap();
let m = month_of(now_secs());
inner
.month_spend
.get(account)
.filter(|(mm, _)| *mm == m)
.map(|(_, v)| *v)
.unwrap_or(0.0)
}
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
inner.recent.push_back(rec.clone());
while inner.recent.len() > ring_size {
@@ -233,6 +284,13 @@ impl Stats {
let mut inner = self.inner.lock().unwrap();
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
Self::push_recent(&mut inner, &rec, self.ring_size);
inner
.by_account_alltime
.entry(rec.account.clone())
.or_default()
.apply(&rec);
Self::apply_month(&mut inner, &rec);
inner.writes_since_check += 1;
}
if let Some(path) = &self.file {
if let Ok(line) = serde_json::to_string(&rec) {
@@ -245,6 +303,141 @@ impl Stats {
}
}
}
self.maybe_compact(false);
}
fn totals_path(&self) -> Option<String> {
self.file.as_ref().map(|f| format!("{f}.totals.json"))
}
fn load_folded(&self) {
let Some(path) = self.totals_path() else {
return;
};
let Ok(content) = std::fs::read_to_string(&path) else {
return;
};
#[derive(serde::Deserialize)]
struct Sidecar {
#[serde(default)]
by_account: HashMap<String, Totals>,
#[serde(default)]
month_spend: HashMap<String, (String, f64)>,
#[serde(default)]
month_upto_ts: u64,
}
// new format {by_account, month_spend, month_upto_ts}; the original
// file held the bare by_account map — accept both
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
let mut inner = self.inner.lock().unwrap();
inner.folded = sc.by_account;
inner.month_spend = sc.month_spend;
inner.month_upto_ts = sc.month_upto_ts;
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
self.inner.lock().unwrap().folded = m;
}
}
fn write_sidecar(&self, inner: &Inner) {
let Some(tpath) = self.totals_path() else {
return;
};
let payload = serde_json::json!({
"by_account": inner.folded,
"month_spend": inner.month_spend,
"month_upto_ts": now_secs(),
});
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, payload.to_string()).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
/// older than the retention window fold into the cumulative per-account
/// sidecar and are dropped from the file (tmp + rename on both files, so
/// a crash mid-compaction loses nothing). `force` checks size regardless
/// of the write counter (startup).
fn maybe_compact(&self, force: bool) {
if self.max_file_bytes == 0 {
return;
}
let Some(path) = &self.file else { return };
{
let mut inner = self.inner.lock().unwrap();
if !force {
if inner.writes_since_check < 256 {
return;
}
inner.writes_since_check = 0;
// piggyback: persist month-to-date spend so budgets survive
// restarts even between compactions
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
}
}
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
if size <= self.max_file_bytes {
return;
}
let Ok(content) = std::fs::read_to_string(path) else {
return;
};
let cutoff = now_secs().saturating_sub(self.retention_secs);
let mut kept = String::with_capacity(content.len() / 2);
let mut inner = self.inner.lock().unwrap();
for line in content.lines() {
let line = line.trim();
if line.is_empty() {
continue;
}
match serde_json::from_str::<RequestRecord>(line) {
Ok(rec) if rec.ts < cutoff => {
inner
.folded
.entry(rec.account.clone())
.or_default()
.apply(&rec);
// fold = leave by_account_alltime (record leaves the file,
// its share moves from "alltime" into "folded")
if let Some(t) = inner.by_account_alltime.get_mut(&rec.account) {
// subtract by rebuilding is costly; instead track the
// invariant additively: remove the record's share
t.requests = t.requests.saturating_sub(1);
if rec.outcome == "ok" {
t.ok = t.ok.saturating_sub(1);
} else {
t.errors = t.errors.saturating_sub(1);
}
if rec.failover {
t.failovers = t.failovers.saturating_sub(1);
}
t.prompt_tokens = t.prompt_tokens.saturating_sub(rec.prompt_tokens as u64);
t.completion_tokens = t
.completion_tokens
.saturating_sub(rec.completion_tokens as u64);
t.cost_usd = (t.cost_usd - rec.cost_usd).max(0.0);
t.latency_ms_sum = t.latency_ms_sum.saturating_sub(rec.latency_ms);
}
}
Ok(_) => {
kept.push_str(line);
kept.push('\n');
}
Err(_) => { /* unparseable line — drop */ }
}
}
// persist the folded totals FIRST, then swap the trimmed file in
inner.month_upto_ts = now_secs();
self.write_sidecar(&inner);
let tmp = format!("{path}.tmp");
if std::fs::write(&tmp, kept).is_ok() {
let _ = std::fs::rename(&tmp, path);
}
tracing::info!(
new_size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0),
"stats log compacted (older records folded into cumulative totals)"
);
}
/// Snapshot for the admin API: aggregates + time series for `range_secs` + breakdown by `groupby`.
@@ -303,7 +496,27 @@ impl Stats {
.cmp(&a["requests"].as_u64().unwrap_or(0))
});
let mut alltime: HashMap<String, Totals> = inner.folded.clone();
for (k, t) in &inner.by_account_alltime {
alltime.entry(k.clone()).or_default().merge(t);
}
let alltime_by_account: serde_json::Map<String, serde_json::Value> = alltime
.iter()
.map(|(k, t)| {
(
k.clone(),
serde_json::json!({
"requests": t.requests,
"prompt_tokens": t.prompt_tokens,
"completion_tokens": t.completion_tokens,
"cost_usd": t.cost_usd,
}),
)
})
.collect();
serde_json::json!({
"alltime_by_account": alltime_by_account,
"totals": {
"requests": win.requests,
"ok": win.ok,
@@ -327,6 +540,9 @@ impl Stats {
/// Reset all in-memory stats and truncate the JSONL log — the "clear logs"
/// admin action.
pub fn clear(&self) {
if let Some(tpath) = self.totals_path() {
let _ = std::fs::remove_file(tpath);
}
{
let mut inner = self.inner.lock().unwrap();
*inner = Inner::default();
@@ -416,6 +632,46 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
}
fn month_of(ts: u64) -> String {
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
let days = ts / 86_400;
let mut year = 1970u64;
let mut d = days;
loop {
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let len = if leap { 366 } else { 365 };
if d < len {
break;
}
d -= len;
year += 1;
}
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
let ml = [
31,
if leap { 29 } else { 28 },
31,
30,
31,
30,
31,
31,
30,
31,
30,
31,
];
let mut month = 1;
for len in ml {
if d < len {
break;
}
d -= len;
month += 1;
}
format!("{year:04}-{month:02}")
}
fn avg(sum: u64, n: u64) -> u64 {
sum.checked_div(n).unwrap_or(0)
}
+86
View File
@@ -0,0 +1,86 @@
//! Host metrics sampler for the admin dashboard: CPU load, RAM, free disk at
//! the models dir, and the gateway's own RSS. One background task refreshes a
//! snapshot every 2 s; the API endpoint only reads it.
use serde::Serialize;
use std::sync::{Arc, Mutex};
#[derive(Clone, Default, Serialize)]
pub struct Snapshot {
pub ts: u64,
/// Whole-machine CPU usage 0..100 (average over all cores).
pub cpu_percent: f32,
pub cores: usize,
pub load1: f64,
pub ram_total: u64,
pub ram_used: u64,
/// Disk that holds models_dir.
pub disk_total: u64,
pub disk_free: u64,
/// Resident memory of the gateway process itself.
pub process_rss: u64,
}
pub struct SysMon {
snap: Mutex<Snapshot>,
}
impl SysMon {
pub fn snapshot(&self) -> Snapshot {
self.snap.lock().unwrap().clone()
}
}
fn now() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
}
/// Start the sampler. `models_dir` picks which disk's free space is reported.
pub fn spawn(models_dir: String) -> Arc<SysMon> {
let mon = Arc::new(SysMon {
snap: Mutex::new(Snapshot::default()),
});
let ret = mon.clone();
tokio::task::spawn_blocking(move || {
use sysinfo::{Disks, ProcessesToUpdate, System};
let mut sys = System::new();
let pid = sysinfo::Pid::from_u32(std::process::id());
let models_abs = std::fs::canonicalize(&models_dir)
.unwrap_or_else(|_| std::path::PathBuf::from(&models_dir));
loop {
sys.refresh_cpu_usage();
sys.refresh_memory();
let _ = sys.refresh_processes(ProcessesToUpdate::Some(&[pid]), true);
let disks = Disks::new_with_refreshed_list();
// the disk whose mount point is the longest prefix of models_dir
let mut best: Option<(usize, u64, u64)> = None;
for d in disks.list() {
let mp = d.mount_point();
if models_abs.starts_with(mp) {
let len = mp.as_os_str().len();
if best.map(|(l, _, _)| len > l).unwrap_or(true) {
best = Some((len, d.total_space(), d.available_space()));
}
}
}
let (disk_total, disk_free) = best.map(|(_, t, f)| (t, f)).unwrap_or((0, 0));
let snap = Snapshot {
ts: now(),
cpu_percent: sys.global_cpu_usage(),
cores: sys.cpus().len(),
load1: System::load_average().one,
ram_total: sys.total_memory(),
ram_used: sys.used_memory(),
disk_total,
disk_free,
process_rss: sys.process(pid).map(|p| p.memory()).unwrap_or(0),
};
*mon.snap.lock().unwrap() = snap;
std::thread::sleep(std::time::Duration::from_secs(2));
}
});
ret
}
+117
View File
@@ -585,6 +585,118 @@ body.modal-open { overflow: hidden; }
/* Featured gallery: the list grows as new models are published, so give it more
room than the search grid before it starts scrolling. */
.featured-section .hf-grid { max-height: min(62vh, 760px); }
/* Money: 4 decimals, last two de-emphasized */
.money-tail { font-size: .72em; opacity: .75; margin-left: 1px; }
.seg { flex-wrap: wrap; }
.ok { color: var(--good); }
.pairs { display: flex; flex-direction: column; gap: 6px; align-items: flex-start; }
.pair-row { display: flex; gap: 6px; align-items: center; }
.pair-model { min-width: 220px; max-width: 340px; padding: 4px 8px; font-size: 12.5px; }
.pair-tier { max-width: 150px; padding: 4px 8px; font-size: 12.5px; }
/* Complexity bands slider */
.bands-wrap { margin: 4px 0 16px; }
/* one-line command chip (install commands etc.) — theme-aware */
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
.bands-track.disabled { opacity: .45; pointer-events: none; }
.bands-zone { position: absolute; top: 0; bottom: 0; }
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
.bands-handle:active { cursor: grabbing; }
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
.bands-val.raised { top: -46px; }
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
/* Benchmark view */
@keyframes bm-in { from { opacity: 0; transform: translateY(-3px); } to { opacity: 1; transform: none; } }
tbody tr:last-child { animation: bm-in .25s ease; }
.bm-sum { margin-bottom: 10px; }
.bm-cell { cursor: pointer; white-space: nowrap; }
.bm-cell:hover { background: var(--hover); }
.badge.ok { background: rgba(52,211,153,.15); color: var(--good); }
/* Routing: per-task rule rows */
.taskrow { align-items: flex-start; }
.taskname { min-width: 190px; line-height: 1.35; }
.taskname .mono { font-size: 10.5px; opacity: .75; }
.chips-empty { padding: 4px 0; font-style: italic; }
/* Shell notices (open admin warning, update available) */
.notice {
display: flex; align-items: center; gap: 8px;
padding: 8px 18px; font-size: 13px;
border-bottom: 1px solid var(--border-soft, #232833);
}
.notice.warn { background: color-mix(in srgb, #e5a54b 12%, var(--bg)); }
.notice.info { background: color-mix(in srgb, var(--accent) 10%, var(--bg)); }
.notice .icon-btn { margin-left: auto; }
/* Dashboard host metrics */
.sysgrid { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 16px; }
.sysitem .k { color: var(--muted); font-size: 12px; }
.sysitem .v { font-weight: 600; }
.sysbar {
height: 8px; border-radius: 6px; overflow: hidden; margin: 6px 0;
background: var(--bg-2, #12151c); border: 1px solid var(--border, #2a2f3a);
}
.sysbar i { display: block; height: 100%; background: var(--accent, #5b8def); }
.sysbar i.bad { background: #e5534b; }
/* Log viewer */
.logs-view { max-height: 68vh; }
/* Chat dialogue bubbles */
.chat-log { max-height: 56vh; overflow-y: auto; display: flex; flex-direction: column; gap: 8px; }
.chat-msg { display: flex; }
.chat-msg.user { justify-content: flex-end; }
.chat-bubble {
max-width: 85%; padding: 8px 12px; border-radius: 12px;
white-space: pre-wrap; word-break: break-word; font-size: 13.5px;
background: var(--bg-2, #12151c); border: 1px solid var(--border-soft, #232833);
}
.chat-msg.user .chat-bubble {
background: color-mix(in srgb, var(--accent) 16%, var(--bg-2));
border-color: color-mix(in srgb, var(--accent) 35%, transparent);
}
/* Generation gallery */
.gal-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(110px, 1fr)); gap: 8px; }
.gal-item {
position: relative; cursor: pointer; border: 1px solid var(--border, #2a2f3a);
border-radius: 8px; overflow: hidden; background: var(--bg-2, #12151c);
}
.gal-item:hover { border-color: var(--accent, #5b8def); }
.gal-thumb { width: 100%; display: block; min-height: 64px; }
.gal-video { display: flex; align-items: center; justify-content: center; font-size: 28px; height: 72px; }
.gal-cap { font-size: 10px; color: var(--muted); padding: 3px 6px; }
.gal-del {
position: absolute; top: 2px; right: 2px; z-index: 2;
border: 0; border-radius: 50%; width: 18px; height: 18px; line-height: 16px;
background: rgba(0,0,0,.55); color: #fff; cursor: pointer; font-size: 12px;
}
/* Bench + badges */
.bench-row { gap: 10px; padding: 6px 0; border-top: 1px solid var(--border-soft, #232833); }
.bench-row:first-of-type { border-top: 0; }
.badge.warn { background: rgba(229,165,75,.15); color: #e5a54b; }
.snippet { max-height: 260px; }
/* Playground media tabs: generated image / video frame */
.media-out { display: flex; flex-direction: column; gap: 8px; }
.media-canvas {
max-width: 100%; height: auto; border-radius: 8px;
border: 1px solid var(--border, #2a2f3a); background: #000;
}
/* Live download progress inside a featured card (empty until a job runs) */
.card-prog:empty { display: none; }
.card-prog { margin-top: 6px; }
@@ -594,3 +706,8 @@ body.modal-open { overflow: hidden; }
@media (prefers-reduced-motion: reduce) {
.sidebar, .mobile-nav-backdrop { transition: none; }
}
/* full-screen farewell after "stop everything & power off" */
.shutdown-screen { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100vh; gap: 12px; text-align: center; padding: 24px; }
.shutdown-screen .shutdown-icon { font-size: 64px; color: var(--bad); }
.shutdown-screen .hint { max-width: 480px; }
+77 -3
View File
@@ -68,10 +68,13 @@ export const api = {
// routing
getRouting: () => j("GET", "/routing"),
routingLabels: () => j("GET", "/routing/labels"),
putRouting: (b) => j("PUT", "/routing", b),
// protocols
getProtocols: () => j("GET", "/protocols"),
harnesses: () => j("GET", "/harnesses"),
balance: () => j("GET", "/balance"),
putProtocols: (b) => j("PUT", "/protocols", b),
// settings
@@ -118,6 +121,23 @@ export const api = {
registerImport: (job) =>
j("POST", "/import/" + encodeURIComponent(job) + "/register"),
// media generation (cortiq imagine / animate over local .cmf)
mediaModels: () => j("GET", "/media/models"),
mediaGenerate: (b) => j("POST", "/media", b),
mediaJobs: () => j("GET", "/media"),
mediaJob: (id) => j("GET", "/media/" + encodeURIComponent(id)),
mediaCancel: (id) => j("POST", "/media/" + encodeURIComponent(id) + "/cancel"),
mediaDelete: (id) => j("DELETE", "/media/" + encodeURIComponent(id)),
// system / logs / lifecycle
system: () => j("GET", "/system"),
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
clearLogs: () => j("DELETE", "/logs"),
restart: () => j("POST", "/restart"),
shutdown: () => j("POST", "/shutdown"),
benchStart: (model) => j("POST", "/bench", { model }),
benchList: () => j("GET", "/bench"),
// first-run onboarding
setupStatus: () => j("GET", "/setup/status"),
// Connects every finished download as a local model unless register === false.
@@ -134,6 +154,21 @@ export const api = {
),
};
// Raw-body POST (no JSON encoding) — the config import takes TOML text as-is.
export async function postRaw(path, text) {
const tok = getToken();
const headers = { "content-type": "application/toml" };
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch(BASE + path, { method: "POST", headers, body: text });
const body = await r.text();
if (!r.ok) {
let msg = body;
try { msg = JSON.parse(body).error?.message || body; } catch { /* raw */ }
throw new Error(msg || r.statusText);
}
return body ? JSON.parse(body) : null;
}
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
// returns the routing decision read from X-Cortiq-* response headers.
export async function testStream(body, onDelta) {
@@ -155,6 +190,11 @@ export async function testStream(body, onDelta) {
const reader = r.body.getReader();
const dec = new TextDecoder();
let buf = "";
const t0 = performance.now();
let ttftMs = null; // time to first content token
let usage = null; // OpenAI-style final-chunk usage, when the provider sends it
let usageEstimated = false; // gateway-injected estimate rather than tokenizer truth
let costUsd = null; // final cost from the gateway's closing event (headers carry 0)
for (;;) {
const { done, value } = await reader.read();
if (done) break;
@@ -167,8 +207,17 @@ export async function testStream(body, onDelta) {
const data = line.slice(5).trim();
if (!data || data === "[DONE]") continue;
try {
const d = JSON.parse(data).choices?.[0]?.delta?.content;
if (d) onDelta(d);
const chunk = JSON.parse(data);
if (chunk.usage) {
usage = chunk.usage;
usageEstimated = !!chunk.cortiq_estimated;
}
if (chunk.cortiq_cost_usd != null) costUsd = chunk.cortiq_cost_usd;
const d = chunk.choices?.[0]?.delta?.content;
if (d) {
if (ttftMs == null) ttftMs = Math.round(performance.now() - t0);
onDelta(d);
}
} catch {
/* ignore malformed chunk */
}
@@ -181,6 +230,31 @@ export async function testStream(body, onDelta) {
score: parseFloat(h.get("x-cortiq-complexity-score") || "0"),
selected_model: h.get("x-cortiq-selected-model") || "",
route_source: h.get("x-cortiq-route-source") || "",
cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"),
cost_usd: costUsd != null ? costUsd : parseFloat(h.get("x-cortiq-cost-usd") || "0"),
usage,
usage_estimated: usageEstimated,
ttft_ms: ttftMs,
};
}
// Fetch a media job's output with the admin token and hand back an object URL
// (plain <img src>/<video src> cannot carry the Authorization header).
export async function mediaFileUrl(job, kind) {
const tok = getToken();
const headers = {};
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
if (!r.ok) throw new Error((await r.text()) || r.statusText);
return { url: URL.createObjectURL(await r.blob()), size: +(r.headers.get("content-length") || 0) };
}
// Same fetch, raw bytes — for outputs the browser cannot display natively
// (P6 PPM, MJPEG AVI) that we decode in JS.
export async function mediaFileBytes(job, kind) {
const tok = getToken();
const headers = {};
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
if (!r.ok) throw new Error((await r.text()) || r.statusText);
return r.arrayBuffer();
}
+44 -1
View File
@@ -11,6 +11,8 @@ import { renderProtocols } from "./views/protocols.js";
import { renderKeys } from "./views/keys.js";
import { renderPlayground } from "./views/playground.js";
import { renderSettings } from "./views/settings.js";
import { renderLogs } from "./views/logs.js";
import { renderBenchmark } from "./views/benchmark.js";
import { renderOnboarding } from "./views/onboarding.js";
const ROUTES = [
@@ -21,6 +23,8 @@ const ROUTES = [
{ id: "protocols", icon: "⇆", view: renderProtocols },
{ id: "keys", icon: "🔑", view: renderKeys },
{ id: "playground", icon: "✦", view: renderPlayground },
{ id: "benchmark", icon: "⏱", view: renderBenchmark },
{ id: "logs", icon: "📜", view: renderLogs },
{ id: "settings", icon: "⚙", view: renderSettings },
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
];
@@ -52,6 +56,27 @@ const FAVICON =
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="%233b82f6"/><stop offset=".5" stop-color="%238b5cf6"/><stop offset="1" stop-color="%2306b6d4"/></linearGradient></defs><rect width="32" height="32" rx="8" fill="url(%23g)"/><path d="M9 16h14M17 10l6 6-6 6" stroke="white" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>'
);
// One-line banners under the topbar: open-admin warning, update available.
function shellNotices() {
const host = h("div", {});
const m = appState.meta || {};
if (m.admin_open_public && !sessionStorage.getItem("hide_open_admin")) {
const close = h("button", { class: "icon-btn", title: "×", onClick: (e) => { sessionStorage.setItem("hide_open_admin", "1"); e.target.closest(".notice").remove(); } }, "×");
host.appendChild(
h("div", { class: "notice warn" }, "⚠ " + t("notice.openAdmin"), " ",
h("a", { href: "#/settings" }, t("nav.settings") + " →"), close)
);
}
if (m.update_available) {
host.appendChild(
h("div", { class: "notice info" },
"⬆ " + t("notice.update", { v: m.update_available }),
" ", h("code", { class: "mono" }, "cargo install cortiq-gateway"))
);
}
return host;
}
function sidebar(activeId) {
const visibleRoutes = ROUTES.filter((r) => !r.hidden);
return h(
@@ -221,7 +246,7 @@ function renderShell() {
{ class: "app" },
sidebar(route.id),
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
h("main", { class: "main" }, topbar(route.id), shellNotices(), h("div", { class: "content", id: "view" }))
)
);
renderView();
@@ -231,12 +256,30 @@ function showLogin() {
mount(document.getElementById("app"), renderLogin(boot));
}
let staleNotified = false;
async function checkUiFresh() {
// the SPA lives in the tab until reload; if the gateway restarted (dev
// rebuild, update, config restart) the embedded UI may have changed —
// tell the user instead of letting them stare at stale views
try {
const m = await api.meta();
if (appState.meta?.started && m.started && m.started !== appState.meta.started && !staleNotified) {
staleNotified = true;
const bar = h("div", { class: "callout warn", style: "position:fixed; left:50%; transform:translateX(-50%); bottom:18px; z-index:99; box-shadow:0 8px 24px rgba(0,0,0,.35)" },
h("span", {}, "🔄 " + t("shell.uiStale")), " ",
h("button", { class: "btn sm primary", onClick: () => location.reload() }, t("shell.reload")));
document.body.append(bar);
}
} catch {}
}
async function refreshHealth() {
try {
appState.health = await api.health();
} catch {
appState.health = null;
}
checkUiFresh();
// re-render topbar only if shell present
const route = currentRoute();
const main = document.querySelector(".main");
+5
View File
@@ -48,6 +48,11 @@ export function onLangChange(fn) {
return () => listeners.delete(fn);
}
// Human name for a complexity tier; custom tier names pass through as-is.
export function trTier(tier) {
return ["low", "medium", "high"].includes(tier) ? t("tier." + tier) : tier;
}
export function t(key, vars) {
const d = DICTS[lang] || en;
let s = d[key] ?? en[key] ?? key;
+182
View File
@@ -468,4 +468,186 @@ export default {
"onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen",
"onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.",
"onboarding.unregistered": "Ebenfalls abgemeldet: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Bild",
"pg.tab.video": "Video",
"pg.generate": "Generieren",
"pg.generating": "Wird generiert…",
"pg.genFailed": "Generierung fehlgeschlagen — siehe Log.",
"pg.result": "Ergebnis",
"pg.genMetrics": "Generierungsmetriken",
"pg.imagePromptPlaceholder": "ein Rotfuchs im Schnee bei Sonnenuntergang, fotorealistisch, detailliertes Fell",
"pg.videoPromptPlaceholder": "ein Corgi mit Kochmütze wirft einen Pfannkuchen, Brutzeln und ein fröhliches Bellen",
"pg.imageEmpty": "Das generierte Bild erscheint hier.",
"pg.videoEmpty": "Der generierte Clip (mit Ton) erscheint hier.",
"pg.width": "Breite",
"pg.height": "Höhe",
"pg.steps": "Schritte",
"pg.seed": "Seed",
"pg.randomSeed": "Zufälliger Seed",
"pg.cfgHint": "Guidance-Stärke; ≤ 1 schaltet CFG ab und halbiert die Arbeit.",
"pg.frames": "Frames",
"pg.framesHint": "Bei 24 fps; auf das Raster des Modells gerundet.",
"pg.quality": "JPEG-Qualität",
"pg.videoSizeHint": "Vielfache von 32; die trainierte kurze Kante ist 768.",
"pg.videoStepsHint": "Die Turbo-LoRA ist auf 4 Schritte trainiert.",
"pg.firstFrame": "Erstes Frame (optional)",
"pg.lastFrame": "Letztes Frame (optional)",
"pg.keyframeHint": "Macht den Lauf zu Keyframe-to-Video.",
"pg.resolution": "Auflösung",
"pg.genTime": "Generierungszeit",
"pg.timePerStep": "Pro Schritt",
"pg.timePerFrame": "Pro Frame",
"pg.fileSize": "Dateigröße",
"pg.duration": "Dauer",
"pg.play": "Abspielen",
"pg.noMediaModels": "Noch kein lokales {kind}-Modell. Lade eines (z. B. aus der infosave-Sammlung) unter Import herunter:",
"pg.tokSpeed": "Geschwindigkeit",
"pg.ttft": "Erstes Token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Logs",
"logs.subtitle": "Jüngstes Gateway-Log — das, was das Terminal zeigen würde.",
"logs.filterPlaceholder": "filtern…",
"logs.pause": "Pause",
"logs.resume": "Fortsetzen",
"logs.empty": "Nichts gefunden.",
"notice.openAdmin": "Die Konsole ist ohne Token offen und von anderen Rechnern erreichbar. Setze einen Token, wenn diese Maschine nicht privat ist.",
"notice.update": "Update verfügbar: v{v}.",
"dash.sys.title": "System",
"dash.sys.cores": "Kerne",
"dash.sys.disk": "Festplatte (Modelle)",
"dash.sys.free": "frei",
"dash.sys.gateway": "Gateway",
"fit.disk": "zu wenig Speicherplatz",
"fit.diskTitle": "Der Download ist größer als der freie Platz auf der Modell-Festplatte.",
"fit.ram": "passt evtl. nicht in den RAM",
"fit.ramTitle": "Das Modell ist nahe an oder größer als der RAM dieser Maschine — es läuft womöglich nicht.",
"pg.eta": "übrig",
"pg.min": "Min",
"pg.queued": "In der Warteschlange — Position {n}. Startet, sobald die aktuelle Generierung fertig ist.",
"pg.gallery": "Letzte Generierungen",
"pg.galDeleteConfirm": "Diese Generierung samt Dateien löschen?",
"pg.clearChat": "Leeren",
"keys.snip.title": "Deine App verbinden",
"keys.snip.hint": "Füge eines der Beispiele in dein Projekt ein — nur der Schlüssel-Platzhalter muss ersetzt werden.",
"keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
"keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
"keys.snip.keyPh": "dein API-Schlüssel",
"settings.restart": "Alles neu starten",
"settings.restartConfirm": "Gateway und alle lokalen Modellserver neu starten? Laufende Anfragen werden abgebrochen.",
"settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
"settings.shutdown": "Alles stoppen & ausschalten",
"settings.shutdownConfirm": "Alle lokalen Modellserver stoppen und das Gateway ausschalten? API und dieses Panel antworten erst wieder, wenn Sie das Gateway neu starten.",
"settings.shutdownDone": "Das Gateway ist ausgeschaltet.",
"settings.shutdownHint": "Alle lokalen Modellserver sind gestoppt, der Prozess wurde beendet. Starten Sie das Gateway auf dem Server neu, um das Panel zurückzubekommen.",
"bench.title": "Geschwindigkeit lokaler Modelle",
"bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
"bench.run": "Messen",
"bench.running": "misst…",
"bench.failed": "fehlgeschlagen",
"bench.none": "Noch keine lokalen .cmf-Chat-Modelle.",
// task-based routing
"task.chitchat": "Smalltalk",
"task.code": "Code",
"task.creative-writing": "Kreatives Schreiben",
"task.extraction": "Datenextraktion",
"task.math": "Mathematik",
"task.qa": "Fragen & Antworten",
"task.summarization": "Zusammenfassung",
"task.translation": "Übersetzung",
"routing.tasks.title": "Aufgabenbasiertes Routing",
"routing.tasks.hint": "Die Modelle einer Regel kommen für ihre Aufgabe zuerst; die Komplexitätskette bleibt als Fallback. Leer = nur nach Komplexität.",
"routing.tasks.inherit": "nach Komplexität (keine Regel)",
"routing.tasks.share": "{p}% der Anfragen (7 T)",
"routing.tasks.fromRouter": "Labels live vom Router",
"routing.tasks.builtin": "Router nicht erreichbar — eingebauter Labelsatz",
"routing.tasks.addCustom": "Aufgabe hinzufügen",
"routing.tasks.addPlaceholder": "eigenes Label…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Qualitäts- und Tempo-Proben durch die Live-Pipeline: Antworten per Schlüsselwort geprüft, Schleifen erkannt, tok/s und Zeit bis zum ersten Token je Modell.",
"bm.models": "Zu testende Modelle",
"bm.quickMode": "Schnelltest",
"bm.quickHint": "Die schnelle Teilmenge mit kurzen Antworten — etwa eine Minute pro lokalem Modell.",
"bm.suiteInfo": "Eingebaute Suite: {q} schnelle / {f} volle Tests, plus Ihre eigenen.",
"bm.run": "Starten",
"bm.pickModel": "Wählen Sie mindestens ein Modell.",
"bm.testOf": "Test {i} von {n}",
"bm.results": "Ergebnisse",
"bm.quick": "schnell",
"bm.test": "Test",
"bm.cellHint": "Klick auf eine Zelle zeigt die vollständige Antwort.",
"bm.history": "Frühere Läufe (in diesem Browser gespeichert)",
"bm.custom": "Eigene Tests",
"bm.customHint": "Prompt + kommagetrennte Schlüsselwörter, die die Antwort enthalten muss (| für Synonyme). Im Browser gespeichert.",
"bm.promptPh": "Prompt…",
"bm.checkPh": "Schlüsselwörter, kommagetrennt",
"bm.addTest": "Test hinzufügen",
"bm.emptyAnswer": "leere Antwort",
"bm.loopDetected": "Schleife erkannt",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Smartes Routing",
"bm.mode.models": "Einzelne Modelle",
"bm.mode.routingDesc": "Das Kernfeature des Produkts im Test: jede Probe läuft über cortiq-auto — Sie sehen, welche Aufgabe der Router erkannt hat, welches Modell er wählte, ob das den Erwartungen entspricht und was es gekostet hat.",
"bm.mode.modelsDesc": "Dieselbe Suite, an gewählte Modelle gepinnt — Qualität, Tempo und Kosten im direkten Vergleich, am Router vorbei.",
"bm.answersOk": "Antworten korrekt",
"bm.classAcc": "Aufgaben-Klassifikation",
"bm.modelDist": "Anfragen gingen an",
"bm.expectedLbl": "erwartet:",
"bm.answer": "Antwort",
"bm.liveRunning": "läuft…",
"bm.spent": "ausgegeben",
"bm.verdict": "🏁 Fertig: Antworten {p}, ausgegeben {s}",
"bm.clearHistory": "Alle löschen",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — ins Terminal einfügen (das Gateway spricht das Anthropic-Protokoll nativ):",
"keys.h.codex": "Codex — hängt den Provider an ~/.codex/config.toml an und startet:",
"keys.h.opencode": "OpenCode — schreibt ~/.config/opencode/opencode.json (überschreibt!) und startet:",
"keys.h.continue": "Continue (VS Code / JetBrains) — hängt das Modell an ~/.continue/config.yaml an:",
"keys.h.cline1": "Cline (VS Code) — in den Erweiterungseinstellungen wählen:",
"keys.h.cline2": "Einstellungen → API Provider → OpenAI Compatible, dann die Werte oben einfügen.",
"keys.h.note": "cortiq-auto = smartes Routing: der Router wählt das Modell pro Anfrage. Für ein festes Modell den exakten Namen angeben. Unter Windows die Skripte in Git Bash / WSL ausführen oder dieselben Variablen über die Systemumgebung setzen.",
"keys.h.protoOff": "Das Anthropic-Protokoll (/v1/messages) ist deaktiviert — Claude Code kann sich nicht verbinden.",
"keys.h.enableProto": "Aktivieren",
"keys.h.st.installed": "Auf der Gateway-Maschine installiert",
"keys.h.st.configured": "Bereits für dieses Gateway konfiguriert",
"keys.h.st.notFound": "Auf der Gateway-Maschine nicht gefunden",
"bm.degraded": "Router degradiert",
"bm.routerDegraded": "Der Router hat nicht rechtzeitig geantwortet — die Anfrage nutzte die Fallback-Kette; die Klassifikation wird nicht gewertet.",
"bm.cutByCap": "Budget ging fürs Denken drauf — keine Antwort mehr",
"shell.uiStale": "Das Gateway wurde aktualisiert — diese Seite zeigt eine ältere Oberfläche.",
"shell.reload": "Neu laden",
"keys.col.requests": "Anfragen",
"keys.col.tokens": "Tokens ein/aus",
"keys.col.costAll": "$ gesamt",
"routing.tasks.tier": "Komplexität",
"routing.tasks.tierAuto": "Auto (Router)",
"routing.tasks.tierHint": "Welches Modell diese Aufgabe bei welcher Komplexität bedient; \"jede\" führt auf allen Stufen. Reihenfolge = Fallback-Reihenfolge. High eskaliert weiterhin über \"jede\"-Einträge.",
"routing.tasks.tierAny": "jede Komplexität",
"routing.tasks.addModel": "Modell",
"routing.bands.enable": "Eigene Komplexitätsgrenzen",
"routing.bands.hint": "Ziehen Sie die Griffe: wo low endet und high beginnt — die Griffe schieben einander, die Zahlen aktualisieren sich live. Der kontinuierliche Router-Score wird mit IHREN Grenzen neu eingeteilt; ohne Häkchen gelten die Grenzen des Routers.",
"tier.low": "leicht",
"tier.medium": "mittel",
"tier.high": "schwer",
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
"settings.cache.entries": "Einträge",
"settings.cache.hits": "Treffer",
"settings.cache.saved": "gespart",
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
"keys.budgetOf": "Budget {b}/Mon.",
"dash.kpi.orBalance": "OpenRouter-Guthaben",
"keys.h.install": "Installieren:",
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
"keys.snip.apply": "Schlüssel einsetzen",
};
+182
View File
@@ -461,4 +461,186 @@ export default {
"onboarding.deleteFile": "Delete the .cmf file from disk",
"onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.",
"onboarding.unregistered": "Also unregistered: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Image",
"pg.tab.video": "Video",
"pg.generate": "Generate",
"pg.generating": "Generating…",
"pg.genFailed": "Generation failed — see the log.",
"pg.result": "Result",
"pg.genMetrics": "Generation metrics",
"pg.imagePromptPlaceholder": "a red fox sitting in snow at sunset, photorealistic, detailed fur",
"pg.videoPromptPlaceholder": "a corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark",
"pg.imageEmpty": "The generated image will appear here.",
"pg.videoEmpty": "The generated clip (with audio) will appear here.",
"pg.width": "Width",
"pg.height": "Height",
"pg.steps": "Steps",
"pg.seed": "Seed",
"pg.randomSeed": "Random seed",
"pg.cfgHint": "Guidance scale; ≤ 1 disables CFG and halves the work.",
"pg.frames": "Frames",
"pg.framesHint": "At 24 fps; snapped to the model's grid.",
"pg.quality": "JPEG quality",
"pg.videoSizeHint": "Multiples of 32; the trained short edge is 768.",
"pg.videoStepsHint": "The Turbo LoRA is trained for 4 steps.",
"pg.firstFrame": "First frame (optional)",
"pg.lastFrame": "Last frame (optional)",
"pg.keyframeHint": "Turns the run into keyframe-to-video.",
"pg.resolution": "Resolution",
"pg.genTime": "Generation time",
"pg.timePerStep": "Per step",
"pg.timePerFrame": "Per frame",
"pg.fileSize": "File size",
"pg.duration": "Duration",
"pg.play": "Play",
"pg.noMediaModels": "No local {kind} model yet. Download one (e.g. from the infosave collection) in Import:",
"pg.tokSpeed": "Speed",
"pg.ttft": "First token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Logs",
"logs.subtitle": "Recent gateway log — what the terminal would show.",
"logs.filterPlaceholder": "filter…",
"logs.pause": "Pause",
"logs.resume": "Resume",
"logs.empty": "Nothing matches.",
"notice.openAdmin": "The admin console is open without a token and reachable from other machines. Set a token if this box is not private.",
"notice.update": "Update available: v{v}.",
"dash.sys.title": "System",
"dash.sys.cores": "cores",
"dash.sys.disk": "Disk (models)",
"dash.sys.free": "free",
"dash.sys.gateway": "gateway",
"fit.disk": "not enough disk",
"fit.diskTitle": "The download is larger than the free space on the models disk.",
"fit.ram": "may not fit in RAM",
"fit.ramTitle": "The model is close to or larger than this machine's RAM — it may not run.",
"pg.eta": "left",
"pg.min": "min",
"pg.queued": "In queue — position {n}. It starts as soon as the current generation finishes.",
"pg.gallery": "Recent generations",
"pg.galDeleteConfirm": "Delete this generation and its files?",
"pg.clearChat": "Clear",
"keys.snip.title": "Connect your app",
"keys.snip.hint": "Paste one of these into your project — only the API key placeholder needs replacing.",
"keys.snip.withKey": "Snippets below include the key you just created.",
"keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
"keys.snip.keyPh": "your API key",
"settings.restart": "Restart everything",
"settings.restartConfirm": "Restart the gateway and all local model servers? Requests in flight will be dropped.",
"settings.restarting": "Restarting — the page reloads when the gateway is back…",
"settings.shutdown": "Stop everything & power off",
"settings.shutdownConfirm": "Stop all local model servers and power the gateway off? The API and this panel stop answering until you start the gateway again.",
"settings.shutdownDone": "The gateway is off.",
"settings.shutdownHint": "All local model servers are stopped and the process has exited. Start the gateway again on the server to bring the panel back.",
"bench.title": "Local model speed",
"bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
"bench.run": "Measure",
"bench.running": "measuring…",
"bench.failed": "failed",
"bench.none": "No local .cmf chat models yet.",
// task-based routing
"task.chitchat": "Small talk",
"task.code": "Code",
"task.creative-writing": "Creative writing",
"task.extraction": "Data extraction",
"task.math": "Math",
"task.qa": "Q&A",
"task.summarization": "Summarization",
"task.translation": "Translation",
"routing.tasks.title": "Task-based routing",
"routing.tasks.hint": "A rule's models go first for that task; the complexity chain below still applies as fallback. Empty = route by complexity only.",
"routing.tasks.inherit": "by complexity (no rule)",
"routing.tasks.share": "{p}% of requests (7d)",
"routing.tasks.fromRouter": "labels live from the router",
"routing.tasks.builtin": "router unreachable — built-in label set",
"routing.tasks.addCustom": "Add task",
"routing.tasks.addPlaceholder": "custom label…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Quality and speed probes through the live pipeline: keyword-checked answers, anti-loop detection, tok/s and time-to-first-token per model.",
"bm.models": "Models to test",
"bm.quickMode": "Quick test",
"bm.quickHint": "The fast subset with short answers — about a minute per local model.",
"bm.suiteInfo": "Built-in suite: {q} quick / {f} full tests, plus your custom ones.",
"bm.run": "Run",
"bm.pickModel": "Pick at least one model.",
"bm.testOf": "test {i} of {n}",
"bm.results": "Results",
"bm.quick": "quick",
"bm.test": "Test",
"bm.cellHint": "Click a cell to see the model's full answer.",
"bm.history": "Previous runs (stored in this browser)",
"bm.custom": "Custom tests",
"bm.customHint": "Prompt + comma-separated keywords the answer must contain (use | for synonyms). Stored in this browser.",
"bm.promptPh": "prompt…",
"bm.checkPh": "keywords, comma-separated",
"bm.addTest": "Add test",
"bm.emptyAnswer": "empty answer",
"bm.loopDetected": "loop detected",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Smart routing",
"bm.mode.models": "Individual models",
"bm.mode.routingDesc": "The product's core feature under test: every probe goes through cortiq-auto — you see which task the router detected, which model it picked, whether that matches expectations, and what it cost.",
"bm.mode.modelsDesc": "The same suite pinned to selected models — compare their quality, speed and cost head-to-head, bypassing the router.",
"bm.answersOk": "answers correct",
"bm.classAcc": "task classification",
"bm.modelDist": "Requests went to",
"bm.expectedLbl": "expected:",
"bm.answer": "Answer",
"bm.liveRunning": "running…",
"bm.spent": "spent",
"bm.verdict": "🏁 Done: answers {p}, spent {s}",
"bm.clearHistory": "Clear all",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — paste into the terminal (the gateway speaks the Anthropic protocol natively):",
"keys.h.codex": "Codex — appends the provider to ~/.codex/config.toml and starts:",
"keys.h.opencode": "OpenCode — writes ~/.config/opencode/opencode.json (overwrites!) and starts:",
"keys.h.continue": "Continue (VS Code / JetBrains) — appends the model to ~/.continue/config.yaml:",
"keys.h.cline1": "Cline (VS Code) — in extension settings pick:",
"keys.h.cline2": "Settings → API Provider → OpenAI Compatible, then paste the values above.",
"keys.h.note": "cortiq-auto = smart routing: the router picks the model per request. Use an exact model name to pin one. On Windows run the scripts in Git Bash / WSL, or set the same variables via the system environment settings.",
"keys.h.protoOff": "The Anthropic protocol (/v1/messages) is disabled — Claude Code will not connect.",
"keys.h.enableProto": "Enable",
"keys.h.st.installed": "Installed on the gateway machine",
"keys.h.st.configured": "Already configured for this gateway",
"keys.h.st.notFound": "Not found on the gateway machine",
"bm.degraded": "router degraded",
"bm.routerDegraded": "The router did not answer in time — the request used the fallback chain; classification is not judged.",
"bm.cutByCap": "budget spent on reasoning — answer never came",
"shell.uiStale": "The gateway was updated — this page is running an older interface.",
"shell.reload": "Reload",
"keys.col.requests": "Requests",
"keys.col.tokens": "Tokens in/out",
"keys.col.costAll": "$ all-time",
"routing.tasks.tier": "Complexity",
"routing.tasks.tierAuto": "Auto (router)",
"routing.tasks.tierHint": "Which model serves this task at which complexity; \"any\" leads at every tier. Order = fallback order. High verdicts still escalate over any-tier pins.",
"routing.tasks.tierAny": "any complexity",
"routing.tasks.addModel": "model",
"routing.bands.enable": "My complexity boundaries",
"routing.bands.hint": "Drag the handles to set where low ends and where high begins — the handles push each other, the numbers update live. The router's continuous score is re-mapped with YOUR boundaries; unchecked = the router's own bands.",
"tier.low": "low",
"tier.medium": "medium",
"tier.high": "high",
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
"settings.cache.entries": "entries",
"settings.cache.hits": "hits",
"settings.cache.saved": "saved",
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
"keys.budgetOf": "budget {b}/mo",
"dash.kpi.orBalance": "OpenRouter balance",
"keys.h.install": "Install:",
"keys.snip.keyInputPh": "paste your API key",
"keys.snip.apply": "Insert key",
};
+182
View File
@@ -468,4 +468,186 @@ export default {
"onboarding.deleteFile": "Eliminar el archivo .cmf del disco",
"onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.",
"onboarding.unregistered": "También se dio de baja: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Imagen",
"pg.tab.video": "Vídeo",
"pg.generate": "Generar",
"pg.generating": "Generando…",
"pg.genFailed": "La generación falló — revisa el registro.",
"pg.result": "Resultado",
"pg.genMetrics": "Métricas de generación",
"pg.imagePromptPlaceholder": "un zorro rojo en la nieve al atardecer, fotorrealista, pelaje detallado",
"pg.videoPromptPlaceholder": "un corgi con gorro de chef voltea una tortita, chisporroteo y un ladrido alegre",
"pg.imageEmpty": "La imagen generada aparecerá aquí.",
"pg.videoEmpty": "El clip generado (con audio) aparecerá aquí.",
"pg.width": "Ancho",
"pg.height": "Alto",
"pg.steps": "Pasos",
"pg.seed": "Seed",
"pg.randomSeed": "Seed aleatorio",
"pg.cfgHint": "Escala de guidance; ≤ 1 desactiva el CFG y reduce el trabajo a la mitad.",
"pg.frames": "Fotogramas",
"pg.framesHint": "A 24 fps; se ajusta a la rejilla del modelo.",
"pg.quality": "Calidad JPEG",
"pg.videoSizeHint": "Múltiplos de 32; el lado corto entrenado es 768.",
"pg.videoStepsHint": "La LoRA Turbo está entrenada para 4 pasos.",
"pg.firstFrame": "Primer fotograma (opcional)",
"pg.lastFrame": "Último fotograma (opcional)",
"pg.keyframeHint": "Convierte la ejecución en keyframe-to-video.",
"pg.resolution": "Resolución",
"pg.genTime": "Tiempo de generación",
"pg.timePerStep": "Por paso",
"pg.timePerFrame": "Por fotograma",
"pg.fileSize": "Tamaño del archivo",
"pg.duration": "Duración",
"pg.play": "Reproducir",
"pg.noMediaModels": "Aún no hay un modelo {kind} local. Descarga uno (p. ej. de la colección infosave) en Importar:",
"pg.tokSpeed": "Velocidad",
"pg.ttft": "Primer token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Registros",
"logs.subtitle": "Registro reciente de la pasarela — lo que mostraría la terminal.",
"logs.filterPlaceholder": "filtrar…",
"logs.pause": "Pausa",
"logs.resume": "Reanudar",
"logs.empty": "Nada coincide.",
"notice.openAdmin": "La consola está abierta sin token y accesible desde otras máquinas. Configura un token si esta máquina no es privada.",
"notice.update": "Actualización disponible: v{v}.",
"dash.sys.title": "Sistema",
"dash.sys.cores": "núcleos",
"dash.sys.disk": "Disco (modelos)",
"dash.sys.free": "libre",
"dash.sys.gateway": "pasarela",
"fit.disk": "no hay espacio suficiente",
"fit.diskTitle": "La descarga supera el espacio libre del disco de modelos.",
"fit.ram": "puede no caber en RAM",
"fit.ramTitle": "El modelo se acerca o supera la RAM de esta máquina — puede que no funcione.",
"pg.eta": "restante",
"pg.min": "min",
"pg.queued": "En cola — posición {n}. Arranca en cuanto termine la generación actual.",
"pg.gallery": "Generaciones recientes",
"pg.galDeleteConfirm": "¿Eliminar esta generación y sus archivos?",
"pg.clearChat": "Limpiar",
"keys.snip.title": "Conecta tu aplicación",
"keys.snip.hint": "Pega uno de estos ejemplos en tu proyecto — solo hay que sustituir el marcador de la clave.",
"keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
"keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
"keys.snip.keyPh": "tu clave API",
"settings.restart": "Reiniciar todo",
"settings.restartConfirm": "¿Reiniciar la pasarela y todos los servidores de modelos locales? Las solicitudes en curso se interrumpirán.",
"settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
"settings.shutdown": "Detener todo y apagar",
"settings.shutdownConfirm": "¿Detener todos los servidores de modelos locales y apagar la pasarela? La API y este panel dejarán de responder hasta que vuelva a iniciar la pasarela.",
"settings.shutdownDone": "La pasarela está apagada.",
"settings.shutdownHint": "Todos los servidores de modelos locales están detenidos y el proceso ha terminado. Inicie de nuevo la pasarela en el servidor para recuperar el panel.",
"bench.title": "Velocidad de modelos locales",
"bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
"bench.run": "Medir",
"bench.running": "midiendo…",
"bench.failed": "falló",
"bench.none": "Aún no hay modelos de chat .cmf locales.",
// task-based routing
"task.chitchat": "Charla",
"task.code": "Código",
"task.creative-writing": "Escritura creativa",
"task.extraction": "Extracción de datos",
"task.math": "Matemáticas",
"task.qa": "Preguntas y respuestas",
"task.summarization": "Resumen",
"task.translation": "Traducción",
"routing.tasks.title": "Enrutado por tarea",
"routing.tasks.hint": "Los modelos de una regla van primero para su tarea; la cadena por complejidad sigue como respaldo. Vacío = solo por complejidad.",
"routing.tasks.inherit": "por complejidad (sin regla)",
"routing.tasks.share": "{p}% de las peticiones (7 d)",
"routing.tasks.fromRouter": "etiquetas en vivo del router",
"routing.tasks.builtin": "router inaccesible — conjunto integrado",
"routing.tasks.addCustom": "Añadir tarea",
"routing.tasks.addPlaceholder": "etiqueta propia…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Pruebas de calidad y velocidad a través del pipeline en vivo: respuestas verificadas por palabras clave, detección de bucles, tok/s y tiempo del primer token por modelo.",
"bm.models": "Modelos a probar",
"bm.quickMode": "Prueba rápida",
"bm.quickHint": "El subconjunto rápido con respuestas cortas — alrededor de un minuto por modelo local.",
"bm.suiteInfo": "Suite integrada: {q} pruebas rápidas / {f} completas, más las tuyas.",
"bm.run": "Ejecutar",
"bm.pickModel": "Elige al menos un modelo.",
"bm.testOf": "prueba {i} de {n}",
"bm.results": "Resultados",
"bm.quick": "rápida",
"bm.test": "Prueba",
"bm.cellHint": "Haz clic en una celda para ver la respuesta completa.",
"bm.history": "Ejecuciones anteriores (guardadas en este navegador)",
"bm.custom": "Pruebas propias",
"bm.customHint": "Prompt + palabras clave separadas por comas que la respuesta debe contener (| para sinónimos). Se guardan en este navegador.",
"bm.promptPh": "prompt…",
"bm.checkPh": "palabras clave, separadas por comas",
"bm.addTest": "Añadir prueba",
"bm.emptyAnswer": "respuesta vacía",
"bm.loopDetected": "bucle detectado",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Enrutado inteligente",
"bm.mode.models": "Modelos individuales",
"bm.mode.routingDesc": "La función clave del producto a prueba: cada sonda pasa por cortiq-auto — ves qué tarea detectó el router, qué modelo eligió, si coincide con lo esperado y cuánto costó.",
"bm.mode.modelsDesc": "La misma suite fijada a los modelos elegidos — compara su calidad, velocidad y coste directamente, sin pasar por el router.",
"bm.answersOk": "respuestas correctas",
"bm.classAcc": "clasificación de tareas",
"bm.modelDist": "Las peticiones fueron a",
"bm.expectedLbl": "esperado:",
"bm.answer": "Respuesta",
"bm.liveRunning": "en curso…",
"bm.spent": "gastado",
"bm.verdict": "🏁 Listo: respuestas {p}, gastado {s}",
"bm.clearHistory": "Borrar todo",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — pega en la terminal (la pasarela habla el protocolo Anthropic de forma nativa):",
"keys.h.codex": "Codex — añade el proveedor a ~/.codex/config.toml y arranca:",
"keys.h.opencode": "OpenCode — escribe ~/.config/opencode/opencode.json (¡sobrescribe!) y arranca:",
"keys.h.continue": "Continue (VS Code / JetBrains) — añade el modelo a ~/.continue/config.yaml:",
"keys.h.cline1": "Cline (VS Code) — en los ajustes de la extensión elige:",
"keys.h.cline2": "Ajustes → API Provider → OpenAI Compatible, luego pega los valores de arriba.",
"keys.h.note": "cortiq-auto = enrutado inteligente: el router elige el modelo en cada petición. Para fijar uno, indica su nombre exacto. En Windows ejecuta los scripts en Git Bash / WSL o define las mismas variables en el entorno del sistema.",
"keys.h.protoOff": "El protocolo Anthropic (/v1/messages) está desactivado — Claude Code no podrá conectarse.",
"keys.h.enableProto": "Activar",
"keys.h.st.installed": "Instalado en la máquina de la pasarela",
"keys.h.st.configured": "Ya configurado para esta pasarela",
"keys.h.st.notFound": "No encontrado en la máquina de la pasarela",
"bm.degraded": "router degradado",
"bm.routerDegraded": "El router no respondió a tiempo — la petición usó la cadena de reserva; la clasificación no se cuenta.",
"bm.cutByCap": "presupuesto gastado en razonar — sin respuesta final",
"shell.uiStale": "La pasarela se actualizó — esta página usa una interfaz antigua.",
"shell.reload": "Recargar",
"keys.col.requests": "Peticiones",
"keys.col.tokens": "Tokens ent/sal",
"keys.col.costAll": "$ total",
"routing.tasks.tier": "Complejidad",
"routing.tasks.tierAuto": "Auto (router)",
"routing.tasks.tierHint": "Qué modelo atiende esta tarea en qué complejidad; «cualquiera» lidera en todos los niveles. El orden = orden de respaldo. Un veredicto high sigue escalando sobre las entradas «cualquiera».",
"routing.tasks.tierAny": "cualquier complejidad",
"routing.tasks.addModel": "modelo",
"routing.bands.enable": "Mis límites de complejidad",
"routing.bands.hint": "Arrastra los tiradores: dónde termina low y dónde empieza high — se empujan entre sí y las cifras se actualizan en vivo. La puntuación continua del router se re-mapea con TUS límites; sin marcar = las bandas del propio router.",
"tier.low": "fácil",
"tier.medium": "medio",
"tier.high": "difícil",
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
"settings.cache.entries": "entradas",
"settings.cache.hits": "aciertos",
"settings.cache.saved": "ahorrado",
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
"keys.budgetOf": "presupuesto {b}/mes",
"dash.kpi.orBalance": "Saldo OpenRouter",
"keys.h.install": "Instalar:",
"keys.snip.keyInputPh": "pega tu clave API",
"keys.snip.apply": "Insertar clave",
};
+182
View File
@@ -468,4 +468,186 @@ export default {
"onboarding.deleteFile": "Supprimer le fichier .cmf du disque",
"onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.",
"onboarding.unregistered": "Également désenregistré : {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Chat",
"pg.tab.image": "Image",
"pg.tab.video": "Vidéo",
"pg.generate": "Générer",
"pg.generating": "Génération…",
"pg.genFailed": "Échec de la génération — voir le journal.",
"pg.result": "Résultat",
"pg.genMetrics": "Métriques de génération",
"pg.imagePromptPlaceholder": "un renard roux dans la neige au coucher du soleil, photoréaliste, fourrure détaillée",
"pg.videoPromptPlaceholder": "un corgi en toque de chef fait sauter une crêpe, grésillements et aboiement joyeux",
"pg.imageEmpty": "L'image générée apparaîtra ici.",
"pg.videoEmpty": "Le clip généré (avec le son) apparaîtra ici.",
"pg.width": "Largeur",
"pg.height": "Hauteur",
"pg.steps": "Étapes",
"pg.seed": "Seed",
"pg.randomSeed": "Seed aléatoire",
"pg.cfgHint": "Force du guidance ; ≤ 1 désactive le CFG et divise le travail par deux.",
"pg.frames": "Images",
"pg.framesHint": "À 24 fps ; arrondi à la grille du modèle.",
"pg.quality": "Qualité JPEG",
"pg.videoSizeHint": "Multiples de 32 ; le petit côté entraîné est 768.",
"pg.videoStepsHint": "La LoRA Turbo est entraînée pour 4 étapes.",
"pg.firstFrame": "Première image (optionnel)",
"pg.lastFrame": "Dernière image (optionnel)",
"pg.keyframeHint": "Transforme la génération en keyframe-to-video.",
"pg.resolution": "Résolution",
"pg.genTime": "Temps de génération",
"pg.timePerStep": "Par étape",
"pg.timePerFrame": "Par image",
"pg.fileSize": "Taille du fichier",
"pg.duration": "Durée",
"pg.play": "Lire",
"pg.noMediaModels": "Pas encore de modèle {kind} local. Téléchargez-en un (par ex. de la collection infosave) dans Importer :",
"pg.tokSpeed": "Vitesse",
"pg.ttft": "Premier token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Journaux",
"logs.subtitle": "Journal récent de la passerelle — ce que montrerait le terminal.",
"logs.filterPlaceholder": "filtrer…",
"logs.pause": "Pause",
"logs.resume": "Reprendre",
"logs.empty": "Rien ne correspond.",
"notice.openAdmin": "La console est ouverte sans jeton et accessible depuis d'autres machines. Définissez un jeton si cette machine n'est pas privée.",
"notice.update": "Mise à jour disponible : v{v}.",
"dash.sys.title": "Système",
"dash.sys.cores": "cœurs",
"dash.sys.disk": "Disque (modèles)",
"dash.sys.free": "libre",
"dash.sys.gateway": "passerelle",
"fit.disk": "espace disque insuffisant",
"fit.diskTitle": "Le téléchargement dépasse l'espace libre du disque des modèles.",
"fit.ram": "risque de ne pas tenir en RAM",
"fit.ramTitle": "Le modèle approche ou dépasse la RAM de cette machine — il pourrait ne pas tourner.",
"pg.eta": "restant",
"pg.min": "min",
"pg.queued": "En file d'attente — position {n}. Démarre dès la fin de la génération en cours.",
"pg.gallery": "Générations récentes",
"pg.galDeleteConfirm": "Supprimer cette génération et ses fichiers ?",
"pg.clearChat": "Effacer",
"keys.snip.title": "Connecter votre application",
"keys.snip.hint": "Collez un de ces exemples dans votre projet — seul le espace réservé de la clé est à remplacer.",
"keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
"keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
"keys.snip.keyPh": "votre clé API",
"settings.restart": "Tout redémarrer",
"settings.restartConfirm": "Redémarrer la passerelle et tous les serveurs de modèles locaux ? Les requêtes en cours seront interrompues.",
"settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
"settings.shutdown": "Tout arrêter et éteindre",
"settings.shutdownConfirm": "Arrêter tous les serveurs de modèles locaux et éteindre la passerelle ? L'API et ce panneau ne répondront plus tant que vous n'aurez pas relancé la passerelle.",
"settings.shutdownDone": "La passerelle est éteinte.",
"settings.shutdownHint": "Tous les serveurs de modèles locaux sont arrêtés et le processus est terminé. Relancez la passerelle sur le serveur pour retrouver le panneau.",
"bench.title": "Vitesse des modèles locaux",
"bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
"bench.run": "Mesurer",
"bench.running": "mesure…",
"bench.failed": "échec",
"bench.none": "Pas encore de modèles de chat .cmf locaux.",
// task-based routing
"task.chitchat": "Bavardage",
"task.code": "Code",
"task.creative-writing": "Écriture créative",
"task.extraction": "Extraction de données",
"task.math": "Mathématiques",
"task.qa": "Questions-réponses",
"task.summarization": "Résumé",
"task.translation": "Traduction",
"routing.tasks.title": "Routage par tâche",
"routing.tasks.hint": "Les modèles d'une règle passent en premier pour leur tâche ; la chaîne par complexité reste en secours. Vide = complexité seule.",
"routing.tasks.inherit": "par complexité (aucune règle)",
"routing.tasks.share": "{p}% des requêtes (7 j)",
"routing.tasks.fromRouter": "étiquettes reçues du routeur",
"routing.tasks.builtin": "routeur injoignable — jeu d'étiquettes intégré",
"routing.tasks.addCustom": "Ajouter une tâche",
"routing.tasks.addPlaceholder": "étiquette personnalisée…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Sondes de qualité et de vitesse via le pipeline en direct : réponses vérifiées par mots-clés, boucles détectées, tok/s et temps du premier token par modèle.",
"bm.models": "Modèles à tester",
"bm.quickMode": "Test rapide",
"bm.quickHint": "Le sous-ensemble rapide aux réponses courtes — environ une minute par modèle local.",
"bm.suiteInfo": "Suite intégrée : {q} tests rapides / {f} complets, plus les vôtres.",
"bm.run": "Lancer",
"bm.pickModel": "Choisissez au moins un modèle.",
"bm.testOf": "test {i} sur {n}",
"bm.results": "Résultats",
"bm.quick": "rapide",
"bm.test": "Test",
"bm.cellHint": "Cliquez sur une cellule pour voir la réponse complète.",
"bm.history": "Exécutions précédentes (stockées dans ce navigateur)",
"bm.custom": "Tests personnalisés",
"bm.customHint": "Prompt + mots-clés séparés par des virgules que la réponse doit contenir (| pour les synonymes). Stockés dans ce navigateur.",
"bm.promptPh": "prompt…",
"bm.checkPh": "mots-clés, séparés par des virgules",
"bm.addTest": "Ajouter un test",
"bm.emptyAnswer": "réponse vide",
"bm.loopDetected": "boucle détectée",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Routage intelligent",
"bm.mode.models": "Modèles individuels",
"bm.mode.routingDesc": "La fonctionnalité clé du produit à l'épreuve : chaque sonde passe par cortiq-auto — vous voyez quelle tâche le routeur a détectée, quel modèle il a choisi, si cela correspond aux attentes et combien cela a coûté.",
"bm.mode.modelsDesc": "La même suite épinglée aux modèles choisis — comparez leur qualité, vitesse et coût en direct, sans passer par le routeur.",
"bm.answersOk": "réponses correctes",
"bm.classAcc": "classification des tâches",
"bm.modelDist": "Requêtes envoyées vers",
"bm.expectedLbl": "attendu :",
"bm.answer": "Réponse",
"bm.liveRunning": "en cours…",
"bm.spent": "dépensé",
"bm.verdict": "🏁 Terminé : réponses {p}, dépensé {s}",
"bm.clearHistory": "Tout effacer",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — collez dans le terminal (la passerelle parle nativement le protocole Anthropic) :",
"keys.h.codex": "Codex — ajoute le fournisseur à ~/.codex/config.toml et démarre :",
"keys.h.opencode": "OpenCode — écrit ~/.config/opencode/opencode.json (écrase !) et démarre :",
"keys.h.continue": "Continue (VS Code / JetBrains) — ajoute le modèle à ~/.continue/config.yaml :",
"keys.h.cline1": "Cline (VS Code) — dans les réglages de l'extension, choisissez :",
"keys.h.cline2": "Réglages → API Provider → OpenAI Compatible, puis collez les valeurs ci-dessus.",
"keys.h.note": "cortiq-auto = routage intelligent : le routeur choisit le modèle à chaque requête. Pour en fixer un, indiquez son nom exact. Sous Windows, lancez les scripts dans Git Bash / WSL ou définissez les mêmes variables via l'environnement système.",
"keys.h.protoOff": "Le protocole Anthropic (/v1/messages) est désactivé — Claude Code ne pourra pas se connecter.",
"keys.h.enableProto": "Activer",
"keys.h.st.installed": "Installé sur la machine de la passerelle",
"keys.h.st.configured": "Déjà configuré pour cette passerelle",
"keys.h.st.notFound": "Introuvable sur la machine de la passerelle",
"bm.degraded": "routeur dégradé",
"bm.routerDegraded": "Le routeur n'a pas répondu à temps — la requête a suivi la chaîne de secours ; la classification n'est pas comptée.",
"bm.cutByCap": "budget épuisé en raisonnement — pas de réponse",
"shell.uiStale": "La passerelle a été mise à jour — cette page utilise une ancienne interface.",
"shell.reload": "Recharger",
"keys.col.requests": "Requêtes",
"keys.col.tokens": "Tokens in/out",
"keys.col.costAll": "$ au total",
"routing.tasks.tier": "Complexité",
"routing.tasks.tierAuto": "Auto (routeur)",
"routing.tasks.tierHint": "Quel modèle sert cette tâche à quelle complexité ; « toute » mène à tous les niveaux. L'ordre = ordre de repli. Un verdict high escalade toujours au-dessus des entrées « toute ».",
"routing.tasks.tierAny": "toute complexité",
"routing.tasks.addModel": "modèle",
"routing.bands.enable": "Mes limites de complexité",
"routing.bands.hint": "Faites glisser les poignées : où finit low et où commence high — elles se poussent l'une l'autre, les chiffres se mettent à jour en direct. Le score continu du routeur est re-découpé avec VOS limites ; décoché = les bandes du routeur.",
"tier.low": "facile",
"tier.medium": "moyen",
"tier.high": "difficile",
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
"settings.cache.entries": "entrées",
"settings.cache.hits": "hits",
"settings.cache.saved": "économisé",
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
"keys.budgetOf": "budget {b}/mois",
"dash.kpi.orBalance": "Solde OpenRouter",
"keys.h.install": "Installer :",
"keys.snip.keyInputPh": "collez votre clé API",
"keys.snip.apply": "Insérer la clé",
};
+184 -2
View File
@@ -25,7 +25,7 @@ export default {
"common.all": "все",
// nav
"nav.dashboard": "Дашборд",
"nav.dashboard": "Рабочий стол",
"nav.models": "Модели",
"nav.import": "Импорт модели",
"nav.routing": "Роутинг",
@@ -56,7 +56,7 @@ export default {
"login.failed": "Неверный токен администратора.",
// dashboard
"dash.title": "Дашборд",
"dash.title": "Рабочий стол",
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
"dash.clear": "Очистить логи",
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
@@ -460,4 +460,186 @@ export default {
"onboarding.deleteFile": "Удалить файл .cmf с диска",
"onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.",
"onboarding.unregistered": "Также отключено из моделей: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Чат",
"pg.tab.image": "Изображение",
"pg.tab.video": "Видео",
"pg.generate": "Сгенерировать",
"pg.generating": "Генерация…",
"pg.genFailed": "Генерация не удалась — смотрите лог.",
"pg.result": "Результат",
"pg.genMetrics": "Метрики генерации",
"pg.imagePromptPlaceholder": "рыжая лиса в снегу на закате, фотореалистично, детальный мех",
"pg.videoPromptPlaceholder": "корги в поварском колпаке подбрасывает блин, шипение сковороды и весёлый лай",
"pg.imageEmpty": "Сгенерированное изображение появится здесь.",
"pg.videoEmpty": "Сгенерированный ролик (со звуком) появится здесь.",
"pg.width": "Ширина",
"pg.height": "Высота",
"pg.steps": "Шаги",
"pg.seed": "Seed",
"pg.randomSeed": "Случайный seed",
"pg.cfgHint": "Сила guidance; ≤ 1 отключает CFG и вдвое ускоряет генерацию.",
"pg.frames": "Кадры",
"pg.framesHint": "При 24 fps; округляется до сетки модели.",
"pg.quality": "Качество JPEG",
"pg.videoSizeHint": "Кратно 32; обученная короткая сторона — 768.",
"pg.videoStepsHint": "Turbo LoRA обучена на 4 шага.",
"pg.firstFrame": "Первый кадр (необязательно)",
"pg.lastFrame": "Последний кадр (необязательно)",
"pg.keyframeHint": "Превращает запуск в keyframe-to-video.",
"pg.resolution": "Разрешение",
"pg.genTime": "Время генерации",
"pg.timePerStep": "На шаг",
"pg.timePerFrame": "На кадр",
"pg.fileSize": "Размер файла",
"pg.duration": "Длительность",
"pg.play": "Играть",
"pg.noMediaModels": "Локальной модели ({kind}) ещё нет. Скачайте её (например, из коллекции infosave) во вкладке Импорт:",
"pg.tokSpeed": "Скорость",
"pg.ttft": "Первый токен",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Логи",
"logs.subtitle": "Последние записи журнала шлюза — то, что показал бы терминал.",
"logs.filterPlaceholder": "фильтр…",
"logs.pause": "Пауза",
"logs.resume": "Продолжить",
"logs.empty": "Ничего не найдено.",
"notice.openAdmin": "Панель управления открыта без токена и доступна с других машин. Задайте токен, если эта машина не приватная.",
"notice.update": "Доступно обновление: v{v}.",
"dash.sys.title": "Система",
"dash.sys.cores": "ядер",
"dash.sys.disk": "Диск (модели)",
"dash.sys.free": "свободно",
"dash.sys.gateway": "шлюз",
"fit.disk": "не хватит места",
"fit.diskTitle": "Загрузка больше свободного места на диске с моделями.",
"fit.ram": "может не влезть в RAM",
"fit.ramTitle": "Модель сравнима с объёмом RAM этой машины или больше — может не запуститься.",
"pg.eta": "осталось",
"pg.min": "мин",
"pg.queued": "В очереди — позиция {n}. Стартует, как только закончится текущая генерация.",
"pg.gallery": "Последние генерации",
"pg.galDeleteConfirm": "Удалить эту генерацию и её файлы?",
"pg.clearChat": "Очистить",
"keys.snip.title": "Подключите ваше приложение",
"keys.snip.hint": "Вставьте один из примеров в свой проект — заменить нужно только плейсхолдер ключа.",
"keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
"keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
"keys.snip.keyPh": "ваш API-ключ",
"settings.restart": "Перезапустить всё",
"settings.restartConfirm": "Перезапустить шлюз и все серверы локальных моделей? Текущие запросы оборвутся.",
"settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
"settings.shutdown": "Остановить всё и выключить",
"settings.shutdownConfirm": "Остановить все серверы локальных моделей и выключить шлюз? API и эта панель перестанут отвечать, пока вы не запустите шлюз заново.",
"settings.shutdownDone": "Шлюз выключен.",
"settings.shutdownHint": "Все серверы локальных моделей остановлены, процесс завершён. Запустите шлюз на сервере заново, чтобы вернуть панель.",
"bench.title": "Скорость локальных моделей",
"bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
"bench.run": "Замерить",
"bench.running": "замер…",
"bench.failed": "ошибка",
"bench.none": "Локальных чат-моделей .cmf пока нет.",
// task-based routing
"task.chitchat": "Светская беседа",
"task.code": "Код",
"task.creative-writing": "Творческие тексты",
"task.extraction": "Извлечение данных",
"task.math": "Математика",
"task.qa": "Вопросы и ответы",
"task.summarization": "Суммаризация",
"task.translation": "Перевод",
"routing.tasks.title": "Распределение по задачам",
"routing.tasks.hint": "Модели правила идут первыми для своей задачи; цепочка по сложности остаётся резервом. Пусто = только по сложности.",
"routing.tasks.inherit": "по сложности (без правила)",
"routing.tasks.share": "{p}% запросов (7д)",
"routing.tasks.fromRouter": "метки получены с роутера",
"routing.tasks.builtin": "роутер недоступен — встроенный набор меток",
"routing.tasks.addCustom": "Добавить задачу",
"routing.tasks.addPlaceholder": "своя метка…",
// benchmark view
"nav.benchmark": "Бенчмарк",
"bm.title": "Бенчмарк",
"bm.subtitle": "Проверки качества и скорости через живой пайплайн: ответы сверяются по ключевым словам, ловится зацикливание, замеряются tok/s и время первого токена.",
"bm.models": "Какие модели тестировать",
"bm.quickMode": "Быстрый тест",
"bm.quickHint": "Короткое подмножество с краткими ответами — около минуты на локальную модель.",
"bm.suiteInfo": "Встроенный набор: {q} быстрых / {f} полных тестов, плюс ваши собственные.",
"bm.run": "Запустить",
"bm.pickModel": "Выберите хотя бы одну модель.",
"bm.testOf": "тест {i} из {n}",
"bm.results": "Результаты",
"bm.quick": "быстрый",
"bm.test": "Тест",
"bm.cellHint": "Клик по ячейке — полный ответ модели.",
"bm.history": "Прошлые прогоны (хранятся в этом браузере)",
"bm.custom": "Свои тесты",
"bm.customHint": "Промпт + ключевые слова через запятую, которые должны быть в ответе (| — синонимы). Хранятся в этом браузере.",
"bm.promptPh": "промпт…",
"bm.checkPh": "ключевые слова через запятую",
"bm.addTest": "Добавить тест",
"bm.emptyAnswer": "пустой ответ",
"bm.loopDetected": "зацикливание",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Умный роутинг",
"bm.mode.models": "Отдельные модели",
"bm.mode.routingDesc": "Проверка главной фишки продукта: каждый тест идёт через cortiq-auto — видно, какую задачу распознал роутер, какую модель выбрал, совпало ли это с ожиданием и сколько это стоило.",
"bm.mode.modelsDesc": "Тот же набор тестов, но закреплённый за выбранными моделями — сравнение их качества, скорости и стоимости напрямую, мимо роутера.",
"bm.answersOk": "ответы верны",
"bm.classAcc": "классификация задач",
"bm.modelDist": "Запросы ушли в",
"bm.expectedLbl": "ожидалось:",
"bm.answer": "Ответ",
"bm.liveRunning": "идёт…",
"bm.spent": "потрачено",
"bm.verdict": "🏁 Готово: ответы {p}, потрачено {s}",
"bm.clearHistory": "Очистить всё",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — вставьте в терминал (шлюз говорит на протоколе Anthropic нативно):",
"keys.h.codex": "Codex — добавит провайдера в ~/.codex/config.toml и запустит:",
"keys.h.opencode": "OpenCode — запишет ~/.config/opencode/opencode.json (перезапишет!) и запустит:",
"keys.h.continue": "Continue (VS Code / JetBrains) — добавит модель в ~/.continue/config.yaml:",
"keys.h.cline1": "Cline (VS Code) — в настройках расширения укажите:",
"keys.h.cline2": "Настройки → API Provider → OpenAI Compatible, затем вставьте значения выше.",
"keys.h.note": "cortiq-auto — умный роутинг: роутер сам выбирает модель под каждый запрос. Чтобы закрепить конкретную, укажите её точное имя. На Windows запускайте скрипты в Git Bash / WSL или задайте те же переменные через настройки окружения системы.",
"keys.h.protoOff": "Протокол Anthropic (/v1/messages) выключен — Claude Code не подключится.",
"keys.h.enableProto": "Включить",
"keys.h.st.installed": "Установлен на машине шлюза",
"keys.h.st.configured": "Уже настроен на этот шлюз",
"keys.h.st.notFound": "Не найден на машине шлюза",
"bm.degraded": "роутер деградировал",
"bm.routerDegraded": "Роутер не успел ответить — запрос ушёл по резервной цепочке; классификация не засчитывается.",
"bm.cutByCap": "бюджет ушёл на размышления — ответ не успел",
"shell.uiStale": "Шлюз обновился — на этой странице старый интерфейс.",
"shell.reload": "Перезагрузить",
"keys.col.requests": "Запросы",
"keys.col.tokens": "Токены вх/исх",
"keys.col.costAll": "$ за всё время",
"routing.tasks.tier": "Сложность",
"routing.tasks.tierAuto": "Авто (по роутеру)",
"routing.tasks.tierHint": "Какая модель обслуживает задачу на какой сложности; «любая» ведёт на всех уровнях. Порядок = порядок фолбэка. Вердикт high по-прежнему эскалирует поверх «любых».",
"routing.tasks.tierAny": "любая сложность",
"routing.tasks.addModel": "модель",
"routing.bands.enable": "Мои границы сложности",
"routing.bands.hint": "Тяните ручки: где кончается low и где начинается high — ручки толкают друг друга, цифры обновляются на лету. Непрерывный score роутера пересчитывается ВАШИМИ границами; галочка снята — действуют пороги самого роутера.",
"tier.low": "лёгкий",
"tier.medium": "средний",
"tier.high": "сложный",
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
"settings.cache.entries": "записей",
"settings.cache.hits": "попаданий",
"settings.cache.saved": "сэкономлено",
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
"keys.budgetOf": "бюджет {b}/мес",
"dash.kpi.orBalance": "Баланс OpenRouter",
"keys.h.install": "Установить:",
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
"keys.snip.apply": "Вставить ключ",
};
+182
View File
@@ -468,4 +468,186 @@ export default {
"onboarding.deleteFile": ".cmf dosyasını diskten sil",
"onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.",
"onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "Sohbet",
"pg.tab.image": "Görsel",
"pg.tab.video": "Video",
"pg.generate": "Oluştur",
"pg.generating": "Oluşturuluyor…",
"pg.genFailed": "Oluşturma başarısız — günlüğe bakın.",
"pg.result": "Sonuç",
"pg.genMetrics": "Oluşturma metrikleri",
"pg.imagePromptPlaceholder": "gün batımında karda oturan kızıl tilki, fotogerçekçi, ayrıntılı tüyler",
"pg.videoPromptPlaceholder": "aşçı şapkalı bir corgi krep çeviriyor, cızırtı sesleri ve neşeli bir havlama",
"pg.imageEmpty": "Oluşturulan görsel burada görünecek.",
"pg.videoEmpty": "Oluşturulan klip (sesiyle) burada görünecek.",
"pg.width": "Genişlik",
"pg.height": "Yükseklik",
"pg.steps": "Adım",
"pg.seed": "Seed",
"pg.randomSeed": "Rastgele seed",
"pg.cfgHint": "Guidance gücü; ≤ 1 CFG'yi kapatır ve işi yarıya indirir.",
"pg.frames": "Kare",
"pg.framesHint": "24 fps'te; modelin ızgarasına yuvarlanır.",
"pg.quality": "JPEG kalitesi",
"pg.videoSizeHint": "32'nin katları; eğitilen kısa kenar 768.",
"pg.videoStepsHint": "Turbo LoRA 4 adım için eğitildi.",
"pg.firstFrame": "İlk kare (isteğe bağlı)",
"pg.lastFrame": "Son kare (isteğe bağlı)",
"pg.keyframeHint": "Çalışmayı keyframe-to-video'ya çevirir.",
"pg.resolution": "Çözünürlük",
"pg.genTime": "Oluşturma süresi",
"pg.timePerStep": "Adım başına",
"pg.timePerFrame": "Kare başına",
"pg.fileSize": "Dosya boyutu",
"pg.duration": "Süre",
"pg.play": "Oynat",
"pg.noMediaModels": "Henüz yerel bir {kind} modeli yok. İçe aktarma sekmesinden bir tane indirin (ör. infosave koleksiyonundan):",
"pg.tokSpeed": "Hız",
"pg.ttft": "İlk token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "Günlükler",
"logs.subtitle": "Ağ geçidinin son günlüğü — terminalde görecekleriniz.",
"logs.filterPlaceholder": "filtrele…",
"logs.pause": "Duraklat",
"logs.resume": "Devam et",
"logs.empty": "Eşleşen yok.",
"notice.openAdmin": "Yönetim konsolu belirteçsiz açık ve diğer makinelerden erişilebilir. Bu makine özel değilse bir belirteç ayarlayın.",
"notice.update": "Güncelleme mevcut: v{v}.",
"dash.sys.title": "Sistem",
"dash.sys.cores": "çekirdek",
"dash.sys.disk": "Disk (modeller)",
"dash.sys.free": "boş",
"dash.sys.gateway": "ağ geçidi",
"fit.disk": "disk alanı yetmez",
"fit.diskTitle": "İndirme, model diskindeki boş alandan büyük.",
"fit.ram": "RAM'e sığmayabilir",
"fit.ramTitle": "Model bu makinenin RAM'ine yakın ya da ondan büyük — çalışmayabilir.",
"pg.eta": "kaldı",
"pg.min": "dk",
"pg.queued": "Kuyrukta — {n}. sırada. Mevcut üretim biter bitmez başlar.",
"pg.gallery": "Son üretimler",
"pg.galDeleteConfirm": "Bu üretim ve dosyaları silinsin mi?",
"pg.clearChat": "Temizle",
"keys.snip.title": "Uygulamanı bağla",
"keys.snip.hint": "Örneklerden birini projene yapıştır — yalnızca anahtar yer tutucusunu değiştirmen gerekir.",
"keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
"keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
"keys.snip.keyPh": "API anahtarın",
"settings.restart": "Tümünü yeniden başlat",
"settings.restartConfirm": "Ağ geçidi ve tüm yerel model sunucuları yeniden başlatılsın mı? Devam eden istekler kesilecek.",
"settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
"settings.shutdown": "Tümünü durdur ve kapat",
"settings.shutdownConfirm": "Tüm yerel model sunucuları durdurulup ağ geçidi kapatılsın mı? Ağ geçidini yeniden başlatana kadar API ve bu panel yanıt vermez.",
"settings.shutdownDone": "Ağ geçidi kapalı.",
"settings.shutdownHint": "Tüm yerel model sunucuları durduruldu ve süreç sonlandı. Paneli geri getirmek için ağ geçidini sunucuda yeniden başlatın.",
"bench.title": "Yerel model hızı",
"bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
"bench.run": "Ölç",
"bench.running": "ölçülüyor…",
"bench.failed": "başarısız",
"bench.none": "Henüz yerel .cmf sohbet modeli yok.",
// task-based routing
"task.chitchat": "Sohbet",
"task.code": "Kod",
"task.creative-writing": "Yaratıcı yazarlık",
"task.extraction": "Veri çıkarma",
"task.math": "Matematik",
"task.qa": "Soru-cevap",
"task.summarization": "Özetleme",
"task.translation": "Çeviri",
"routing.tasks.title": "Göreve göre yönlendirme",
"routing.tasks.hint": "Kuralın modelleri kendi görevi için önce gelir; karmaşıklık zinciri yedek olarak kalır. Boş = yalnızca karmaşıklığa göre.",
"routing.tasks.inherit": "karmaşıklığa göre (kural yok)",
"routing.tasks.share": "isteklerin %{p}'i (7 g)",
"routing.tasks.fromRouter": "etiketler yönlendiriciden canlı",
"routing.tasks.builtin": "yönlendiriciye ulaşılamıyor — yerleşik etiket seti",
"routing.tasks.addCustom": "Görev ekle",
"routing.tasks.addPlaceholder": "özel etiket…",
// benchmark view
"nav.benchmark": "Kıyaslama",
"bm.title": "Kıyaslama",
"bm.subtitle": "Canlı hat üzerinden kalite ve hız probları: yanıtlar anahtar kelimeyle doğrulanır, döngü tespit edilir, model başına tok/s ve ilk token süresi ölçülür.",
"bm.models": "Test edilecek modeller",
"bm.quickMode": "Hızlı test",
"bm.quickHint": "Kısa yanıtlı hızlı alt küme — yerel model başına yaklaşık bir dakika.",
"bm.suiteInfo": "Yerleşik küme: {q} hızlı / {f} tam test, artı kendi testleriniz.",
"bm.run": "Çalıştır",
"bm.pickModel": "En az bir model seçin.",
"bm.testOf": "{n} testten {i}.",
"bm.results": "Sonuçlar",
"bm.quick": "hızlı",
"bm.test": "Test",
"bm.cellHint": "Modelin tam yanıtı için hücreye tıklayın.",
"bm.history": "Önceki çalıştırmalar (bu tarayıcıda saklanır)",
"bm.custom": "Özel testler",
"bm.customHint": "İstem + yanıtın içermesi gereken virgülle ayrılmış anahtar kelimeler (| eşanlamlılar için). Bu tarayıcıda saklanır.",
"bm.promptPh": "istem…",
"bm.checkPh": "anahtar kelimeler, virgülle",
"bm.addTest": "Test ekle",
"bm.emptyAnswer": "boş yanıt",
"bm.loopDetected": "döngü tespit edildi",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Akıllı yönlendirme",
"bm.mode.models": "Tek tek modeller",
"bm.mode.routingDesc": "Ürünün temel özelliği testte: her prob cortiq-auto üzerinden gider — yönlendiricinin hangi görevi algıladığını, hangi modeli seçtiğini, bunun beklentiyle örtüşüp örtüşmediğini ve maliyetini görürsünüz.",
"bm.mode.modelsDesc": "Aynı küme seçilen modellere sabitlenir — yönlendiriciyi atlayarak kalite, hız ve maliyeti doğrudan karşılaştırın.",
"bm.answersOk": "yanıtlar doğru",
"bm.classAcc": "görev sınıflandırması",
"bm.modelDist": "İstekler şuraya gitti",
"bm.expectedLbl": "beklenen:",
"bm.answer": "Yanıt",
"bm.liveRunning": "sürüyor…",
"bm.spent": "harcanan",
"bm.verdict": "🏁 Bitti: yanıtlar {p}, harcanan {s}",
"bm.clearHistory": "Tümünü temizle",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — terminale yapıştırın (ağ geçidi Anthropic protokolünü doğal olarak konuşur):",
"keys.h.codex": "Codex — sağlayıcıyı ~/.codex/config.toml dosyasına ekler ve başlatır:",
"keys.h.opencode": "OpenCode — ~/.config/opencode/opencode.json dosyasını yazar (üzerine yazar!) ve başlatır:",
"keys.h.continue": "Continue (VS Code / JetBrains) — modeli ~/.continue/config.yaml dosyasına ekler:",
"keys.h.cline1": "Cline (VS Code) — eklenti ayarlarında şunları seçin:",
"keys.h.cline2": "Ayarlar → API Provider → OpenAI Compatible, ardından yukarıdaki değerleri yapıştırın.",
"keys.h.note": "cortiq-auto = akıllı yönlendirme: yönlendirici her istek için modeli kendisi seçer. Belirli bir model için tam adını yazın. Windows'ta betikleri Git Bash / WSL içinde çalıştırın veya aynı değişkenleri sistem ortamından ayarlayın.",
"keys.h.protoOff": "Anthropic protokolü (/v1/messages) kapalı — Claude Code bağlanamaz.",
"keys.h.enableProto": "Etkinleştir",
"keys.h.st.installed": "Ağ geçidi makinesinde kurulu",
"keys.h.st.configured": "Bu ağ geçidi için zaten yapılandırılmış",
"keys.h.st.notFound": "Ağ geçidi makinesinde bulunamadı",
"bm.degraded": "yönlendirici yedeğe düştü",
"bm.routerDegraded": "Yönlendirici zamanında yanıt vermedi — istek yedek zincire gitti; sınıflandırma sayılmaz.",
"bm.cutByCap": "bütçe düşünmeye gitti — yanıt gelemedi",
"shell.uiStale": "Ağ geçidi güncellendi — bu sayfa eski arayüzü çalıştırıyor.",
"shell.reload": "Yeniden yükle",
"keys.col.requests": "İstekler",
"keys.col.tokens": "Token giriş/çıkış",
"keys.col.costAll": "$ toplam",
"routing.tasks.tier": "Karmaşıklık",
"routing.tasks.tierAuto": "Otomatik (yönlendirici)",
"routing.tasks.tierHint": "Bu görevi hangi karmaşıklıkta hangi model karşılar; \"her\" tüm katmanlarda önde gider. Sıra = yedekleme sırası. High kararı \"her\" girdilerinin üzerine yine tırmanır.",
"routing.tasks.tierAny": "her karmaşıklık",
"routing.tasks.addModel": "model",
"routing.bands.enable": "Kendi karmaşıklık sınırlarım",
"routing.bands.hint": "Tutamaçları sürükleyin: low nerede biter, high nerede başlar — tutamaçlar birbirini iter, sayılar anlık güncellenir. Yönlendiricinin sürekli puanı SİZİN sınırlarınızla yeniden bölümlenir; işaret kaldırılırsa yönlendiricinin kendi bantları geçerli.",
"tier.low": "kolay",
"tier.medium": "orta",
"tier.high": "zor",
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
"settings.cache.entries": "kayıt",
"settings.cache.hits": "isabet",
"settings.cache.saved": "tasarruf",
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
"keys.budgetOf": "bütçe {b}/ay",
"dash.kpi.orBalance": "OpenRouter bakiyesi",
"keys.h.install": "Kurulum:",
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
"keys.snip.apply": "Anahtarı ekle",
};
+182
View File
@@ -468,4 +468,186 @@ export default {
"onboarding.deleteFile": "从磁盘删除该 .cmf 文件",
"onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。",
"onboarding.unregistered": "同时已取消注册:{ids}",
// playground: media generation + streaming metrics
"pg.tab.chat": "对话",
"pg.tab.image": "图像",
"pg.tab.video": "视频",
"pg.generate": "生成",
"pg.generating": "生成中……",
"pg.genFailed": "生成失败 — 请查看日志。",
"pg.result": "结果",
"pg.genMetrics": "生成指标",
"pg.imagePromptPlaceholder": "夕阳下雪地里的红狐狸,照片级真实感,毛发细节丰富",
"pg.videoPromptPlaceholder": "戴厨师帽的柯基翻煎饼,滋滋声和欢快的犬吠",
"pg.imageEmpty": "生成的图像将显示在这里。",
"pg.videoEmpty": "生成的短片(含音频)将显示在这里。",
"pg.width": "宽度",
"pg.height": "高度",
"pg.steps": "步数",
"pg.seed": "Seed",
"pg.randomSeed": "随机 seed",
"pg.cfgHint": "引导强度;≤ 1 关闭 CFG,工作量减半。",
"pg.frames": "帧数",
"pg.framesHint": "24 fps;按模型网格取整。",
"pg.quality": "JPEG 质量",
"pg.videoSizeHint": "32 的倍数;训练短边为 768。",
"pg.videoStepsHint": "Turbo LoRA 按 4 步训练。",
"pg.firstFrame": "首帧(可选)",
"pg.lastFrame": "末帧(可选)",
"pg.keyframeHint": "使运行变为关键帧生视频。",
"pg.resolution": "分辨率",
"pg.genTime": "生成耗时",
"pg.timePerStep": "每步",
"pg.timePerFrame": "每帧",
"pg.fileSize": "文件大小",
"pg.duration": "时长",
"pg.play": "播放",
"pg.noMediaModels": "还没有本地{kind}模型。请在“导入模型”中下载一个(例如 infosave 合集):",
"pg.tokSpeed": "速度",
"pg.ttft": "首个 token",
// ops: logs, system, notices, gallery, chat, snippets, bench
"nav.logs": "日志",
"logs.subtitle": "网关最近的日志 — 终端里会看到的内容。",
"logs.filterPlaceholder": "过滤……",
"logs.pause": "暂停",
"logs.resume": "继续",
"logs.empty": "没有匹配内容。",
"notice.openAdmin": "管理控制台未设置令牌即可从其他机器访问。若这台机器不是私有的,请设置令牌。",
"notice.update": "有可用更新:v{v}。",
"dash.sys.title": "系统",
"dash.sys.cores": "核",
"dash.sys.disk": "磁盘(模型)",
"dash.sys.free": "可用",
"dash.sys.gateway": "网关",
"fit.disk": "磁盘空间不足",
"fit.diskTitle": "下载大小超过模型磁盘的可用空间。",
"fit.ram": "可能超出内存",
"fit.ramTitle": "模型接近或超过本机内存 — 可能无法运行。",
"pg.eta": "剩余",
"pg.min": "分钟",
"pg.queued": "排队中 — 第 {n} 位。当前生成结束后立即开始。",
"pg.gallery": "最近的生成",
"pg.galDeleteConfirm": "删除这次生成及其文件?",
"pg.clearChat": "清空",
"keys.snip.title": "接入你的应用",
"keys.snip.hint": "把示例粘贴到你的项目里 — 只需替换密钥占位符。",
"keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
"keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
"keys.snip.keyPh": "你的 API 密钥",
"settings.restart": "全部重启",
"settings.restartConfirm": "重启网关和所有本地模型服务器?正在处理的请求将被中断。",
"settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
"settings.shutdown": "全部停止并关机",
"settings.shutdownConfirm": "停止所有本地模型服务器并关闭网关?在你重新启动网关之前,API 和此面板都将无法响应。",
"settings.shutdownDone": "网关已关闭。",
"settings.shutdownHint": "所有本地模型服务器已停止,进程已退出。请在服务器上重新启动网关以恢复面板。",
"bench.title": "本地模型速度",
"bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s,一次一个模型。",
"bench.run": "测速",
"bench.running": "测速中……",
"bench.failed": "失败",
"bench.none": "还没有本地 .cmf 聊天模型。",
// task-based routing
"task.chitchat": "闲聊",
"task.code": "代码",
"task.creative-writing": "创意写作",
"task.extraction": "数据提取",
"task.math": "数学",
"task.qa": "问答",
"task.summarization": "摘要",
"task.translation": "翻译",
"routing.tasks.title": "按任务路由",
"routing.tasks.hint": "规则中的模型对该任务优先;下方的复杂度链仍作为回退。留空 = 仅按复杂度。",
"routing.tasks.inherit": "按复杂度(无规则)",
"routing.tasks.share": "{p}% 的请求(7 天)",
"routing.tasks.fromRouter": "标签实时来自路由器",
"routing.tasks.builtin": "路由器不可达 — 使用内置标签集",
"routing.tasks.addCustom": "添加任务",
"routing.tasks.addPlaceholder": "自定义标签……",
// benchmark view
"nav.benchmark": "基准测试",
"bm.title": "基准测试",
"bm.subtitle": "通过实时管线做质量与速度探测:按关键词核对答案、检测循环输出,并按模型测量 tok/s 与首个 token 时间。",
"bm.models": "要测试的模型",
"bm.quickMode": "快速测试",
"bm.quickHint": "答案简短的快速子集 — 每个本地模型约一分钟。",
"bm.suiteInfo": "内置套件:{q} 个快速 / {f} 个完整测试,另加你自己的。",
"bm.run": "运行",
"bm.pickModel": "请至少选择一个模型。",
"bm.testOf": "第 {i}/{n} 个测试",
"bm.results": "结果",
"bm.quick": "快速",
"bm.test": "测试",
"bm.cellHint": "点击单元格查看模型的完整回答。",
"bm.history": "历史运行(保存在本浏览器)",
"bm.custom": "自定义测试",
"bm.customHint": "提示词 + 逗号分隔的关键词,回答必须包含它们(| 表示同义词)。保存在本浏览器。",
"bm.promptPh": "提示词……",
"bm.checkPh": "关键词,逗号分隔",
"bm.addTest": "添加测试",
"bm.emptyAnswer": "回答为空",
"bm.loopDetected": "检测到循环",
// benchmark: routing mode, cost, live
"bm.mode.routing": "智能路由",
"bm.mode.models": "单独模型",
"bm.mode.routingDesc": "测试产品核心能力:每个探测都经过 cortiq-auto — 可以看到路由器识别的任务、选中的模型、是否符合预期,以及花费了多少。",
"bm.mode.modelsDesc": "同一套测试固定在所选模型上 — 绕过路由器,直接对比它们的质量、速度与成本。",
"bm.answersOk": "回答正确",
"bm.classAcc": "任务分类",
"bm.modelDist": "请求流向",
"bm.expectedLbl": "预期:",
"bm.answer": "回答",
"bm.liveRunning": "进行中……",
"bm.spent": "花费",
"bm.verdict": "🏁 完成:回答 {p},花费 {s}",
"bm.clearHistory": "全部清除",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — 粘贴到终端(网关原生支持 Anthropic 协议):",
"keys.h.codex": "Codex — 将提供商追加到 ~/.codex/config.toml 并启动:",
"keys.h.opencode": "OpenCode — 写入 ~/.config/opencode/opencode.json(会覆盖!)并启动:",
"keys.h.continue": "Continue(VS Code / JetBrains)— 将模型追加到 ~/.continue/config.yaml:",
"keys.h.cline1": "Cline(VS Code)— 在扩展设置中选择:",
"keys.h.cline2": "设置 → API Provider → OpenAI Compatible,然后粘贴上面的值。",
"keys.h.note": "cortiq-auto = 智能路由:路由器为每个请求选择模型。要固定某个模型,请填写其精确名称。Windows 上请在 Git Bash / WSL 中运行脚本,或通过系统环境变量设置相同的值。",
"keys.h.protoOff": "Anthropic 协议(/v1/messages)已关闭 — Claude Code 无法连接。",
"keys.h.enableProto": "启用",
"keys.h.st.installed": "已安装在网关所在机器上",
"keys.h.st.configured": "已配置为使用此网关",
"keys.h.st.notFound": "网关所在机器上未找到",
"bm.degraded": "路由器降级",
"bm.routerDegraded": "路由器未及时响应 — 请求走了备用链路;分类不计入统计。",
"bm.cutByCap": "预算全用于思考——未能给出答案",
"shell.uiStale": "网关已更新 — 此页面运行的是旧界面。",
"shell.reload": "重新加载",
"keys.col.requests": "请求数",
"keys.col.tokens": "Token 入/出",
"keys.col.costAll": "$ 累计",
"routing.tasks.tier": "复杂度",
"routing.tasks.tierAuto": "自动(路由器)",
"routing.tasks.tierHint": "哪个模型在哪个复杂度上处理该任务;「任意」在所有层级领先。顺序即回退顺序。high 判定仍会越过「任意」条目升级。",
"routing.tasks.tierAny": "任意复杂度",
"routing.tasks.addModel": "模型",
"routing.bands.enable": "自定义复杂度边界",
"routing.bands.hint": "拖动手柄设定 low 到哪里结束、high 从哪里开始 — 手柄会相互推动,数字实时更新。路由器的连续得分将按你的边界重新划分;取消勾选则使用路由器自身的边界。",
"tier.low": "简单",
"tier.medium": "中等",
"tier.high": "困难",
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
"settings.cache.entries": "条目",
"settings.cache.hits": "命中",
"settings.cache.saved": "已节省",
"keys.form.budget": "每月预算,$(0 = 不限)",
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
"keys.budgetOf": "预算 {b}/月",
"dash.kpi.orBalance": "OpenRouter 余额",
"keys.h.install": "安装:",
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
"keys.snip.apply": "插入密钥",
};
+167
View File
@@ -0,0 +1,167 @@
// Browser-side codecs for cortiq's dependency-free media outputs:
// - P6 PPM → canvas (cortiq imagine links no image library)
// - MJPEG+PCM AVI → frame list + canvas player synced to the .wav
// (cortiq animate never touches ffmpeg; neither do we)
/// Parse binary P6 PPM into {width, height, rgb: Uint8Array}.
export function parsePpm(buf) {
const b = new Uint8Array(buf);
// header: "P6" <ws> width <ws> height <ws> maxval <single ws> raster
let pos = 0;
const isWs = (c) => c === 32 || c === 9 || c === 10 || c === 13;
function token() {
while (pos < b.length && (isWs(b[pos]) || b[pos] === 35 /* # comment */)) {
if (b[pos] === 35) while (pos < b.length && b[pos] !== 10) pos++;
else pos++;
}
let start = pos;
while (pos < b.length && !isWs(b[pos])) pos++;
return String.fromCharCode(...b.slice(start, pos));
}
if (token() !== "P6") throw new Error("not a P6 PPM");
const width = parseInt(token());
const height = parseInt(token());
const maxval = parseInt(token());
pos++; // single whitespace after maxval
if (!width || !height || !maxval) throw new Error("bad PPM header");
return { width, height, rgb: b.subarray(pos, pos + width * height * 3) };
}
/// Draw a parsed PPM onto a (new or given) canvas. Returns the canvas.
export function ppmToCanvas(buf, canvas) {
const { width, height, rgb } = parsePpm(buf);
const c = canvas || document.createElement("canvas");
c.width = width;
c.height = height;
const ctx = c.getContext("2d");
const img = ctx.createImageData(width, height);
for (let i = 0, j = 0; i < width * height; i++, j += 3) {
img.data[i * 4] = rgb[j];
img.data[i * 4 + 1] = rgb[j + 1];
img.data[i * 4 + 2] = rgb[j + 2];
img.data[i * 4 + 3] = 255;
}
ctx.putImageData(img, 0, 0);
return c;
}
/// Rasterize an uploaded image file to WxH (stretched, as the engine does with
/// the first frame) and return it as base64-encoded binary P6 PPM.
export async function fileToPpmB64(file, width, height) {
const bmp = await createImageBitmap(file);
const c = document.createElement("canvas");
c.width = width;
c.height = height;
const ctx = c.getContext("2d");
ctx.drawImage(bmp, 0, 0, width, height);
const img = ctx.getImageData(0, 0, width, height).data;
const header = `P6\n${width} ${height}\n255\n`;
const out = new Uint8Array(header.length + width * height * 3);
for (let i = 0; i < header.length; i++) out[i] = header.charCodeAt(i);
let p = header.length;
for (let i = 0; i < width * height; i++) {
out[p++] = img[i * 4];
out[p++] = img[i * 4 + 1];
out[p++] = img[i * 4 + 2];
}
// chunked btoa — the frame can be a few MB
let bin = "";
for (let i = 0; i < out.length; i += 0x8000)
bin += String.fromCharCode.apply(null, out.subarray(i, i + 0x8000));
return btoa(bin);
}
/// Extract JPEG frames from an MJPEG AVI (RIFF): walk chunks from 'movi',
/// collect '00dc'/'00dj' payloads as Blobs.
export function aviJpegFrames(buf) {
const b = new Uint8Array(buf);
const dv = new DataView(buf);
const fourcc = (o) => String.fromCharCode(b[o], b[o + 1], b[o + 2], b[o + 3]);
if (fourcc(0) !== "RIFF" || fourcc(8) !== "AVI ") throw new Error("not an AVI");
// find the 'movi' LIST
let movi = -1;
for (let o = 12; o + 12 <= b.length; ) {
const id = fourcc(o);
const size = dv.getUint32(o + 4, true);
if (id === "LIST" && fourcc(o + 8) === "movi") {
movi = o + 12;
break;
}
o += 8 + size + (size & 1);
}
if (movi < 0) throw new Error("no movi list");
const frames = [];
for (let o = movi; o + 8 <= b.length; ) {
const id = fourcc(o);
const size = dv.getUint32(o + 4, true);
if (id === "00dc" || id === "00dj") {
if (size > 0) frames.push(new Blob([b.subarray(o + 8, o + 8 + size)], { type: "image/jpeg" }));
} else if (id === "LIST") {
o += 12; // descend (rec lists)
continue;
} else if (id === "idx1") {
break;
}
o += 8 + size + (size & 1);
}
return frames;
}
/// Canvas player for extracted MJPEG frames at a fixed fps, synced to an
/// optional <audio> element (the .wav). Returns {canvas, play, pause, seek0, playing}.
export function mjpegPlayer(frames, fps, audio) {
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
let bitmaps = null; // decoded lazily on first play
let raf = null;
let t0 = 0;
let startFrame = 0;
const state = { playing: false };
async function decodeAll() {
if (bitmaps) return;
bitmaps = await Promise.all(frames.map((f) => createImageBitmap(f)));
if (bitmaps.length) {
canvas.width = bitmaps[0].width;
canvas.height = bitmaps[0].height;
ctx.drawImage(bitmaps[0], 0, 0);
}
}
function drawAt(i) {
if (bitmaps && bitmaps[i]) ctx.drawImage(bitmaps[i], 0, 0);
}
function tick() {
// audio clock when available (keeps A/V in sync), wall clock otherwise
const t = audio && !audio.paused ? audio.currentTime : (performance.now() - t0) / 1000;
const i = Math.min(frames.length - 1, startFrame + Math.floor(t * fps));
drawAt(i);
if (i >= frames.length - 1) {
pause();
return;
}
raf = requestAnimationFrame(tick);
}
async function play() {
await decodeAll();
state.playing = true;
startFrame = 0;
t0 = performance.now();
if (audio) {
audio.currentTime = 0;
audio.play().catch(() => {});
}
raf = requestAnimationFrame(tick);
}
function pause() {
state.playing = false;
if (raf) cancelAnimationFrame(raf);
raf = null;
if (audio) audio.pause();
}
async function seek0() {
pause();
await decodeAll();
drawAt(0);
}
return { canvas, play, pause, seek0, state, decodeAll };
}
+16 -2
View File
@@ -95,11 +95,25 @@ export function confirmDialog(message) {
}
// number formatting
// Money as a DOM node: 4 decimals, the last two rendered smaller — tiny
// per-request prices stay readable without shouting six zeros.
export function moneyEl(n) {
if (n == null) return document.createTextNode("—");
if (n === 0) return document.createTextNode("$0");
// sub-0.0001 costs (cheap flash models) would collapse to $0.0000 — grow
// precision until two significant digits survive
const s = Math.abs(n) >= 0.0001 ? n.toFixed(4) : n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
return h("span", { class: "money" }, "$" + s.slice(0, -2), h("small", { class: "money-tail" }, s.slice(-2)));
}
export function money(n) {
if (n == null) return "—";
if (n === 0) return "$0";
if (Math.abs(n) < 0.01) return "$" + n.toFixed(6);
return "$" + n.toFixed(4);
if (Math.abs(n) > 0 && Math.abs(n) < 0.0001) return "$" + n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
// up to 4 decimals, trailing zeros trimmed (but keep money-like 2 minimum)
let s = n.toFixed(4).replace(/0+$/, "").replace(/\.$/, "");
if (/\.\d$/.test(s)) s += "0";
return "$" + s;
}
export function ms(n) {
return n == null ? "—" : `${Math.round(Number(n))} ms`;
+530
View File
@@ -0,0 +1,530 @@
// Benchmark — two modes over the live pipeline:
// 🧭 Smart routing (default): the product's core promise under test — every
// probe goes through cortiq-auto and the router's task/tier decision and
// chosen model are judged alongside the answer itself.
// 🎯 Individual models: the same suite pinned to selected models — compare
// quality and speed head-to-head.
// A redesign of the old standalone benchmark.html: same scoring ideas
// (keyword checks with | synonyms, think-tag stripping, anti-loop detection,
// pass/partial/fail), now embedded, localized and CDN-free.
import { h, mount, toast, ms, money, moneyEl } from "../ui.js";
import { t, trTier } from "../i18n.js";
import { api, testStream } from "../api.js";
// ── built-in suite ──────────────────────────────────────────────────────────
// Checks are language-neutral where possible (digits, identifiers, emails);
// `check` entries may hold |-separated synonyms — one hit per entry counts.
// `quick: true` marks the fast subset.
// `label`: router task labels accepted as a correct classification (null = not
// judged — the prompt has no natural label in the taxonomy).
const SUITE = [
{ id: "math-mult", cat: "math", label: ["math"], quick: true, max: 96, prompt: "What is 17 * 23? Answer with the number only.", check: ["391"] },
{ id: "math-eq", cat: "math", label: ["math"], max: 128, prompt: "Solve 2x + 6 = 14. Answer with the value of x only.", check: ["4"] },
{ id: "logic-syll", cat: "logic", label: ["qa", "math"], max: 96, prompt: "If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? Answer yes or no.", check: ["yes|да|oui|ja|sí|si|evet|是"] },
{ id: "count-letters", cat: "logic", label: ["qa", "math"], quick: true, max: 64, prompt: "How many letters are in the word 'cat'? Answer with a digit.", check: ["3"] },
{ id: "apples", cat: "logic", label: ["math", "qa"], quick: true, max: 96, prompt: "John has 3 apples, buys 2 more, then eats 1. How many apples does he have? Answer with a digit.", check: ["4"] },
{ id: "code-fact", cat: "code", label: ["code"], max: 256, prompt: "Write a Python function factorial(n) that returns n!.", check: ["def", "factorial", "return"] },
{ id: "qa-capital", cat: "qa", label: ["qa"], quick: true, max: 48, prompt: "What is the capital of France? One word.", check: ["paris|париж"] },
{ id: "primes", cat: "qa", label: ["math", "qa"], max: 96, prompt: "List the first five prime numbers, comma-separated.", check: ["2", "3", "5", "7", "11"] },
{ id: "instr-ok", cat: "instruction", label: null, quick: true, max: 32, prompt: "Reply with exactly the word OK and nothing else.", check: ["ok"] },
{ id: "extract-email", cat: "extraction", label: ["extraction"], quick: true, max: 64, prompt: "Extract the email address from: 'Contact us at info@example.com today.' Reply with the address only.", check: ["info@example.com"] },
{ id: "translate", cat: "translation", label: ["translation"], max: 64, prompt: "Translate to English: «Доброе утро». Reply with the translation only.", check: ["good morning"] },
{ id: "format-json", cat: "instruction", label: null, max: 96, prompt: 'Return a JSON object with a single key "n" whose value is 7. JSON only.', check: ['"n"', "7"] },
{ id: "summarize", cat: "summarization", label: ["summarization"], max: 96, prompt: "Summarize in one short sentence: 'The cat sat on the warm mat by the window. After a while it curled up and fell asleep in the sun.'", check: ["cat|кот|кошк"] },
{ id: "creative", cat: "creative-writing", label: ["creative-writing"], max: 512, prompt: "Write a two-line poem about the sea.", check: ["sea|море|ocean"] },
{ id: "chitchat", cat: "chitchat", label: ["chitchat"], quick: true, max: 48, prompt: "Hi! How are you today?", check: [] },
// hard probes: dense enough that the router's complexity model rates them
// high tier — this is where tier-based routing actually earns its keep.
// Prompts are GENERATED fresh each run (domain + numbers rotate): the
// router deliberately scores familiar text as less novel, so any fixed
// wording would decay to medium/low over repeated runs.
// label: null — ambiguity between task labels is exactly what makes these high
{ id: "hard-science", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "science" },
{ id: "hard-ops", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "ops" },
];
// Fresh hard prompts per run. Two findings drive the design (measured against
// the live router): ambiguity carries the biggest weight in the complexity
// score, so INTERDISCIPLINARY prompts (science + engineering + planning) rate
// high reliably, while single-domain ones classify too confidently and drop
// to medium; and familiar text loses novelty on every repeat, so domains and
// numbers rotate to keep each run lexically fresh.
function hardPrompt(kind) {
const pick = (a) => a[(Math.random() * a.length) | 0];
const n = (lo, hi) => lo + ((Math.random() * (hi - lo)) | 0);
const science = [
() => `Architect a protein-folding inference pipeline for a wet lab: ensemble MSA generation with taxonomy-aware subsampling, template selection under a ${n(30, 60)} percent identity ceiling, distributed attention across ${n(4, 16)} GPUs with activation checkpointing, confidence calibration against experimental B-factors, and an active-learning loop that decides which of ${n(300, 2000)} candidate mutants to synthesize next week.`,
() => `Schedule observations for a network of ${n(8, 30)} robotic telescopes chasing gravitational-wave counterparts: probability skymaps updated every ${n(5, 20)} minutes, slew-time-aware target assignment with decaying rewards, cloud-cover forecasts folded in as stochastic availability, spectroscopy triggered only above a ${n(40, 85)} percent kilonova likelihood, and a plan to validate the scheduler against the one historical multi-messenger event.`,
() => `Run a closed-loop materials-discovery campaign for solid-state electrolytes: DFT screening of ${n(2000, 9000)} candidates, Bayesian optimization driving ${n(3, 12)} parallel synthesis robots, CNN-based XRD phase identification, an ionic-conductivity target of ${n(2, 25)} mS/cm, and an explicit decision policy for when simulation is trusted over synthesis.`,
() => `Design a surface-code error-correction pipeline for a ${n(400, 3000)}-qubit superconducting processor: choose a code distance for a logical error target of 1e-${n(9, 14)} at physical error rate 1e-3, syndrome extraction avoiding hook errors, a real-time decoder under a ${n(1, 4)} microsecond budget, and the classical co-processor layout with feedback for lattice surgery.`,
() => `Design a monitoring-reporting-verification pipeline for soil-carbon credits across ${n(500, 5000)} farms: satellite spectral indices calibrated against ${n(800, 4000)} physical core samples, hierarchical Bayesian estimation with shrinkage, additionality argued against county-level counterfactuals, uncertainty discounts priced into issuance, and an audit sampling plan catching over-reporting with ${n(90, 99)} percent power.`,
() => `Plan a cryo-EM structure-determination campaign for a ${n(300, 900)} kDa membrane complex: grid-screening strategy across ${n(4, 12)} detergent conditions, particle picking with a distilled transformer at ${n(2, 9)} million particles, ab-initio reconstruction ambiguity resolved by tilt-pair validation, local-resolution-aware model building, and a decision rule for when to stop collecting and publish.`,
];
const ops = [
() => `Model a dengue outbreak across ${n(20, 90)} districts by coupling an SEIR compartment model to vaccine cold-chain optimization: transmission parameters estimated from noisy case counts with a particle filter, uncertainty propagated into weekly demand forecasts, refrigerated routing under a ${n(120, 400)}-dose-per-crate constraint, and a defended ethical weighting for which ${n(3, 12)} districts get priority when supply covers ${n(20, 60)} percent of demand.`,
() => `Co-optimize a regional power grid's day-ahead market with a fleet of ${n(200, 2000)} grid-scale batteries: bid curves learned from ${n(2, 6)} years of price data, unit commitment as mixed-integer programming with ramp constraints, degradation amortized into marginal costs, N-1 contingency screening after every dispatch, and a mechanism-design argument that the bidding rules cannot be gamed by a coalition controlling ${n(10, 30)} percent of storage.`,
() => `Couple regional climate downscaling at ${n(2, 12)} km resolution to catastrophe-bond pricing: bias-correct ${n(10, 40)} CMIP ensemble members, generate stochastic hurricane tracks, propagate landfall distributions into a portfolio of ${n(30, 200)} bonds, price the attachment probabilities, and defend the discount rate applied to century-scale tail risk.`,
() => `Co-design a hospital emergency department's triage policy with its nurse staffing schedule: arrivals as a time-varying Poisson process fitted to ${n(2, 6)} years of data, acuity drift as a hidden Markov chain, staffing as integer programming over ${n(8, 12)}-hour shifts, a queueing bound on door-to-doctor time at the ${n(90, 99)}th percentile, and a fairness constraint preventing systematic delays for low-acuity elderly patients.`,
() => `Optimize a container port's berth allocation jointly with tidal windows and rail departures: vessel ETAs with heavy-tailed uncertainty, berth-crane assignment as a mixed-integer program re-solved hourly, tide-constrained draft limits for the ${n(3, 8)} deepest berths, a queueing model of the truck gate coupled to city traffic, and a robustness argument for surviving a ${n(24, 72)}-hour crane outage.`,
() => `Schedule observations for a fleet of ${n(6, 40)} agricultural drones across ${n(200, 2000)} fields: multispectral revisit deadlines driven by crop growth-stage models, battery-swap logistics as a vehicle-routing problem with time windows, rain forecasts as stochastic field availability, disease-outbreak alerts pre-empting the plan, and a proof the replanner never starves a field past its agronomic deadline.`,
];
return pick(kind === "science" ? science : ops)();
}
const LS_CUSTOM = "allaigate_bench_custom";
const LS_HISTORY = "allaigate_bench_history";
function customTests() {
try {
return JSON.parse(localStorage.getItem(LS_CUSTOM) || "[]");
} catch {
return [];
}
}
function saveCustom(list) {
localStorage.setItem(LS_CUSTOM, JSON.stringify(list));
}
function history() {
try {
return JSON.parse(localStorage.getItem(LS_HISTORY) || "[]");
} catch {
return [];
}
}
function pushHistory(run) {
const l = history();
l.unshift(run);
localStorage.setItem(LS_HISTORY, JSON.stringify(l.slice(0, 12)));
}
function saveHistory(l) {
localStorage.setItem(LS_HISTORY, JSON.stringify(l));
}
// ── evaluation (ported from the old benchmark, trimmed) ─────────────────────
function stripThink(text) {
return text.replace(/<(?:\|)?(?:think|thought)(?:\|)?>[\s\S]*?(?:<\/(?:\|)?(?:think|thought)(?:\|)?>|$)/gi, "").trim();
}
// tail loop: the same 12+ char chunk repeated 3+ times back to back
function looksLooped(text) {
const tail = text.slice(-360);
for (let len = 12; len <= 60; len += 8) {
const probe = tail.slice(-len);
if (probe && tail.split(probe).length - 1 >= 3) return true;
}
return false;
}
function evaluate(raw, test) {
let text = stripThink(raw);
if (!text && raw) text = raw; // max_tokens cut inside <think> — judge the raw
const resp = text.toLowerCase();
if (!resp.trim()) return { status: "fail", score: 0, note: t("bm.emptyAnswer") };
if (looksLooped(resp)) return { status: "fail", score: 0, note: t("bm.loopDetected") };
if (!test.check.length) return { status: "pass", score: 1, note: "✓" };
let hit = 0;
for (const cond of test.check) {
const opts = cond.toLowerCase().split("|").map((s) => s.trim());
if (opts.some((o) => o && resp.includes(o))) hit++;
}
const frac = hit / test.check.length;
const status = frac >= 1 ? "pass" : frac >= 0.5 ? "partial" : "fail";
return { status, score: frac, note: `${hit}/${test.check.length}` };
}
export async function renderBenchmark() {
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
const root = h("div");
let running = false;
let stopFlag = false;
let mode = "routing"; // routing (cortiq-auto, the product's feature) | models
const modelCbs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox", checked: ids.length === 1 ? true : null }) }));
const quickCb = h("input", { type: "checkbox", checked: true });
const resultsHost = h("div", {});
const progressHost = h("div", {});
const historyHost = h("div", {});
const modeDesc = h("p", { class: "muted", style: "margin:8px 0 0" }, t("bm.mode.routingDesc"));
const modelsPick = h("div", { class: "flex wrap", style: "gap:10px; margin-top:12px; display:none" },
...modelCbs.map(({ id, cb }) => h("label", { class: "check" }, cb, h("span", { class: "mono" }, id))));
const modeSeg = h("div", { class: "seg" },
h("button", { class: "active" }, "🧭 " + t("bm.mode.routing")),
h("button", {}, "🎯 " + t("bm.mode.models")));
modeSeg.querySelectorAll("button").forEach((b, i) =>
b.addEventListener("click", () => {
mode = i === 0 ? "routing" : "models";
modeSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === i));
modelsPick.style.display = mode === "models" ? "" : "none";
modeDesc.textContent = t(mode === "routing" ? "bm.mode.routingDesc" : "bm.mode.modelsDesc");
}));
function suite() {
const base = SUITE.concat(customTests());
return quickCb.checked ? base.filter((x) => x.quick || x.custom) : base;
}
const runBtn = h("button", { class: "btn primary" }, "▶ " + t("bm.run"));
const stopBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
stopBtn.addEventListener("click", () => {
stopFlag = true;
stopBtn.disabled = true;
});
runBtn.addEventListener("click", async () => {
const models = mode === "routing" ? ["cortiq-auto"] : modelCbs.filter((m) => m.cb.checked).map((m) => m.id);
if (!models.length) {
toast(t("bm.pickModel"), "warn");
return;
}
running = true;
stopFlag = false;
runBtn.disabled = true;
stopBtn.style.display = "";
stopBtn.disabled = false;
const tests = suite().filter((x) => mode === "routing" || x.judge !== "route");
const run = { ts: Date.now(), mode, quick: quickCb.checked, models: {}, nTests: tests.length };
for (const model of models) {
const rec = { results: [], score: 0, tokS: [], ttft: [], failed: false };
run.models[model] = rec;
for (let i = 0; i < tests.length; i++) {
if (stopFlag) break;
const test = tests[i];
const prompt = test.gen ? hardPrompt(test.gen) : test.prompt;
mount(
progressHost,
h("div", { class: "card" },
h("div", { class: "flex" },
h("span", { class: "spinner" }), " ",
h("b", { class: "mono" }, mode === "routing" ? t("bm.mode.routing") : model), " · ",
t("bm.testOf", { i: i + 1, n: tests.length }), " · ",
h("span", { class: "muted" }, test.id)),
h("div", { class: "muted small", style: "margin-top:6px; font-style:italic" }, "«" + prompt.slice(0, 110) + (prompt.length > 110 ? "…»" : "»")),
h("div", { class: "pbar", style: "margin-top:8px" },
h("div", { class: "pbar-fill det", style: `width:${Math.round((i / tests.length) * 100)}%` })))
);
const started = performance.now();
let text = "";
try {
const info = await testStream(
// +256: headroom for reasoning models — a tight cap cuts them
// mid-<think> and turns a correct model into a false ❌
{ model, messages: [{ role: "user", content: prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
(d) => (text += d)
);
const latency = performance.now() - started;
let ev;
if (test.judge === "route") {
// the test asserts the ROUTER's decision: did it rate this high?
// Two honest retries. (1) The SAME prompt after a pause: the
// router judges difficulty in the background — the first-ever
// occurrence routes by prior, the judgement lands in the cache a
// couple of seconds later, so the second occurrence is the real
// steady-state answer. (2) A fresh template, in case this one's
// wording clustered into a burnt region.
if (info.tier !== "high" && test.gen) {
const attempts = [
{ prompt, delayMs: 4000 },
{ prompt: hardPrompt(test.gen), delayMs: 0 },
];
for (const a of attempts) {
if (info.tier === "high") break;
if (a.delayMs) await new Promise((r) => setTimeout(r, a.delayMs));
try {
const again = await testStream(
{ model, messages: [{ role: "user", content: a.prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
() => {}
);
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
if (again.tier === "high") {
info.tier = again.tier;
info.score = again.score;
info.selected_model = again.selected_model;
info.task_label = again.task_label;
info.route_source = again.route_source;
}
} catch { /* keep previous attempt */ }
}
}
ev = info.tier === "high"
? { status: "pass", score: 1, note: "high ✓" }
: { status: "fail", score: 0, note: (info.tier || "—") + " ≠ high" };
} else {
ev = evaluate(text, test);
}
const compl = info.usage?.completion_tokens;
if (test.judge !== "route" && ev.status === "fail" && !stripThink(text).trim() && compl >= (test.max || 128) + 250) {
ev.note = t("bm.cutByCap"); // the whole budget went into <think>
}
if (compl && latency > 0) {
const gen = info.ttft_ms != null ? Math.max(1, latency - info.ttft_ms) : latency;
rec.tokS.push(compl / (gen / 1000));
}
if (info.ttft_ms != null) rec.ttft.push(info.ttft_ms);
const row = { id: test.id, cat: test.cat, ...ev, latency: Math.round(latency), cost: info.cost_usd || 0, answer: stripThink(text).slice(0, 300) };
if (mode === "routing") {
// router timed out / fell back — there was no classification to
// judge, so it must not count against accuracy
row.degraded = !info.task_label
|| info.task_label === "degraded"
|| String(info.route_source || "").includes("degraded");
row.task = info.task_label || "—";
row.tier = info.tier || "—";
row.cscore = info.score || 0; // непрерывный коэффициент сложности
// (row.score занят долей правильности ответа из evaluate())
row.chosen = info.selected_model || "—";
row.source = info.route_source || "—";
row.expected = test.label; // null = classification not judged
row.taskOk = test.label && !row.degraded ? test.label.includes(info.task_label) : null;
}
rec.results.push(row);
} catch (e) {
rec.results.push({ id: test.id, cat: test.cat, status: "fail", score: 0, note: String(e.message || e).slice(0, 80), latency: 0, answer: "" });
rec.failed = true;
}
renderResults(run, true); // live: строка появляется сразу после теста
}
if (stopFlag) break;
}
mount(progressHost);
running = false;
runBtn.disabled = false;
stopBtn.style.display = "none";
if (!stopFlag) {
pushHistory(run);
// финальный аккорд: короткий вердикт с процентами и деньгами
const first = run.models[Object.keys(run.models)[0]];
if (first) {
const st = stats(first);
const judged = first.results.filter((r) => r.taskOk != null);
const cls = judged.length ? ` · ${judged.filter((r) => r.taskOk).length}/${judged.length}` : "";
const total = Object.values(run.models).reduce((a, rec) => a + stats(rec).cost, 0);
toast(t("bm.verdict", { p: Math.round(st.score * 100) + "%" + cls, s: money(total) }), "ok", 6000);
}
}
renderResults(run);
renderHistory();
});
function statusIcon(s) {
return s === "pass" ? "✅" : s === "partial" ? "◐" : "❌";
}
function stats(rec) {
const n = rec.results.length || 1;
return {
score: rec.results.reduce((a, r) => a + r.score, 0) / n,
tokS: rec.tokS.length ? rec.tokS.reduce((a, b) => a + b, 0) / rec.tokS.length : null,
ttft: rec.ttft.length ? rec.ttft.reduce((a, b) => a + b, 0) / rec.ttft.length : null,
cost: rec.results.reduce((a, r) => a + (r.cost || 0), 0),
};
}
function renderResults(run, live) {
const models = Object.keys(run.models);
if (!models.length) {
mount(resultsHost);
return;
}
const routing = run.mode === "routing";
const detail = h("div", {});
let summary;
if (routing) {
const rec = run.models[models[0]];
const st = stats(rec);
const judged = rec.results.filter((r) => r.taskOk != null);
const okCount = judged.filter((r) => r.taskOk).length;
// какие модели реально выбирал роутер
const dist = {};
for (const r of rec.results) {
if (!r.chosen) continue;
dist[r.chosen] = dist[r.chosen] || { n: 0, cost: 0 };
dist[r.chosen].n++;
dist[r.chosen].cost += r.cost || 0;
}
const pct = Math.round(st.score * 100);
summary = h("div", {},
h("div", { class: "kpis", style: "grid-template-columns: repeat(auto-fit, minmax(150px, 1fr))" },
h("div", { class: "kpi" }, h("div", { class: "big" }, pct + "%"), h("div", { class: "lbl" }, t("bm.answersOk"))),
h("div", { class: "kpi" }, h("div", { class: "big" }, judged.length ? `${okCount}/${judged.length}` : "—"), h("div", { class: "lbl" }, t("bm.classAcc"))),
h("div", { class: "kpi" }, h("div", { class: "big" }, st.ttft != null ? ms(Math.round(st.ttft)) : "—"), h("div", { class: "lbl" }, t("pg.ttft"))),
h("div", { class: "kpi" }, h("div", { class: "big" }, st.tokS != null ? st.tokS.toFixed(1) : "—"), h("div", { class: "lbl" }, "tok/s")),
h("div", { class: "kpi" }, h("div", { class: "big" }, moneyEl(st.cost)), h("div", { class: "lbl" }, t("bm.spent")))),
h("div", { class: "flex wrap", style: "gap:6px; margin-top:4px" },
h("span", { class: "muted small" }, t("bm.modelDist") + ":"),
...Object.entries(dist).map(([m, d]) =>
h("span", { class: "badge" }, `${m} × ${d.n}`, d.cost > 0 ? " · " : "", d.cost > 0 ? moneyEl(d.cost) : ""))));
} else {
const sorted = models.slice().sort((a, b) => stats(run.models[b]).score - stats(run.models[a]).score);
summary = h("div", {},
...sorted.map((m) => {
const st = stats(run.models[m]);
const pct = Math.round(st.score * 100);
return h("div", { class: "bm-sum" },
h("div", { class: "flex" },
h("b", { class: "mono grow" }, m),
h("span", { class: "badge " + (pct >= 80 ? "ok" : pct >= 50 ? "" : "bad") }, pct + "%"),
st.tokS != null ? h("span", { class: "muted small", style: "margin-left:8px" }, st.tokS.toFixed(1) + " tok/s") : null,
st.ttft != null ? h("span", { class: "muted small", style: "margin-left:8px" }, "TTFT " + ms(Math.round(st.ttft))) : null,
st.cost > 0 ? h("span", { class: "muted small", style: "margin-left:8px" }, t("bm.spent") + " ", moneyEl(st.cost)) : null),
h("div", { class: "sysbar" }, h("i", { class: pct < 50 ? "bad" : "", style: `width:${pct}%` })));
}));
}
function answerCell(r, m) {
const cell = h("td", { class: "bm-cell", title: r.answer || r.note },
statusIcon(r.status) + " ",
h("span", { class: "muted small" }, r.note + (r.latency ? ` · ${(r.latency / 1000).toFixed(1)}s` : "")));
cell.addEventListener("click", () => {
mount(detail, h("div", { class: "card" },
h("div", { class: "card-title" }, `${r.id} · ${m} · ${statusIcon(r.status)}`),
h("pre", { class: "conv-log", style: "max-height:200px" }, r.answer || "—")));
});
return cell;
}
let header, rows;
if (routing) {
const rec = run.models[models[0]];
header = h("tr", {},
h("th", {}, t("bm.test")),
h("th", {}, t("pg.task")),
h("th", {}, t("pg.tier")),
h("th", {}, t("pg.model")),
h("th", {}, t("bm.answer")));
rows = rec.results.map((r) =>
h("tr", {},
h("td", {}, h("div", {}, r.id), h("div", { class: "muted small" }, r.cat)),
h("td", {},
r.degraded
? h("span", { class: "muted", title: t("bm.routerDegraded") }, "⚠ " + t("bm.degraded"))
: r.taskOk == null
? h("span", { class: "muted" }, r.task || "—")
: h("span", { class: r.taskOk ? "" : "err" },
(r.taskOk ? "✓ " : "✗ ") + r.task,
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
h("td", {},
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
h("td", {},
h("span", { class: "mono small" }, r.chosen || "—"),
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
r.source && r.source.includes("task-rule") ? h("div", { class: "badge", style: "margin-top:2px" }, "task-rule") : null),
answerCell(r, r.chosen || "cortiq-auto")));
} else {
header = h("tr", {}, h("th", {}, t("bm.test")), ...models.map((m) => h("th", { class: "mono" }, m)));
rows = (run.models[models[0]].results || []).map((_, i) =>
h("tr", {},
h("td", {},
h("div", {}, run.models[models[0]].results[i].id),
h("div", { class: "muted small" }, run.models[models[0]].results[i].cat)),
...models.map((m) => {
const r = run.models[m].results[i];
return r ? answerCell(r, m) : h("td", {}, "—");
})));
}
mount(resultsHost,
h("div", { class: "card" },
h("div", { class: "card-head" },
h("h3", {}, (routing ? "🧭 " : "🎯 ") + t("bm.results")),
h("span", { class: "card-head sub" },
(live ? t("bm.liveRunning") + " · " : "") + new Date(run.ts).toLocaleString() + (run.quick ? " · " + t("bm.quick") : ""))),
summary,
h("div", { class: "table-wrap", style: "margin-top:14px" }, h("table", {}, h("thead", {}, header), h("tbody", {}, ...rows))),
h("div", { class: "muted small", style: "margin-top:6px" }, t("bm.cellHint"))),
detail);
}
function renderHistory() {
const l = history();
if (!l.length) {
mount(historyHost);
return;
}
const clearAll = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("bm.clearHistory"));
clearAll.addEventListener("click", () => {
saveHistory([]);
renderHistory();
});
mount(historyHost, h("div", { class: "card" },
h("div", { class: "card-head" },
h("h3", {}, t("bm.history")),
h("div", { class: "right" }, clearAll)),
...l.map((run, idx) => {
const models = Object.keys(run.models);
const scoreOf = (m) => {
const rs = run.models[m].results || [];
return rs.length ? rs.reduce((a, r) => a + r.score, 0) / rs.length : 0;
};
const best = models.slice().sort((a, b) => scoreOf(b) - scoreOf(a))[0];
const row = h("div", { class: "flex bench-row", style: "cursor:pointer" },
h("span", {}, run.mode === "routing" ? "🧭" : "🎯"),
h("span", { class: "muted small" }, new Date(run.ts).toLocaleString()),
h("span", { class: "grow mono small" }, run.mode === "routing" ? t("bm.mode.routing") : models.join(", ")),
run.quick ? h("span", { class: "badge" }, t("bm.quick")) : null,
best ? h("b", {}, Math.round(scoreOf(best) * 100) + "%") : null,
h("button", { class: "btn sm ghost danger", title: t("common.delete"), onClick: (e) => {
e.stopPropagation();
l.splice(idx, 1);
saveHistory(l);
renderHistory();
} }, "×"));
row.addEventListener("click", () => renderResults(run));
return row;
})));
}
// ── custom tests (stored in the browser) ──
function customCard() {
const host = h("div", { class: "card" });
function paint() {
const list = customTests();
const promptIn = h("input", { placeholder: t("bm.promptPh") });
const checkIn = h("input", { placeholder: t("bm.checkPh") });
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("bm.addTest"));
addBtn.addEventListener("click", () => {
const prompt = promptIn.value.trim();
const check = checkIn.value.split(",").map((s) => s.trim()).filter(Boolean);
if (!prompt || !check.length) return;
list.push({ id: "custom-" + (list.length + 1), cat: "custom", custom: true, max: 128, prompt, check });
saveCustom(list);
paint();
});
mount(host,
h("div", { class: "card-title" }, t("bm.custom")),
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bm.customHint")),
...list.map((tst, i) =>
h("div", { class: "flex bench-row" },
h("span", { class: "grow small" }, tst.prompt.slice(0, 90)),
h("span", { class: "muted small mono" }, tst.check.join(", ").slice(0, 40)),
h("button", { class: "btn sm ghost danger", onClick: () => { list.splice(i, 1); saveCustom(list); paint(); } }, "×"))),
h("div", { class: "flex", style: "gap:8px; margin-top:8px" }, promptIn, checkIn, addBtn));
}
paint();
return host;
}
mount(root,
h("div", { class: "page-head" },
h("h2", {}, t("bm.title")),
h("p", {}, t("bm.subtitle"))),
ids.length === 0 ? h("div", { class: "callout" }, t("routing.empty")) : null,
h("div", { class: "card" },
modeSeg,
modeDesc,
modelsPick,
h("div", { class: "flex", style: "justify-content:space-between; margin-top:14px" },
h("label", { class: "check", style: "margin:0", title: t("bm.quickHint") }, quickCb, "⚡ " + t("bm.quickMode")),
h("div", { class: "flex", style: "gap:8px" }, stopBtn, runBtn)),
h("div", { class: "muted small", style: "margin-top:6px" },
t("bm.suiteInfo", { q: SUITE.filter((x) => x.quick).length, f: SUITE.length }))),
progressHost,
resultsHost,
customCard(),
historyHost);
renderHistory();
return root;
}
+97 -6
View File
@@ -1,6 +1,6 @@
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
import { h, mount, money, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { t, trTier } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
import { SITE_URL } from "../app.js";
@@ -27,14 +27,96 @@ export async function renderDashboard() {
let groupby = "model";
const root = h("div");
// ── host metrics: CPU sparkline + RAM / disk bars, refreshed every 2 s ──
const sysHost = h("div", {});
const cpuHistory = []; // last ~60 samples (2 min)
function fmtG(n) {
return (n / 1e9).toFixed(1) + " GB";
}
function bar(frac, cls) {
return h("div", { class: "sysbar" }, h("i", { class: cls || "", style: `width:${Math.min(100, frac * 100)}%` }));
}
function sparkline(values, w, hgt) {
const c = document.createElement("canvas");
c.width = w;
c.height = hgt;
const ctx = c.getContext("2d");
const accent = getComputedStyle(document.documentElement).getPropertyValue("--accent").trim() || "#5b8def";
ctx.strokeStyle = accent;
ctx.lineWidth = 1.5;
ctx.beginPath();
const n = Math.max(values.length, 2);
values.forEach((v, i) => {
const x = (i / (n - 1)) * (w - 2) + 1;
const y = hgt - 2 - (Math.min(v, 100) / 100) * (hgt - 4);
if (i === 0) ctx.moveTo(x, y);
else ctx.lineTo(x, y);
});
ctx.stroke();
return c;
}
async function refreshSystem() {
let sys;
try {
sys = await api.system();
} catch {
return;
}
if (!sys.ts) return;
cpuHistory.push(sys.cpu_percent || 0);
if (cpuHistory.length > 60) cpuHistory.shift();
const ramFrac = sys.ram_total ? sys.ram_used / sys.ram_total : 0;
const diskFrac = sys.disk_total ? (sys.disk_total - sys.disk_free) / sys.disk_total : 0;
mount(
sysHost,
h(
"div",
{ class: "card" },
h("div", { class: "card-title" }, t("dash.sys.title")),
h(
"div",
{ class: "sysgrid" },
h("div", { class: "sysitem" },
h("div", { class: "flex" },
h("span", { class: "k grow" }, `CPU · ${sys.cores} ${t("dash.sys.cores")}`),
h("span", { class: "v mono" }, (sys.cpu_percent || 0).toFixed(0) + "%")),
sparkline(cpuHistory, 220, 36),
h("div", { class: "muted small" }, `load ${Number(sys.load1 || 0).toFixed(1)}`)),
h("div", { class: "sysitem" },
h("div", { class: "flex" },
h("span", { class: "k grow" }, "RAM"),
h("span", { class: "v mono" }, `${fmtG(sys.ram_used)} / ${fmtG(sys.ram_total)}`)),
bar(ramFrac, ramFrac > 0.9 ? "bad" : ""),
h("div", { class: "muted small" }, t("dash.sys.gateway") + " " + (sys.process_rss / 1e6).toFixed(0) + " MB")),
h("div", { class: "sysitem" },
h("div", { class: "flex" },
h("span", { class: "k grow" }, t("dash.sys.disk")),
h("span", { class: "v mono" }, t("dash.sys.free") + " " + fmtG(sys.disk_free)),
),
bar(diskFrac, diskFrac > 0.92 ? "bad" : ""),
h("div", { class: "muted small" }, `${fmtG(sys.disk_total - sys.disk_free)} / ${fmtG(sys.disk_total)}`))
)
)
);
}
const sysTimer = setInterval(() => {
if (!root.isConnected) {
clearInterval(sysTimer);
return;
}
refreshSystem();
}, 2000);
refreshSystem();
async function load() {
const rangeSecs = range;
let stats, recent, health;
let stats, recent, health, balance;
try {
[stats, recent, health] = await Promise.all([
[stats, recent, health, balance] = await Promise.all([
api.stats(rangeSecs, groupby),
api.requests(40, 0),
api.health().catch(() => null),
api.balance().catch(() => null),
]);
} catch (e) {
mount(root, h("div", { class: "callout" }, String(e.message || e)));
@@ -80,12 +162,19 @@ export async function renderDashboard() {
{ class: "kpis" },
kpi(num(tot.requests), t("dash.kpi.requests")),
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
kpi(money(tot.cost_usd), t("dash.kpi.cost")),
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
balance && balance.openrouter_usd != null
? kpi(
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
t("dash.kpi.orBalance"))
: null,
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
kpi(pct(tot.success_rate), t("dash.kpi.success")),
kpi(num(tot.failovers), t("dash.kpi.failovers"))
),
sysHost,
h(
"div",
{ class: "grid cols-2" },
@@ -188,7 +277,9 @@ function recentTable(rows) {
{},
h("td", { class: "mono" }, timeAgo(r.ts)),
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
h("td", {},
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
h("td", {}, r.task_label || "—"),
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
h("td", { class: "mono" }, money(r.cost_usd)),
+16 -2
View File
@@ -78,7 +78,19 @@ function descLine(desc) {
return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null;
}
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v) }.
/// Disk/RAM sanity for a model of `bytes` size against the live system
/// snapshot (ctx.sys): running a model needs it in RAM; downloading needs disk.
function fitWarnings(bytes, sys) {
const out = [];
if (!bytes || !sys) return out;
if (sys.disk_free && bytes > sys.disk_free)
out.push(h("span", { class: "badge bad", title: t("fit.diskTitle") }, "⚠ " + t("fit.disk")));
else if (sys.ram_total && bytes > sys.ram_total * 0.75)
out.push(h("span", { class: "badge warn", title: t("fit.ramTitle") }, "⚠ " + t("fit.ram")));
return out;
}
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v), sys? }.
/// Live progress lands in the per-variant host so a poll can repaint it in place,
/// without re-rendering the grid under the user's cursor.
export function featuredCard(m, ctx) {
@@ -101,6 +113,7 @@ export function featuredCard(m, ctx) {
h("div", { class: "hf-meta" },
h("span", { class: "badge" }, badge),
descBadge(desc),
...fitWarnings(m.size_bytes, ctx.sys),
isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null),
descLine(desc),
h("div", { class: "muted small" }, m.id),
@@ -149,7 +162,8 @@ export function featuredCard(m, ctx) {
h("div", { class: "hf-meta" },
h("span", { class: "badge" }, badge),
size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null,
descBadge(desc)
descBadge(desc),
...(done0 ? [] : fitWarnings((v0 && v0.size_bytes) || m.size_bytes, ctx.sys))
),
descLine(desc),
h("div", { class: "muted small" }, m.id),
+3 -1
View File
@@ -424,8 +424,10 @@ export function renderImport() {
async function renderFeatured(reload = true) {
if (reload) featured = await loadFeatured();
let sys = null;
try { sys = await api.system(); } catch { /* metrics optional */ }
featuredHosts = new Map();
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured };
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured, sys };
mount(featuredGrid, ...(featured.length
? featured.map((m) => featuredCard(m, ctx))
: [h("div", { class: "muted pad" }, t("import.noResults"))]));
+329 -14
View File
@@ -1,30 +1,23 @@
// API keys — list, create (with copy-once), revoke.
import { h, mount, modal, toast, confirmDialog } from "../ui.js";
import { h, mount, modal, toast, confirmDialog, moneyEl, num } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
export async function renderKeys() {
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
const root = h("div");
let usageRange = "7d";
function openCreate(reload) {
const accountIn = h("input", { placeholder: "team-a" });
const rateIn = h("input", { type: "number", value: 600 });
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
const customIn = h("input", { placeholder: "sk-gw-…" });
const allowInputs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox" }) }));
const body = h(
"div",
{},
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
h(
"label",
{ class: "field" },
h("span", {}, t("keys.form.allow")),
h("div", { class: "flex wrap" }, ...allowInputs.map(({ id, cb }) => h("label", { class: "check" }, cb, id))),
h("div", { class: "hint" }, t("keys.form.allowHint"))
),
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
);
modal(t("keys.form.title"), body, async () => {
@@ -33,7 +26,8 @@ export async function renderKeys() {
key: customIn.value.trim(),
account: accountIn.value.trim() || "default",
rate_per_min: parseInt(rateIn.value) || 0,
allow_models: allowInputs.filter(({ cb }) => cb.checked).map(({ id }) => id),
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
allow_models: [],
});
reload();
showCreated(r.key);
@@ -48,11 +42,310 @@ export async function renderKeys() {
const input = h("input", { value: key, readonly: true, class: "mono" });
const copyBtn = h("button", { class: "btn", onClick: () => { input.select(); navigator.clipboard?.writeText(key); toast(t("common.copied"), "good"); } }, t("common.copy"));
modal(t("common.ok"), h("div", {}, h("p", {}, t("keys.created")), h("div", { class: "flex" }, input, copyBtn)), null);
lastCreatedKey = key;
reload();
}
let lastCreatedKey = null;
// "Connect your app" — copy-paste snippets with this gateway's real URL.
// Keys are stored hashed-masked in the list, so the key itself is a
// placeholder unless one was just created on this page.
function snippetsCard(openMode) {
const origin = location.origin;
const base = origin + "/v1";
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
// Every scripty snippet exists in two OS flavors: bash heredocs for
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
// (python/js) and GUI instructions are OS-independent.
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
const buildSnips = (osKind) => osKind === "ps" ? {
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
$env:ANTHROPIC_MODEL = "cortiq-auto"
claude`,
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
Add-Content "$HOME\\.codex\\config.toml" @'
model = "cortiq-auto"
model_provider = "cortiq"
[model_providers.cortiq]
name = "Cortiq Gateway"
base_url = "${base}"
env_key = "CORTIQ_API_KEY"
wire_api = "chat"
'@
$env:CORTIQ_API_KEY = "${key}"
codex`,
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"cortiq": {
"npm": "@ai-sdk/openai-compatible",
"name": "Cortiq Gateway",
"options": { "baseURL": "${base}", "apiKey": "${key}" },
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
}
}
}
'@
opencode`,
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
Add-Content "$HOME\\.continue\\config.yaml" @'
models:
- name: Cortiq Auto
provider: openai
model: cortiq-auto
apiBase: ${base}
apiKey: ${key}
'@`,
Cline: `${t("keys.h.cline1")}
API Provider: OpenAI Compatible
Base URL: ${base}
API Key: ${key}
Model ID: cortiq-auto
${t("keys.h.cline2")}`,
curl: `curl.exe ${base}/chat/completions \`
-H "Authorization: Bearer ${key}" \`
-H "Content-Type: application/json" \`
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
} : {
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
export ANTHROPIC_AUTH_TOKEN="${key}"
export ANTHROPIC_MODEL="cortiq-auto"
claude`,
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
model = "cortiq-auto"
model_provider = "cortiq"
[model_providers.cortiq]
name = "Cortiq Gateway"
base_url = "${base}"
env_key = "CORTIQ_API_KEY"
wire_api = "chat"
EOF
export CORTIQ_API_KEY="${key}"
codex`,
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"cortiq": {
"npm": "@ai-sdk/openai-compatible",
"name": "Cortiq Gateway",
"options": { "baseURL": "${base}", "apiKey": "${key}" },
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
}
}
}
EOF
opencode`,
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
models:
- name: Cortiq Auto
provider: openai
model: cortiq-auto
apiBase: ${base}
apiKey: ${key}
EOF`,
Cline: `${t("keys.h.cline1")}
API Provider: OpenAI Compatible
Base URL: ${base}
API Key: ${key}
Model ID: cortiq-auto
${t("keys.h.cline2")}`,
curl: `curl ${base}/chat/completions \\
-H "Authorization: Bearer ${key}" \\
-H "Content-Type: application/json" \\
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
python: `from openai import OpenAI
client = OpenAI(base_url="${base}", api_key="${key}")
resp = client.chat.completions.create(
model="cortiq-auto", # авто-роутинг; или точное имя модели
messages=[{"role": "user", "content": "Привет!"}],
)
print(resp.choices[0].message.content)`,
javascript: `import OpenAI from "openai";
const client = new OpenAI({ baseURL: "${base}", apiKey: "${key}" });
const resp = await client.chat.completions.create({
model: "cortiq-auto",
messages: [{ role: "user", content: "Привет!" }],
});
console.log(resp.choices[0].message.content);`,
};
let shSnips, psSnips, snips;
function rebuildSnips() {
shSnips = buildSnips("sh");
psSnips = buildSnips("ps");
// python/js/Cline одинаковы — дополняем ps-набор из sh
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
snips = os === "ps" ? psSnips : shSnips;
}
rebuildSnips();
let cur = "Claude Code";
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
// in a pasted snippet is not a comment in default interactive zsh, and a
// "!" inside it triggers history expansion ("event not found")
const HINTS = {
"Claude Code": () => t("keys.h.claude"),
Codex: () => t("keys.h.codex"),
OpenCode: () => t("keys.h.opencode"),
Continue: () => t("keys.h.continue"),
};
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
paintHint();
const seg = h(
"div",
{ class: "seg" },
...Object.keys(snips).map((k, i) =>
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
cur = k;
pre.textContent = snips[k];
paintHint();
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
paintStatus();
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
e.currentTarget.classList.add("active");
} }, k)
)
);
// Claude Code needs the anthropic_messages protocol — offer a one-click
// enable when it is off, right where the user will hit the problem.
const protoWarn = h("div", { style: "display:none; margin-top:8px" });
api.getProtocols().then((prot) => {
if (prot.anthropic_messages) return;
const fixBtn = h("button", { class: "btn sm primary" }, t("keys.h.enableProto"));
fixBtn.addEventListener("click", async () => {
try {
await api.putProtocols({ ...prot, anthropic_messages: true });
protoWarn.style.display = "none";
toast(t("common.saved"), "ok");
} catch (e) {
toast(String(e.message || e), "err");
}
});
mount(protoWarn, h("div", { class: "callout warn" }, h("span", {}, "⚠ " + t("keys.h.protoOff")), " ", fixBtn));
if (cur === "Claude Code") protoWarn.style.display = "";
}).catch(() => {});
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
// status line under the snippet ("installed" / "already configured") —
// and a one-line INSTALL command (per OS) when it's missing.
const INSTALL = {
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
};
let detected = {};
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
function paintStatus() {
const d = detected[cur];
if (!d) {
statusHost.textContent = "";
return;
}
statusHost.innerHTML = "";
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
else {
const cmd = (INSTALL[cur] || {})[os];
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
await navigator.clipboard.writeText(cmd);
toast(t("common.copied"), "good");
} }, "⧉");
mount(statusHost,
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
h("span", { class: "muted" }, t("keys.h.install")),
h("code", { class: "mono small cmd-chip" }, cmd),
copyInstall) : null);
}
}
api.harnesses().then((d) => {
detected = d || {};
seg.querySelectorAll("button").forEach((b) => {
const st = detected[b.dataset.snip];
if (st && st.installed) b.append(" ✓");
});
paintStatus();
}).catch(() => {});
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
const osSeg = h(
"div",
{ class: "seg" },
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
);
osSeg.querySelectorAll("button").forEach((b, i) =>
b.addEventListener("click", () => {
os = i === 0 ? "sh" : "ps";
rebuildSnips();
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
pre.textContent = snips[cur];
paintStatus();
}));
// paste-your-key: substitutes the key into every snippet immediately
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
const applyKey = () => {
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
rebuildSnips();
pre.textContent = snips[cur];
};
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
copy.addEventListener("click", async () => {
await navigator.clipboard.writeText(pre.textContent);
toast(t("common.copied"), "ok");
});
return h(
"div",
{ class: "card" },
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" },
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
hintHost,
pre,
statusHost,
protoWarn,
h("div", { class: "muted small", style: "margin-top:6px" }, t("keys.h.note"))
);
}
async function reload() {
const data = await api.listKeys();
const keys = data.keys || [];
// per-account usage: window breakdown + cumulative-forever totals — the
// "who spends what" view for departments/developers sharing one gateway
let usage = {};
let alltime = {};
try {
const st = await api.stats(usageRange, "account");
for (const b of st.breakdown || []) usage[b.key] = b;
alltime = st.alltime_by_account || {};
} catch { /* stats disabled — columns show dashes */ }
mount(
root,
h(
@@ -62,6 +355,9 @@ export async function renderKeys() {
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("keys.title")), h("p", {}, t("keys.subtitle"))),
h("div", { class: "seg", style: "margin-right:10px" },
...["24h", "7d", "30d"].map((r) =>
h("button", { class: r === usageRange ? "active" : "", onClick: () => { usageRange = r; reload(); } }, r))),
h("button", { class: "btn primary", onClick: () => openCreate(reload) }, "+ " + t("keys.add"))
)
),
@@ -80,7 +376,16 @@ export async function renderKeys() {
h(
"thead",
{},
h("tr", {}, h("th", {}, t("keys.col.key")), h("th", {}, t("keys.col.account")), h("th", {}, t("keys.col.rate")), h("th", {}, t("keys.col.models")), h("th", { class: "right-align" }, t("common.actions")))
h("tr", {},
h("th", {}, t("keys.col.key")),
h("th", {}, t("keys.col.account")),
h("th", {}, t("keys.col.rate")),
h("th", {}, t("keys.col.models")),
h("th", { class: "right-align" }, t("keys.col.requests") + " (" + usageRange + ")"),
h("th", { class: "right-align" }, t("keys.col.tokens") + " (" + usageRange + ")"),
h("th", { class: "right-align" }, "$ (" + usageRange + ")"),
h("th", { class: "right-align" }, t("keys.col.costAll")),
h("th", { class: "right-align" }, t("common.actions")))
),
h(
"tbody",
@@ -93,6 +398,15 @@ export async function renderKeys() {
h("td", {}, k.account),
h("td", { class: "mono" }, k.rate_per_min || "—"),
h("td", {}, (k.allow_models && k.allow_models.length) ? k.allow_models.join(", ") : h("span", { class: "muted" }, t("keys.allowAll"))),
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
h("td", { class: "right-align" },
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
k.monthly_budget_usd
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
: null),
h(
"td",
{ class: "right-align" },
@@ -119,7 +433,8 @@ export async function renderKeys() {
)
)
)
)
),
snippetsCard(!!data.open_mode)
);
}
+95
View File
@@ -0,0 +1,95 @@
// Gateway log viewer — the in-memory ring served by /admin/api/logs, for
// everyone who never sees the terminal (Docker, autostart, double-clicked exe).
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
export function renderLogs() {
const root = h("div");
let timer = null;
let paused = false;
let filter = "";
let level = "";
const pre = h("pre", { class: "conv-log logs-view" }, t("common.loading"));
const filterIn = h("input", { placeholder: t("logs.filterPlaceholder"), style: "max-width:260px" });
const levelSel = h(
"select",
{ style: "max-width:130px" },
h("option", { value: "" }, t("common.all")),
...["ERROR", "WARN", "INFO", "DEBUG"].map((l) => h("option", { value: l }, l))
);
const pauseBtn = h("button", { class: "btn ghost sm" }, "⏸ " + t("logs.pause"));
const copyBtn = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
const clearBtn = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("dash.clear"));
clearBtn.addEventListener("click", async () => {
try {
await api.clearLogs();
lines = [];
paint();
toast(t("dash.cleared"), "ok");
} catch (e) {
toast(String(e.message || e), "err");
}
});
let lines = [];
function paint() {
let shown = lines;
if (level) shown = shown.filter((l) => l.includes(` ${level} `) || l.includes(`${level} `));
if (filter) shown = shown.filter((l) => l.toLowerCase().includes(filter));
pre.textContent = shown.length ? shown.join("\n") : t("logs.empty");
if (!paused) pre.scrollTop = pre.scrollHeight;
}
async function refresh() {
if (paused) return;
try {
lines = (await api.logs(500)).lines || [];
paint();
} catch {
/* transient */
}
}
filterIn.addEventListener("input", () => {
filter = filterIn.value.trim().toLowerCase();
paint();
});
levelSel.addEventListener("change", () => {
level = levelSel.value;
paint();
});
pauseBtn.addEventListener("click", () => {
paused = !paused;
pauseBtn.textContent = (paused ? "▶ " : "⏸ ") + t(paused ? "logs.resume" : "logs.pause");
if (!paused) refresh();
});
copyBtn.addEventListener("click", async () => {
try {
await navigator.clipboard.writeText(pre.textContent);
toast(t("common.copied"), "ok");
} catch (e) {
toast(String(e.message || e), "err");
}
});
timer = setInterval(() => {
if (!root.isConnected) {
clearInterval(timer);
return;
}
refresh();
}, 2000);
refresh();
mount(
root,
h("div", { class: "view-head" }, h("h2", {}, t("nav.logs")), h("div", { class: "muted" }, t("logs.subtitle"))),
h(
"div",
{ class: "card" },
h("div", { class: "flex", style: "gap:8px; flex-wrap:wrap; margin-bottom:10px" }, filterIn, levelSel, pauseBtn, copyBtn, clearBtn),
pre
)
);
return root;
}
+54 -1
View File
@@ -315,10 +315,63 @@ export async function renderModels() {
)
)
)
)
),
benchCard()
);
}
// ── one-click benchmark of local .cmf files (cortiq bench --json) ──
function benchCard() {
const host = h("div", { class: "card" }, h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
h("div", { class: "empty" }, t("common.loading")));
let timer = null;
async function paint() {
let files = [];
let runs = {};
try {
files = (await api.mediaModels()).models || [];
for (const r of (await api.benchList()).runs || []) runs[r.model] = r;
} catch {
return;
}
const chat = files.filter((m) => m.modality === "chat");
const anyRunning = Object.values(runs).some((r) => r.state === "running");
const rows = chat.map((m) => {
const r = runs[m.name];
const go = h("button", { class: "btn sm ghost", disabled: anyRunning ? true : null }, "⏱ " + t("bench.run"));
go.addEventListener("click", async () => {
try {
await api.benchStart(m.name);
paint();
if (!timer) timer = setInterval(() => {
if (!host.isConnected) { clearInterval(timer); return; }
paint();
}, 3000);
} catch (e) {
toast(String(e.message || e), "bad");
}
});
let result = h("span", { class: "muted" }, "—");
if (r?.state === "running") result = h("span", { class: "flex" }, h("span", { class: "spinner" }), " ", t("bench.running"));
else if (r?.state === "done") result = h("b", { class: "mono" }, (r.tok_s != null ? r.tok_s.toFixed(1) : "?") + " tok/s");
else if (r?.state === "error") result = h("span", { class: "err", title: (r.log || []).slice(-2).join(" · ") }, t("bench.failed"));
return h("div", { class: "flex bench-row" },
h("span", { class: "mono grow" }, m.name),
h("span", { class: "muted small" }, (m.size_bytes / 1e9).toFixed(2) + " GB"),
result,
go);
});
mount(host,
h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
...(rows.length ? rows : [h("div", { class: "empty" }, t("bench.none"))]));
if (!anyRunning && timer) { clearInterval(timer); timer = null; }
}
paint();
return host;
}
await reload();
return root;
}
+5 -2
View File
@@ -45,6 +45,7 @@ export function renderOnboarding() {
const root = h("div", { class: "onb-wrap" });
let jobsPoll = null;
let jobs = [];
let sysSnapshot = null; // disk/RAM для плашек "не влезет" на карточках
let jobStates = new Map(); // job id → last seen state, to spot completions
let hosts = new Map(); // job id → card progress host (rebuilt on every render)
let jobsHost = h("div", {});
@@ -131,6 +132,7 @@ export function renderOnboarding() {
async function loadFeatured() {
featured = await fetchFeatured();
try { sysSnapshot = await api.system(); } catch { /* optional */ }
featuredLoaded = true;
}
@@ -199,7 +201,7 @@ export function renderOnboarding() {
);
} else if (step === 1) {
const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip")));
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete };
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete, sys: sysSnapshot };
// the list is fetched from HuggingFace — say so instead of showing a blank grid
const grid = featured.length
? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx)))
@@ -290,7 +292,8 @@ export function renderOnboarding() {
// let the toast be readable before the reload swaps the page
await new Promise((res) => setTimeout(res, 900));
}
location.hash = "#/models";
// land where the reward is: try the model right away
location.hash = "#/playground";
location.reload();
});
const skipBtn = h("button", { class: "btn ghost", onClick: async () => {
+544 -39
View File
@@ -1,17 +1,64 @@
// Playground — send a prompt through the live pipeline, inspect routing decision.
import { h, mount, money, ms } from "../ui.js";
import { t } from "../i18n.js";
import { api, testStream } from "../api.js";
// Playground — three tabs over the live gateway:
// chat → the routing pipeline (with streaming, token counts, tok/s, TTFT)
// image → `cortiq imagine` over a local text-to-image .cmf (Lumina)
// video → `cortiq animate` over a local text-to-video .cmf (MiniMax-H3)
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
import { h, mount, toast, money, moneyEl, ms } from "../ui.js";
import { t, trTier } from "../i18n.js";
import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
import { appState } from "../app.js";
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
function fmtBytes(n) {
if (!n && n !== 0) return "—";
if (n >= 1e9) return (n / 1e9).toFixed(2) + " GB";
if (n >= 1e6) return (n / 1e6).toFixed(1) + " MB";
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
return n + " B";
}
function kv(k, v) {
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
}
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
}
export async function renderPlayground() {
const meta = appState.meta || (await api.meta());
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
let mediaModels = [];
try {
mediaModels = (await api.mediaModels()).models || [];
} catch {
/* media API unavailable — tabs will show the hint */
}
const imageModels = mediaModels.filter((m) => m.modality === "image");
const videoModels = mediaModels.filter((m) => m.modality === "video");
let mode = "auto";
const root = h("div");
let tab = "chat";
// ── tabs ──
const tabsSeg = h(
"div",
{ class: "seg" },
h("button", { class: "active", "data-tab": "chat" }, "💬 " + t("pg.tab.chat")),
h("button", { "data-tab": "image" }, "🖼 " + t("pg.tab.image")),
h("button", { "data-tab": "video" }, "🎬 " + t("pg.tab.video"))
);
const panelHost = h("div", {});
tabsSeg.querySelectorAll("button").forEach((b) =>
b.addEventListener("click", () => {
tab = b.dataset.tab;
tabsSeg.querySelectorAll("button").forEach((x) => x.classList.toggle("active", x === b));
mount(panelHost, panels[tab]);
})
);
// ════════════════════════════ chat panel ════════════════════════════
let mode = "auto";
const sysIn = h("textarea", { placeholder: t("pg.systemPlaceholder"), rows: 2 });
const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 });
const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 });
@@ -44,16 +91,33 @@ export async function renderPlayground() {
modeExtra = next;
}
const out = h("div", { class: "answer" }, h("div", { class: "empty" }, t("pg.empty")));
const chatLog = []; // the running conversation: {role, content}
const out = h("div", { class: "answer chat-log" }, h("div", { class: "empty" }, t("pg.empty")));
const decision = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
function bubble(role, text) {
return h("div", { class: "chat-msg " + role }, h("div", { class: "chat-bubble" }, text));
}
function renderChat(liveText) {
const items = chatLog.map((m) => bubble(m.role, m.content));
if (liveText != null) items.push(bubble("assistant", liveText || "…"));
mount(out, ...(items.length ? items : [h("div", { class: "empty" }, t("pg.empty"))]));
out.scrollTop = out.scrollHeight;
}
const clearBtn = h("button", { class: "btn ghost sm" }, "🧹 " + t("pg.clearChat"));
clearBtn.addEventListener("click", () => {
chatLog.length = 0;
renderChat();
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
});
const sendBtn = h("button", { class: "btn primary" }, t("pg.send"));
sendBtn.addEventListener("click", async () => {
const prompt = promptIn.value.trim();
if (!prompt) return;
const messages = [];
if (sysIn.value.trim()) messages.push({ role: "system", content: sysIn.value.trim() });
messages.push({ role: "user", content: prompt });
messages.push(...chatLog, { role: "user", content: prompt });
let model = "cortiq-auto";
if (mode === "auto") {
if (profileSel.value) model = "cortiq-auto:" + profileSel.value;
@@ -62,7 +126,9 @@ export async function renderPlayground() {
}
sendBtn.disabled = true;
sendBtn.textContent = t("pg.sending");
mount(out, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.sending")));
chatLog.push({ role: "user", content: prompt });
promptIn.value = "";
renderChat("");
const body = {
model,
messages,
@@ -73,12 +139,13 @@ export async function renderPlayground() {
if (streamCb.checked) {
const started = performance.now();
let acc = "";
out.textContent = "";
const info = await testStream(body, (d) => {
acc += d;
out.textContent = acc;
renderChat(acc);
});
out.textContent = acc || "—";
chatLog.push({ role: "assistant", content: acc || "—" });
renderChat();
const latency = Math.round(performance.now() - started);
renderDecision({
cortiq: {
task_label: info.task_label,
@@ -88,37 +155,58 @@ export async function renderPlayground() {
cost_usd: info.cost_usd,
failover: false,
},
usage: {},
latency_ms: Math.round(performance.now() - started),
usage: info.usage || {},
usage_estimated: info.usage_estimated,
latency_ms: latency,
ttft_ms: info.ttft_ms,
});
} else {
const r = await api.test(body);
if (r.ok) {
mount(out, r.answer || h("span", { class: "muted" }, "—"));
chatLog.push({ role: "assistant", content: r.answer || "—" });
renderChat();
renderDecision(r);
} else {
mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
chatLog.pop(); // the user turn failed — let them retry it
renderChat();
promptIn.value = prompt;
mount(out, out.firstChild || "", h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
}
}
} catch (e) {
mount(out, h("div", { class: "callout" }, String(e.message || e)));
chatLog.pop();
renderChat();
promptIn.value = prompt;
toast(String(e.message || e), "err");
}
sendBtn.disabled = false;
sendBtn.textContent = t("pg.send");
});
// Cmd/Ctrl+Enter sends
promptIn.addEventListener("keydown", (e) => {
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
e.preventDefault();
sendBtn.click();
}
});
function kv(k, v) {
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
}
function renderDecision(r) {
const c = r.cortiq || {};
const score = c.complexity?.score ?? 0;
const u = r.usage || {};
const haveUsage = u.completion_tokens != null;
// generation speed over the produce phase (after first token when known)
let tokS = null;
if (haveUsage && r.latency_ms) {
const genMs = r.ttft_ms != null ? Math.max(1, r.latency_ms - r.ttft_ms) : r.latency_ms;
tokS = (u.completion_tokens / (genMs / 1000)).toFixed(1);
}
mount(
decision,
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
kv(t("pg.task"), c.task_label || "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, trTier(c.complexity.tier)) : "—"),
h(
"div",
{ class: "kv", style: "display:block" },
@@ -127,17 +215,278 @@ export async function renderPlayground() {
),
kv(t("pg.selected"), h("span", { class: "mono" }, c.selected_model || "—")),
kv(t("pg.source"), c.route_source || "—"),
kv(t("pg.tokens"), `${r.usage?.prompt_tokens ?? 0}/${r.usage?.completion_tokens ?? 0}`),
kv(t("pg.cost"), money(c.cost_usd)),
kv(t("pg.tokens"), haveUsage ? `${r.usage_estimated ? "≈ " : ""}${u.prompt_tokens ?? 0} / ${u.completion_tokens ?? 0}` : "—"),
kv(t("pg.tokSpeed"), tokS != null ? tokS + " tok/s" : "—"),
kv(t("pg.ttft"), r.ttft_ms != null ? ms(r.ttft_ms) : "—"),
kv(t("pg.cost"), moneyEl(c.cost_usd)),
kv(t("pg.latency"), ms(r.latency_ms)),
kv(t("pg.failover"), c.failover ? "yes" : "no")
);
}
mount(
root,
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
const chatPanel = h(
"div",
{ class: "pg" },
h(
"div",
{},
h(
"div",
{ class: "card" },
field(t("pg.system"), sysIn),
field(t("pg.prompt"), promptIn),
field(t("pg.mode"), modeSeg),
modeExtra,
h("div", { class: "row" }, field(t("pg.temp"), tempIn), field(t("pg.maxTokens"), maxTokIn)),
h(
"div",
{ class: "flex", style: "justify-content:space-between; gap:8px" },
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
h("div", { class: "flex", style: "gap:8px" }, clearBtn, sendBtn)
)
),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
),
decision
);
// ═══════════════════════ shared media machinery ═══════════════════════
let mediaPoll = null;
function stopMediaPoll() {
if (mediaPoll) clearInterval(mediaPoll);
mediaPoll = null;
}
// one generation at a time server-side → one active job client-side
function watchJob(id, ui, onDone) {
stopMediaPoll();
const started = performance.now();
mediaPoll = setInterval(async () => {
if (!root.isConnected) return stopMediaPoll();
let j;
try {
j = await api.mediaJob(id);
} catch {
return;
}
const elapsed = (performance.now() - started) / 1000;
const pct = j.progress != null ? Math.round(j.progress * 100) : null;
// honest ETA once there is enough signal: total ≈ elapsed / progress
let eta = "";
if (j.state === "running" && j.progress > 0.03) {
const left = elapsed / j.progress - elapsed;
if (left > 1) eta = ` · ${t("pg.eta")} ~${left > 90 ? Math.round(left / 60) + " " + t("pg.min") : Math.round(left) + " s"}`;
}
let statusLine;
if (j.state === "queued") {
const pos = (j.queue_position ?? 0) + 1;
statusLine = t("pg.queued", { n: pos });
} else {
statusLine =
(j.phase || j.state) +
(pct != null && j.state === "running" ? ` · ${pct}%` : "") +
` · ${elapsed.toFixed(0)}s` + eta;
}
mount(
ui.bar,
h("div", { class: "pbar" },
j.state === "running"
? (pct != null
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
: h("div", { class: "pbar-fill run" }))
: j.state === "queued"
? h("div", { class: "pbar-fill run" })
: h("div", { class: "pbar-fill " + (j.state === "done" ? "ok" : "err"), style: "width:100%" })
),
h("div", { class: "muted small" }, statusLine),
(j.log || []).length && j.state === "running" ? h("pre", { class: "conv-log", style: "max-height:90px" }, j.log.slice(-5).join("\n")) : null
);
if (j.state !== "running" && j.state !== "queued") {
stopMediaPoll();
ui.goBtn.disabled = false;
ui.cancelBtn.style.display = "none";
if (j.state === "done") onDone(j);
else if (j.state === "error") toast(t("pg.genFailed"), "err");
}
}, 1000);
}
function mediaControls(goBtn, onCancelId) {
const cancelBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
cancelBtn.addEventListener("click", async () => {
try {
await api.mediaCancel(onCancelId.id);
} catch (e) {
toast(e.message, "err");
}
});
return cancelBtn;
}
function noModelsHint(kindLabel) {
return h(
"div",
{ class: "card" },
h("div", { class: "callout" },
t("pg.noMediaModels", { kind: kindLabel }),
" ",
h("a", { href: "#/import" }, t("settings.cmf.importLink") + " ↗"))
);
}
function dice(seedIn) {
const b = h("button", { class: "btn ghost sm", title: t("pg.randomSeed") }, "🎲");
b.addEventListener("click", () => (seedIn.value = Math.floor(Math.random() * 1e9)));
return b;
}
// ── gallery: past generations of one kind, loaded lazily ──
function buildGallery(kind, onOpen) {
const host = h("div", {});
async function refresh() {
let jobs = [];
try {
jobs = ((await api.mediaJobs()).jobs || []).filter((j) => j.kind === kind && j.state === "done");
} catch {
return;
}
if (!jobs.length) {
mount(host);
return;
}
const grid = h("div", { class: "gal-grid" });
for (const j of jobs.slice(0, 12)) {
const del = h("button", { class: "gal-del", title: t("import.delete") }, "×");
del.addEventListener("click", async (e) => {
e.stopPropagation();
if (!window.confirm(t("pg.galDeleteConfirm"))) return;
try {
await api.mediaDelete(j.id);
refresh();
} catch (err) {
toast(err.message, "err");
}
});
const cap = h("div", { class: "gal-cap" }, `${j.width}×${j.height} · seed ${j.seed}`);
const item = h("div", { class: "gal-item", title: j.prompt }, del, cap);
if (kind === "image") {
// lazy thumbnail: decode the PPM only when the item scrolls into view
const thumb = h("canvas", { class: "gal-thumb" });
item.prepend(thumb);
const io = new IntersectionObserver(async (entries) => {
if (!entries.some((x) => x.isIntersecting)) return;
io.disconnect();
try {
const buf = await mediaFileBytes(j.id, "image");
const full = ppmToCanvas(buf);
thumb.width = 128;
thumb.height = Math.round((128 * full.height) / full.width) || 128;
thumb.getContext("2d").drawImage(full, 0, 0, thumb.width, thumb.height);
} catch {
/* file gone */
}
});
io.observe(item);
} else {
item.prepend(h("div", { class: "gal-thumb gal-video" }, "🎬"));
}
item.addEventListener("click", () => onOpen(j));
grid.appendChild(item);
}
mount(host, h("div", { class: "card-title", style: "margin-top:14px" }, t("pg.gallery")), grid);
}
refresh();
return { host, refresh };
}
// ════════════════════════════ image panel ════════════════════════════
let imagePanel;
if (!imageModels.length) {
imagePanel = noModelsHint(t("pg.tab.image"));
} else {
const imModel = h("select", {}, ...imageModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
const imPrompt = h("textarea", { placeholder: t("pg.imagePromptPlaceholder"), rows: 3 });
const imW = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
const imH = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
const imSteps = h("input", { type: "number", min: 1, max: 100, value: 30 });
const imCfg = h("input", { type: "number", step: "0.5", value: 4 });
const imSeed = h("input", { type: "number", value: 42 });
const imBar = h("div", {});
const imOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.imageEmpty")));
const imMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
const imGo = h("button", { class: "btn primary" }, "🖼 " + t("pg.generate"));
const imCancelRef = { id: null };
const imCancel = mediaControls(imGo, imCancelRef);
imGo.addEventListener("click", async () => {
const prompt = imPrompt.value.trim();
if (!prompt) return;
imGo.disabled = true;
imCancel.style.display = "";
mount(imOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
const params = {
kind: "image",
model: imModel.value,
prompt,
width: +imW.value,
height: +imH.value,
steps: +imSteps.value || 30,
cfg: parseFloat(imCfg.value) || 4,
seed: +imSeed.value || 42,
};
try {
const r = await api.mediaGenerate(params);
imCancelRef.id = r.job;
const t0 = performance.now();
watchJob(r.job, { bar: imBar, goBtn: imGo, cancelBtn: imCancel }, async (j) => {
mount(imBar);
const secs = (performance.now() - t0) / 1000;
await showImageResult(j, secs);
imGallery.refresh();
});
} catch (e) {
imGo.disabled = false;
imCancel.style.display = "none";
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
}
});
async function showImageResult(j, secs) {
try {
const buf = await mediaFileBytes(j.id, "image");
const canvas = ppmToCanvas(buf);
canvas.className = "media-canvas";
const dl = h("button", { class: "btn sm" }, "⬇ PNG");
dl.addEventListener("click", () =>
canvas.toBlob((b) => {
const a = h("a", { href: URL.createObjectURL(b), download: `cortiq-${j.id}.png` });
a.click();
}, "image/png")
);
mount(imOut, canvas, h("div", { class: "actions" }, dl));
const rows = [
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
kv(t("pg.resolution"), `${j.width}×${j.height}`),
kv(t("pg.steps"), String(j.steps)),
kv(t("pg.seed"), String(j.seed)),
kv(t("pg.fileSize"), fmtBytes(j.out_bytes)),
];
if (secs != null) {
rows.push(kv(t("pg.genTime"), secs.toFixed(1) + " s"));
rows.push(kv(t("pg.timePerStep"), (secs / j.steps).toFixed(2) + " s"));
} else if (j.finished && j.started) {
const dur = j.finished - j.started;
rows.push(kv(t("pg.genTime"), dur + " s"));
rows.push(kv(t("pg.timePerStep"), (dur / j.steps).toFixed(2) + " s"));
}
mount(imMetrics, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), ...rows);
} catch (e) {
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
}
}
const imGallery = buildGallery("image", (j) => {
imSeed.value = j.seed;
imSteps.value = j.steps;
showImageResult(j, null);
});
imagePanel = h(
"div",
{ class: "pg" },
h(
@@ -146,22 +495,178 @@ export async function renderPlayground() {
h(
"div",
{ class: "card" },
h("label", { class: "field" }, h("span", {}, t("pg.system")), sysIn),
h("label", { class: "field" }, h("span", {}, t("pg.prompt")), promptIn),
h("label", { class: "field" }, h("span", {}, t("pg.mode")), modeSeg),
modeExtra,
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("pg.temp")), tempIn), h("label", { class: "field" }, h("span", {}, t("pg.maxTokens")), maxTokIn)),
h(
"div",
{ class: "flex", style: "justify-content:space-between" },
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
sendBtn
)
field(t("pg.model"), imModel),
field(t("pg.prompt"), imPrompt),
h("div", { class: "row" }, field(t("pg.width"), imW), field(t("pg.height"), imH)),
h("div", { class: "row" },
field(t("pg.steps"), imSteps),
field("CFG", imCfg, t("pg.cfgHint"))),
h("div", { class: "row" },
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, imSeed, dice(imSeed)))),
h("div", { class: "actions" }, imCancel, imGo),
imBar
),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), imOut, imGallery.host)
),
decision
)
imMetrics
);
}
// ════════════════════════════ video panel ════════════════════════════
let videoPanel;
if (!videoModels.length) {
videoPanel = noModelsHint(t("pg.tab.video"));
} else {
const vModel = h("select", {}, ...videoModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
const vPrompt = h("textarea", { placeholder: t("pg.videoPromptPlaceholder"), rows: 3 });
const vW = h("input", { type: "number", step: 32, value: 512 });
const vH = h("input", { type: "number", step: 32, value: 288 });
const vFrames = h("input", { type: "number", value: 39 });
const vSteps = h("input", { type: "number", value: 4 });
const vSeed = h("input", { type: "number", value: 42 });
const vQuality = h("input", { type: "number", min: 10, max: 100, value: 92 });
const vFirst = h("input", { type: "file", accept: "image/*" });
const vLast = h("input", { type: "file", accept: "image/*" });
const vBar = h("div", {});
const vOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.videoEmpty")));
const vMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
const vGo = h("button", { class: "btn primary" }, "🎬 " + t("pg.generate"));
const vCancelRef = { id: null };
const vCancel = mediaControls(vGo, vCancelRef);
vGo.addEventListener("click", async () => {
const prompt = vPrompt.value.trim();
if (!prompt) return;
vGo.disabled = true;
vCancel.style.display = "";
mount(vOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
const params = {
kind: "video",
model: vModel.value,
prompt,
width: +vW.value || 512,
height: +vH.value || 288,
frames: +vFrames.value || 39,
steps: +vSteps.value || 4,
seed: +vSeed.value || 42,
quality: +vQuality.value || 92,
};
try {
if (vFirst.files[0]) params.first_frame_b64 = await fileToPpmB64(vFirst.files[0], params.width, params.height);
if (vLast.files[0]) params.last_frame_b64 = await fileToPpmB64(vLast.files[0], params.width, params.height);
const r = await api.mediaGenerate(params);
vCancelRef.id = r.job;
const t0 = performance.now();
watchJob(r.job, { bar: vBar, goBtn: vGo, cancelBtn: vCancel }, async (j) => {
mount(vBar);
vGallery.refresh();
const secs = (performance.now() - t0) / 1000;
try {
await showVideoResult(j, vOut);
const durS = (j.frames || 0) / 24;
mount(
vMetrics,
h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))),
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
kv(t("pg.resolution"), `${j.width}×${j.height}`),
kv(t("pg.frames"), `${j.frames} @ 24 fps`),
kv(t("pg.duration"), durS.toFixed(1) + " s"),
kv(t("pg.steps"), String(j.steps)),
kv(t("pg.seed"), String(j.seed)),
kv(t("pg.genTime"), secs.toFixed(1) + " s"),
kv(t("pg.timePerFrame"), j.frames ? (secs / j.frames).toFixed(2) + " s" : "—"),
kv(t("pg.fileSize"), fmtBytes(j.out_bytes))
);
} catch (e) {
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
}
});
} catch (e) {
vGo.disabled = false;
vCancel.style.display = "none";
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
}
});
async function showVideoResult(j, host) {
const downloads = h("div", { class: "actions" });
for (const kind of j.outputs || []) {
const label = { video: "⬇ AVI", audio: "⬇ WAV", mp4: "⬇ MP4" }[kind];
if (!label) continue;
const b = h("button", { class: "btn sm" }, label);
b.addEventListener("click", async () => {
const { url } = await mediaFileUrl(j.id, kind);
const ext = kind === "video" ? "avi" : kind === "audio" ? "wav" : "mp4";
h("a", { href: url, download: `cortiq-${j.id}.${ext}` }).click();
});
downloads.appendChild(b);
}
if ((j.outputs || []).includes("mp4")) {
// ffmpeg was present server-side → native playback
const { url } = await mediaFileUrl(j.id, "mp4");
mount(host, h("video", { src: url, controls: true, class: "media-canvas" }), downloads);
return;
}
// no ffmpeg anywhere: decode the MJPEG AVI ourselves, sync to the wav
const buf = await mediaFileBytes(j.id, "video");
const frames = aviJpegFrames(buf);
let audio = null;
if ((j.outputs || []).includes("audio")) {
const { url } = await mediaFileUrl(j.id, "audio");
audio = h("audio", { src: url });
}
const player = mjpegPlayer(frames, 24, audio);
player.canvas.className = "media-canvas";
await player.seek0();
const playBtn = h("button", { class: "btn primary sm" }, "▶ " + t("pg.play"));
playBtn.addEventListener("click", async () => {
if (player.state.playing) {
player.pause();
playBtn.textContent = "▶ " + t("pg.play");
} else {
playBtn.textContent = "⏸";
await player.play();
}
});
mount(host, player.canvas, h("div", { class: "actions" }, playBtn, ...downloads.childNodes ? [downloads] : []));
}
const vGallery = buildGallery("video", (j) => showVideoResult(j, vOut));
videoPanel = h(
"div",
{ class: "pg" },
h(
"div",
{},
h(
"div",
{ class: "card" },
field(t("pg.model"), vModel),
field(t("pg.prompt"), vPrompt),
h("div", { class: "row" }, field(t("pg.width"), vW, t("pg.videoSizeHint")), field(t("pg.height"), vH)),
h("div", { class: "row" }, field(t("pg.frames"), vFrames, t("pg.framesHint")), field(t("pg.steps"), vSteps, t("pg.videoStepsHint"))),
h("div", { class: "row" },
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, vSeed, dice(vSeed))),
field(t("pg.quality"), vQuality)),
h("div", { class: "row" }, field(t("pg.firstFrame"), vFirst, t("pg.keyframeHint")), field(t("pg.lastFrame"), vLast)),
h("div", { class: "actions" }, vCancel, vGo),
vBar
),
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), vOut, vGallery.host)
),
vMetrics
);
}
const panels = { chat: chatPanel, image: imagePanel, video: videoPanel };
mount(
root,
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
tabsSeg,
panelHost
);
mount(panelHost, panels.chat);
return root;
}
+262 -32
View File
@@ -1,74 +1,293 @@
// Routing — tier editor (ordered model chips), default model, and policy.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
// Known labels of the hosted taxonomy — used for localized names/descriptions
// and as the offline fallback; the live list comes from /routing/labels.
const TASK_META = {
chitchat: { icon: "💬" },
code: { icon: "⌨" },
"creative-writing": { icon: "✍" },
extraction: { icon: "📑" },
math: { icon: "∑" },
qa: { icon: "❓" },
summarization: { icon: "📋" },
translation: { icon: "🌍" },
};
export async function renderRouting() {
const meta = appState.meta || (await api.meta());
const [routing, modelsData] = await Promise.all([api.getRouting(), api.listModels()]);
const ids = (modelsData.models || []).map((m) => m.id);
// Task labels: live from the router when reachable, built-in set otherwise;
// labels that already have a rule are always shown (custom taxonomies).
let taskLabels = Object.keys(TASK_META);
let labelsSource = "builtin";
try {
const r = await api.routingLabels();
if (r.labels && r.labels.length) {
taskLabels = r.labels;
labelsSource = "router";
}
} catch {
/* offline — builtin list */
}
for (const k of Object.keys(routing.tasks || {})) {
if (!taskLabels.includes(k)) taskLabels.push(k);
}
// Share of recent traffic per task — shows which rules actually matter.
let taskShare = {};
try {
const stat = await api.stats("7d", "label");
const total = (stat.breakdown || []).reduce((a, b) => a + (b.requests || 0), 0);
if (total > 0) {
for (const b of stat.breakdown || []) taskShare[b.key] = (b.requests || 0) / total;
}
} catch {
/* no stats yet */
}
// local editable state — ensure standard tiers exist for a friendly editor
const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {});
const tasks = {};
// A task rule is edited as ordered PAIRS (model, tier): "this model serves
// this task at this complexity" (tier "any" = at every complexity).
// GET returns {any:[], low:[], medium:[], high:[]}; legacy arrays = any.
const taskPairs = {};
for (const l of taskLabels) {
const v = (routing.tasks || {})[l];
const pairs = [];
if (Array.isArray(v)) for (const m of v) pairs.push({ model: m, tier: "any" });
else if (v && typeof v === "object") {
for (const tier of ["low", "medium", "high", "any"])
for (const m of v[tier] || []) pairs.push({ model: m, tier });
}
taskPairs[l] = pairs;
}
const st = {
tiers,
taskPairs,
bands: routing.bands || null, // null = пороги роутера как есть
default: routing.default || ids[0] || "",
policy: routing.policy || { mode: "fixed_table", min_class: {} },
};
const root = h("div");
function moveChip(tier, i, dir) {
const arr = st.tiers[tier];
function moveChip(map, key, i, dir) {
const arr = map[key];
const ni = i + dir;
if (ni < 0 || ni >= arr.length) return;
[arr[i], arr[ni]] = [arr[ni], arr[i]];
render();
}
function removeChip(tier, i) {
st.tiers[tier].splice(i, 1);
function removeChip(map, key, i) {
map[key].splice(i, 1);
render();
}
function addChip(tier, id) {
if (id && !st.tiers[tier].includes(id)) st.tiers[tier].push(id);
function addChip(map, key, id) {
if (id && !map[key].includes(id)) map[key].push(id);
render();
}
function tierRow(tier) {
const arr = st.tiers[tier];
function chipEditor(map, key, emptyLabel) {
const arr = map[key];
const avail = ids.filter((id) => !arr.includes(id));
const addSel = h(
"select",
{
style: "width:auto",
onChange: (e) => {
addChip(tier, e.target.value);
},
},
{ style: "width:auto", onChange: (e) => addChip(map, key, e.target.value) },
h("option", { value: "" }, t("routing.addModel")),
...avail.map((id) => h("option", { value: id }, id))
);
return h(
"div",
{ class: "chips" },
arr.length === 0 && emptyLabel ? h("span", { class: "muted small chips-empty" }, emptyLabel) : null,
...arr.map((id, i) =>
h(
"div",
{ class: "chip" },
h("span", { class: "ord" }, i + 1),
id,
h("span", { class: "arrow", title: "←", onClick: () => moveChip(map, key, i, -1) }, "◄"),
h("span", { class: "arrow", title: "→", onClick: () => moveChip(map, key, i, 1) }, "►"),
h("span", { class: "x", onClick: () => removeChip(map, key, i) }, "×")
)
),
avail.length ? addSel : null
);
}
// Linked boundary sliders: one track, two handles (low|medium boundaries),
// three colored zones. Handles push each other, numbers update live —
// nobody types score ranges by hand.
function bandsCard() {
const DEF = { low: 0.33, medium: 0.48 };
const enabled = !!st.bands;
const b = st.bands || { ...DEF };
const MIN_GAP = 0.02;
const track = h("div", { class: "bands-track" });
const zoneLow = h("div", { class: "bands-zone z-low" });
const zoneMed = h("div", { class: "bands-zone z-med" });
const zoneHigh = h("div", { class: "bands-zone z-high" });
const hLow = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
const hMed = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
const legend = h("div", { class: "bands-legend" });
track.append(zoneLow, zoneMed, zoneHigh, hLow, hMed);
function paint() {
const lo = b.low, me = b.medium;
zoneLow.style.width = lo * 100 + "%";
zoneMed.style.left = lo * 100 + "%";
zoneMed.style.width = (me - lo) * 100 + "%";
zoneHigh.style.left = me * 100 + "%";
zoneHigh.style.width = (1 - me) * 100 + "%";
hLow.style.left = lo * 100 + "%";
hMed.style.left = me * 100 + "%";
hLow.querySelector(".bands-val").textContent = lo.toFixed(2);
hMed.querySelector(".bands-val").textContent = me.toFixed(2);
// подписи не наезжают, когда ручки рядом
hMed.querySelector(".bands-val").classList.toggle("raised", me - lo < 0.09);
mount(legend,
h("span", { class: "badge tier-low" }, `${trTier("low")} ≤ ${lo.toFixed(2)}`),
h("span", { class: "badge tier-medium" }, `${trTier("medium")} ≤ ${me.toFixed(2)}`),
h("span", { class: "badge tier-high" }, `${trTier("high")} > ${me.toFixed(2)}`));
track.classList.toggle("disabled", !st.bands);
}
function drag(handle, which) {
handle.addEventListener("pointerdown", (e) => {
if (!st.bands) return;
handle.setPointerCapture(e.pointerId);
const rect = track.getBoundingClientRect();
const move = (ev) => {
let v = (ev.clientX - rect.left) / rect.width;
v = Math.min(0.98, Math.max(0.02, Math.round(v * 100) / 100));
if (which === "low") {
b.low = v;
// связанность: low толкает medium перед собой
if (b.medium < b.low + MIN_GAP) b.medium = Math.min(0.99, b.low + MIN_GAP);
} else {
b.medium = v;
// и medium толкает low назад
if (b.low > b.medium - MIN_GAP) b.low = Math.max(0.01, b.medium - MIN_GAP);
}
st.bands = { low: b.low, medium: b.medium };
paint();
};
const up = () => {
handle.removeEventListener("pointermove", move);
handle.removeEventListener("pointerup", up);
};
handle.addEventListener("pointermove", move);
handle.addEventListener("pointerup", up);
});
}
drag(hLow, "low");
drag(hMed, "medium");
const toggle = h("input", { type: "checkbox", checked: enabled ? "" : null });
toggle.checked = enabled;
toggle.addEventListener("change", () => {
st.bands = toggle.checked ? { low: b.low, medium: b.medium } : null;
paint();
});
paint();
return h(
"div",
{ class: "bands-wrap" },
h("label", { class: "check", style: "margin-bottom:8px" }, toggle, t("routing.bands.enable")),
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.bands.hint")),
track,
legend
);
}
function tierRow(tier) {
return h(
"div",
{ class: "tierrow" },
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, trTier(tier))),
chipEditor(st.tiers, tier)
);
}
function taskRow(label) {
const known = TASK_META[label];
const name = known ? t("task." + label) : label;
const share = taskShare[label];
return h(
"div",
{ class: "tierrow taskrow" },
h(
"div",
{ class: "chips" },
...arr.map((id, i) =>
h(
"div",
{ class: "chip" },
h("span", { class: "ord" }, i + 1),
id,
h("span", { class: "arrow", title: "←", onClick: () => moveChip(tier, i, -1) }, "◄"),
h("span", { class: "arrow", title: "→", onClick: () => moveChip(tier, i, 1) }, "►"),
h("span", { class: "x", onClick: () => removeChip(tier, i) }, "×")
)
),
avail.length ? addSel : null
)
{ class: "tname taskname" },
h("div", {}, (known ? known.icon + " " : "") + name),
h("div", { class: "muted small mono" }, label),
share != null ? h("div", { class: "muted small" }, t("routing.tasks.share", { p: Math.round(share * 100) })) : null
),
pairsEditor(label)
);
}
// Pairs editor: each row is [model ▾][complexity ▾][×]; "any" serves every
// tier. Order matters — it is the fallback order inside the tier.
function pairsEditor(label) {
const pairs = st.taskPairs[label];
const wrap = h("div", { class: "pairs grow" });
function paint() {
mount(
wrap,
...pairs.map((p, i) => {
const modelSel = h(
"select",
{ class: "pair-model" },
...ids.map((id) => h("option", { value: id }, id))
);
modelSel.value = p.model;
modelSel.addEventListener("change", () => { p.model = modelSel.value; });
const tierSel = h(
"select",
{ class: "pair-tier", title: t("routing.tasks.tierHint") },
h("option", { value: "any" }, t("routing.tasks.tierAny")),
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, trTier(tr)))
);
tierSel.value = p.tier;
tierSel.addEventListener("change", () => { p.tier = tierSel.value; });
const del = h("button", { class: "btn sm ghost danger", onClick: () => { pairs.splice(i, 1); paint(); } }, "×");
return h("div", { class: "pair-row" }, modelSel, tierSel, del);
}),
pairs.length === 0 ? h("div", { class: "muted small", style: "align-self:center" }, t("routing.tasks.inherit")) : null,
h("button", { class: "btn ghost sm", onClick: () => { pairs.push({ model: ids[0] || "", tier: "any" }); paint(); } }, "+ " + t("routing.tasks.addModel"))
);
}
paint();
return wrap;
}
function tasksCard() {
const addIn = h("input", { placeholder: t("routing.tasks.addPlaceholder"), style: "max-width:220px" });
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("routing.tasks.addCustom"));
addBtn.addEventListener("click", () => {
const v = addIn.value.trim().toLowerCase();
if (!v || st.taskPairs[v]) return;
st.taskPairs[v] = [];
render();
});
return h(
"div",
{ class: "card" },
h("div", { class: "card-head" },
h("h3", {}, "🎯 " + t("routing.tasks.title")),
h("span", { class: "card-head sub" },
labelsSource === "router" ? t("routing.tasks.fromRouter") : t("routing.tasks.builtin"))),
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.tasks.hint")),
...Object.keys(st.taskPairs).map(taskRow),
h("div", { class: "flex", style: "gap:8px; margin-top:10px" }, addIn, addBtn)
);
}
@@ -116,7 +335,16 @@ export async function renderRouting() {
saveBtn.addEventListener("click", async () => {
saveBtn.disabled = true;
try {
await api.putRouting({ tiers: st.tiers, default: st.default, policy: st.policy });
const tasks = {};
for (const [k, pairs] of Object.entries(st.taskPairs)) {
const rule = {};
for (const p of pairs) {
if (!p.model) continue;
(rule[p.tier] = rule[p.tier] || []).push(p.model);
}
if (Object.keys(rule).length) tasks[k] = rule;
}
await api.putRouting({ tiers: st.tiers, tasks, bands: st.bands, default: st.default, policy: st.policy });
toast(t("toast.saved"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
@@ -141,8 +369,10 @@ export async function renderRouting() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
bandsCard(),
...Object.keys(st.tiers).map(tierRow)
),
tasksCard(),
h(
"div",
{ class: "card" },
+96 -11
View File
@@ -1,7 +1,7 @@
// Settings — router/route/breaker/log/telemetry/stats + export/import config.
import { h, mount, toast } from "../ui.js";
import { h, mount, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { api, getToken, postRaw } from "../api.js";
import { appState, SITE_URL } from "../app.js";
// A small "?" badge with a hover/focus tooltip explaining a setting.
@@ -194,8 +194,30 @@ export async function renderSettings() {
const modelsData = await api.listModels();
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
// Live proof-of-life for the cache card: entries/hits/saved — and an
// honest warning when the cache is on but has no embedding model to work
// with (it silently does nothing in that state).
let cacheStats = null;
try {
cacheStats = (await api.stats("24h", "model")).cache || null;
} catch { /* stats off */ }
function cacheStatus() {
const wrap = h("div", { style: "margin-bottom:10px" });
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
return wrap;
}
if (cacheStats && s.cache?.enabled) {
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
}
return wrap;
}
const cacheEmbedSel = h(
"select",
{},
@@ -480,22 +502,33 @@ export async function renderSettings() {
});
async function exportConfig() {
const cfg = await api.getConfig();
const blob = new Blob([JSON.stringify(cfg, null, 2)], { type: "application/json" });
const url = URL.createObjectURL(blob);
const a = h("a", { href: url, download: "gateway-config.json" });
// the real gateway.toml from disk — comments and all
const tok = getToken();
const headers = {};
if (tok) headers["Authorization"] = "Bearer " + tok;
const r = await fetch("/admin/api/config/export", { headers });
if (!r.ok) {
toast(await r.text(), "bad");
return;
}
const url = URL.createObjectURL(await r.blob());
const a = h("a", { href: url, download: "gateway.toml" });
document.body.appendChild(a);
a.click();
a.remove();
URL.revokeObjectURL(url);
}
const fileIn = h("input", { type: "file", accept: ".json,application/json", style: "display:none" });
const fileIn = h("input", { type: "file", accept: ".toml,.json,application/json", style: "display:none" });
fileIn.addEventListener("change", async () => {
const f = fileIn.files[0];
if (!f) return;
try {
const cfg = JSON.parse(await f.text());
await api.putConfig(cfg);
const text = await f.text();
if (f.name.endsWith(".json")) {
await api.putConfig(JSON.parse(text)); // legacy JSON export
} else {
await postRaw("/config/import", text); // gateway.toml — validated + hot-reloaded
}
toast(t("toast.imported"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
@@ -503,6 +536,54 @@ export async function renderSettings() {
fileIn.value = "";
});
const restartBtn = h("button", { class: "btn danger" }, "⟳ " + t("settings.restart"));
restartBtn.addEventListener("click", async () => {
if (!(await confirmDialog(t("settings.restartConfirm")))) return;
restartBtn.disabled = true;
try {
await api.restart();
toast(t("settings.restarting"), "good");
// the gateway comes back on the same port — reload once it answers
const poll = setInterval(async () => {
try {
await api.health();
clearInterval(poll);
location.reload();
} catch {
/* still restarting */
}
}, 1000);
} catch (e) {
toast(String(e.message || e), "bad");
restartBtn.disabled = false;
}
});
const stopBtn = h("button", { class: "btn danger" }, "\u23fb " + t("settings.shutdown"));
stopBtn.addEventListener("click", async () => {
if (!(await confirmDialog(t("settings.shutdownConfirm")))) return;
stopBtn.disabled = true;
restartBtn.disabled = true;
try {
await api.shutdown();
// the server is gone in ~300ms — replace the app with a static farewell
document.body.innerHTML = "";
document.body.appendChild(
h(
"div",
{ class: "shutdown-screen" },
h("div", { class: "shutdown-icon" }, "\u23fb"),
h("h2", {}, t("settings.shutdownDone")),
h("p", { class: "hint" }, t("settings.shutdownHint"))
)
);
} catch (e) {
toast(String(e.message || e), "bad");
stopBtn.disabled = false;
restartBtn.disabled = false;
}
});
mount(
root,
h(
@@ -576,6 +657,7 @@ export async function renderSettings() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
cacheStatus(),
cacheEnabled.node,
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
field(t("settings.cache.embed"), cacheEmbedSel),
@@ -623,7 +705,10 @@ export async function renderSettings() {
{ class: "flex wrap" },
h("button", { class: "btn", onClick: exportConfig }, t("settings.export")),
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
fileIn
fileIn,
h("div", { class: "grow" }),
restartBtn,
stopBtn
)
)
);