Compare commits

..
21 Commits
Author SHA1 Message Date
infosave2007 6f4d83ce13 chore: release 0.2.41
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 09:55:20 +03:00
infosave2007 0040bfd7e9 fix(routing): the high zone is unmistakably red
At 35% opacity over the dark background the amber medium zone and the
light-red high zone read as the same color right of the medium handle.
Saturation stepped up per zone (45/55/65%) and high uses a deeper red —
green / amber / red now read at a glance.
2026-08-08 09:53:25 +03:00
infosave2007 10b5101757 fix(i18n): complexity tier names are translated everywhere they're shown
low/medium/high leaked as raw English into the bands legend, the tier badges
on the Routing page, the pair complexity selects, the benchmark's tier column
and the Playground decision panel. New trTier() helper renders the human name
(ru: лёгкий/средний/сложный, de: leicht/mittel/schwer, fr/es/zh/tr likewise);
custom tier names pass through untouched, badge colors stay keyed by the raw
tier. Verified in the browser in Russian and German.
2026-08-08 09:51:38 +03:00
infosave2007 788c87c565 feat(routing): linked complexity-boundary sliders — your own low/medium/high
The tier boundaries lived on the router and were global for every customer
of a hosted router; per-gateway tuning meant hand-editing score ranges. The
Routing page now has one track with two draggable handles (the low|medium
and medium|high boundaries) over three colored zones: handles push each
other (low can't cross medium — it shoves it along, and vice versa), the
numbers ride on the handles and the legend updates live, so nobody types
ranges by hand. Unchecked = the router's own bands, untouched behavior.

Under the hood the gateway re-maps the router's CONTINUOUS complexity score
with the user's boundaries ([routing.bands] low/medium in gateway.toml,
validated 0 < low < medium < 1) before tier chains, escalation and stats —
verified live: with bands at 0.70/0.72 a score-0.27 request that used to be
medium routed low, and back after restoring the defaults.

2 new strings in all 7 languages.
2026-08-08 09:48:36 +03:00
infosave2007 179c414a0e fix(benchmark): retry the SAME hard prompt after a pause before reaching for a fresh one
Difficulty is judged in the background: the first-ever occurrence of a hard
prompt routes by prior (often low/medium), the oracle's judgement lands in
the cache a couple of seconds later. The route-judged retry fired instantly
with a FRESH template — which has the same first-encounter fate — so the
test could go red exactly when the system was working as designed. The retry
now re-asks the same prompt after 4s (steady-state answer from the cache),
and only then tries a fresh template.
2026-08-08 09:23:53 +03:00
infosave2007 115b04f20e chore: release 0.2.40
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-08 09:13:12 +03:00
infosave2007 b24d4cdbab fix(routing): the task-rule badge only when the rule actually chose the head
route_source reported "+task-rule" whenever a rule existed for the task —
so a math prompt routed low by the ordinary tier chain still wore the badge
next to the local model, reading as "the rule picked this" when it hadn't
(the rule only defines math@high). matched now means the head of the chain
came from the rule's tier-specific or any-tier list; verified live: the same
low math request now reports route_source "router".
2026-08-08 09:02:25 +03:00
infosave2007 06813b3e97 feat(routing)!: task rules become per-complexity model maps
The single per-task «Complexity» pin was the wrong shape — what you actually
decide is WHICH model serves a task at WHICH complexity. A task rule is now a
per-tier map, edited in the console as rows of “model ▾ + complexity ▾” pairs
(plus ×/add) inside each task:

  [routing.tasks.code]
  low  = ["cheap-coder"]
  high = ["best-coder"]
  any  = ["generic-coder"]   # leads at every tier after the tier pick

Chain order: the task's tier-specific list → (at high: the global high chain,
so escalation still beats any-tier pins) → the task's any-tier list → the
global tier chain → default as the last resort; deduplicated. Plain-array
rules keep meaning "any tier" unchanged, and the short-lived {models, tier}
form from the previous commit is normalized into the map on load.

Verified live: legacy config migrated visibly in the editor, pairs survive
save+reload, a math prompt at low ignored the math@high pair, per-tier leads
confirmed by unit tests (19 passing).
2026-08-08 08:54:27 +03:00
infosave2007 95e560dc72 feat: per-key spend accounting and per-task complexity pinning
API keys page: the table now answers "which department spends what" — per-key
requests, tokens in/out and cost for a selectable window (24h/7d/30d) plus a
cumulative all-time cost column, joined from stats by the key's account.

The stats log stays bounded while the accounting survives: once stats.jsonl
exceeds max_file_mb (default 16, 0 disables), records older than the
retention window fold into cumulative per-account totals in a small sidecar
file (tmp+rename on both) and leave the log. alltime = folded + in-file, a
record is never counted twice, verified end-to-end on synthetic data
(1.2 MB → 301 KB, totals identical before/after compaction and restarts).

Routing: a task rule can now PIN the complexity tier —
`[routing.tasks.legal] models=[...] tier="high"` (old plain-array rules parse
unchanged). The pinned tier replaces the router's verdict before the
escalation check and reports itself as route_source "router+task-rule@high".
The Routing page grows a per-task «Complexity» selector (Auto/low/medium/
high); verified live: a code prompt with a pinned high went
router+task-rule@high → the expensive model.

13 new strings in all 7 languages; docs/ROUTING(.ru) updated.
2026-08-08 08:42:11 +03:00
infosave2007 ef28a6633d fix(benchmark): hard probes survive their own popularity
The router deliberately scores familiar text as less novel, so every
benchmark run slightly burns the hard-prompt pool — with 3 templates per
test a variant would drift below the high band within a handful of runs
and flip the test red for no real reason.

Two-part fix: pools doubled to 6 interdisciplinary families each (science:
+ quantum error correction, soil-carbon MRV, cryo-EM; ops: + ED triage +
staffing, port berth/tides/rail, agri-drone fleets — all probed high against
the live router), and a route-judged test that lands non-high gets ONE retry
with a freshly generated template before failing. A single burnt wording no
longer fails the assertion; a genuinely unreachable high tier still does.
2026-08-07 22:47:28 +03:00
infosave2007 b4d36334bf chore: release 0.2.39
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-07 22:25:52 +03:00
infosave2007 dd9ccaea94 fix(shell): stale tabs now know the gateway was updated
Recurring confusion all day: the SPA lives in an open tab until reload, so
after a gateway rebuild/update users kept seeing (and benchmarking with!) an
old interface — "where is the feature", "why are the test ids old". The meta
endpoint now carries the process start timestamp; the existing 20s health
poll compares it, and when the gateway has restarted the tab shows a fixed
banner: "The gateway was updated — this page is running an older interface"
with a one-click Reload. 2 strings in all 7 languages.
2026-08-07 22:14:08 +03:00
infosave2007 ff8415d7d4 feat(routing): high tier escalates over task rules; benchmark generates hard prompts that stay hard
Routing semantics fix observed live: a task rule ("code → cheap model") was
pinning even high-complexity prompts to the cheap model — the tier system
never got a say. At tier=high the tier chain now leads and the task rule
demotes to fallback; low/medium keep task-rule-first. Unit-tested both ways.

The benchmark's hard probes are now GENERATED per run from two pools
(science: protein folding / telescope scheduling / materials discovery;
ops: epidemic logistics / grid markets / climate risk) with rotating domains
and numbers. Two findings against the live router drove this: ambiguity
carries the biggest weight in the complexity score, so interdisciplinary
prompts rate high reliably (0.52-0.66) while single-domain "hard" prompts
classify too confidently and sink to medium; and any fixed wording loses
novelty on every repeat. Their labels are null — straddling task labels is
precisely what makes them high, so classification is not judged.

Verified end-to-end: full run 100% answers, 13/13 classification, both hard
probes high → routed to the expensive model (one of them over a code task
rule), at $0.0018 each thanks to the tiny route-judge token budget.
2026-08-07 21:34:24 +03:00
infosave2007 7479cc51c2 feat(benchmark): hard tests that light up the high tier — judged on routing, not essays
Two "hard" probes (full suite only): a compiler-backend design and a security
protocol design, each dense enough for the router's complexity model to rate
them high. They are judged on the ROUTER's decision (judge: "route") — pass
means tier=high was reached and the request went to the big model; the answer
itself is not scored (open-ended essays aren't keyword-judgeable, and
reasoning models happily burn 3000+ thinking tokens on them — measured). With
max_tokens kept tiny the probes cost cents even on the priciest model, while
the cost column tells the whole story: $0.0148 for the high-tier request next
to $0.00005 for the easy ones.

Each run picks a random variant of the prompt: the router deliberately scores
familiar text as less novel, so a fixed wording would decay to medium over
repeated runs. In pinned-models mode route-judged tests are skipped.

Also: creative-writing budget 96→512 (deepseek was spending the whole cap
thinking and returning an empty poem), and when a model does burn its entire
budget inside <think> the table now says so ("budget spent on reasoning")
instead of the misleading "empty answer".

Server-side (cortiq-router prod, both replicas rolling-restarted): the high
band started at 0.66 while the complexity model tops out ~0.65 on the hardest
single prompts — high was unreachable. Band boundary moved to 0.50 based on
measured distribution (easy 0.25-0.36, medium 0.37-0.49, hard 0.50-0.65);
config backed up as config.prod.toml.bak-tiers.
2026-08-07 20:15:46 +03:00
infosave2007 b1e0dc76ec fix(benchmark): fair to reasoning models, fair to router degradation, real sub-cent costs
Three false signals in benchmark results, all seen on a live run:

- Reasoning models were scored ❌ because the per-test max_tokens cap cut
  them mid-<think> before the actual answer. Every run now gets +256 tokens
  of headroom; non-thinking models stop at EOS as before, thinking ones get
  to finish (math-mult on nanbeige: false 0/1 → honest 1/1).
- When the router degraded (timeout → fallback chain) the table showed
  "✗ degraded" and counted it against classification accuracy. There was no
  classification to judge — such rows now show a muted "⚠ router degraded"
  with an explanatory tooltip and are excluded from the accuracy KPI.
- Costs below $0.0001 (cheap flash models) collapsed to "$0.0000". money
  formatting now grows precision until two significant digits survive
  ($0.000035 instead of $0.0000).

2 new strings in all 7 languages.
2026-08-07 19:48:03 +03:00
infosave2007 c3dd251cce fix(keys): drop the allowed-models picker from the new-key dialog
Creating a key now asks only account, rate limit and optional custom value —
allow_models is sent empty (= all models). Per-key model restrictions stay
editable in Settings for whoever actually needs them.
2026-08-07 19:41:56 +03:00
infosave2007 9af51b679e fix(bench): newer cortiq pretty-prints --json — parse it, and report honest tok/s
The one-click model benchmark showed "error" even though cortiq bench ran
fine: the parser expected the JSON result on a single line, while newer
cortiq releases pretty-print it across many. Extraction now finds the last
balanced JSON object anywhere in the mixed CLI output (both formats work).

Also hardened tok/s selection: steady decode speed (decode_tok_s_steady)
is preferred over incl-prefill, and the loose "contains tok+per" fallback —
which would happily report allocs_per_token=769 as tok/s — now only matches
*tok_s* / *tokens_per* fields.
2026-08-07 19:32:47 +03:00
infosave2007 1bba427042 feat(keys): one-paste harness setup — Claude Code, Codex, OpenCode, Continue, Cline
The "connect your app" card in the API-keys view now leads with auto-setup
scripts for popular coding harnesses, each pre-filled with this gateway's URL
and the freshly created key:

  Claude Code — env exports (the gateway speaks Anthropic /v1/messages
      natively; the protocol is now ON by default, and when a config has it
      off the tab shows a warning with a one-click Enable button)
  Codex — appends a [model_providers.cortiq] block to ~/.codex/config.toml
  OpenCode — writes the provider into opencode.json
  Continue — appends the model to ~/.continue/config.yaml
  Cline — the exact values to paste into the extension settings

The gateway also detects which harnesses are already present on its machine
(binaries across PATH + common install dirs, config files under home) via
GET /admin/api/harnesses: installed tabs get a ✓, and the status line under
the snippet says "installed" or "already configured for this gateway" when
the harness config mentions cortiq. Works on Linux/macOS/Windows.

curl / python / javascript SDK snippets stay as before. 16 new strings in
all 7 languages.
2026-08-07 19:26:54 +03:00
infosave2007 54b73f1365 fix: honest costs in streams, oracle-sized timeouts, history & log cleanup
Cost showed $0 for every streamed request: the X-Cortiq-Cost-USD header is
written before generation, when the cost cannot be known. The gateway now
ships the final number in a closing SSE event (`cortiq_cost_usd`, alongside
the token estimate when the provider sent no usage), and the console prefers
it over the header. Money renders with 4 decimals, the last two in a smaller
font — $0.0006 stays readable instead of collapsing to $0.

Timeouts, both ends: the router default was 800 ms, silently degrading every
prompt the hosted router escalates to its oracle (~10 s) — now 15 s. Provider
clients had NO timeout at all (a dead OpenRouter host hung requests forever) —
now a 10 s connect timeout, with no overall cap so long generations and
streams are never cut.

Benchmark history: per-run × and a Clear-all button. Logs view: a clear
button backed by DELETE /admin/api/logs.
2026-08-07 19:15:27 +03:00
infosave2007 ec92bdddb7 feat(benchmark): smart-routing mode is the point — live results, per-model cost
The benchmark now leads with the product's actual feature. Two explicit modes:

  🧭 Smart routing (default) — every probe goes through cortiq-auto; each row
     shows the router's task label judged against the expected one (✓/✗ with
     "expected: …"), the complexity tier, the chosen model with what that
     request cost, and a task-rule badge when a routing rule fired. KPIs:
     answer quality, classification accuracy, TTFT, tok/s and money spent,
     plus a "requests went to" distribution with per-model cost.
  🎯 Individual models — the old head-to-head comparison, now as the clearly
     separate option (model checkboxes appear only here).

The suite covers all 8 taxonomy labels (added summarization, creative-writing
and chitchat probes; expected labels annotated per test). Results are LIVE:
the table grows a row after every test (with a small fade-in), the progress
card shows the prompt being asked, and a 🏁 verdict toast lands at the end
with the score and the total spend.

12 new strings in all 7 languages.
2026-08-07 19:04:41 +03:00
infosave2007 1120d8f11a feat(benchmark): the built-in benchmark, redesigned into the console
The old standalone benchmark.html (Russian-only, Tailwind+Chart.js from CDNs,
never shipped in the crate) becomes a proper sidebar view. Same scoring ideas,
reimplemented: keyword checks with |-synonyms, think-tag stripping, tail-loop
detection, pass/partial/fail — plus what the old page never had: runs go
through the LIVE pipeline (pinned per model, streamed), so every test also
measures tok/s and time-to-first-token with the gateway's real token counts.

A built-in suite of 12 language-neutral probes (math, logic, code, QA,
instruction-following, extraction, translation, JSON formatting); the ⚡ quick
mode runs the 6 short ones — about a minute on a local model. Results: score
bars per model sorted best-first, a per-test grid (click a cell for the full
answer), previous runs and custom tests kept in the browser.

22 new strings in all 7 languages.
2026-08-07 18:56:47 +03:00
33 changed files with 2265 additions and 134 deletions
+84
View File
@@ -3,6 +3,90 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.41] - 2026-08-08
### Added
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
(the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
other, the numbers ride on the handles and the legend updates live — no hand-typed score
ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
(`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
stats; unchecked = the router's own bands, untouched behavior.
### Fixed
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
fresh template — difficulty is judged in the background, so the second occurrence is the
honest steady-state answer.
## [0.2.40] - 2026-08-08
### Added
- **Per-key spend accounting** on the API-keys page: requests, tokens in/out and cost per key
for a selectable window (24h / 7d / 30d) plus a cumulative all-time cost column — "which
department spends what" at a glance.
- **Bounded stats on disk**: once `stats.jsonl` outgrows `stats.max_file_mb` (default 16, 0
disables), records older than the retention window fold into cumulative per-account totals in
a small sidecar file (tmp+rename on both) and leave the log. All-time accounting survives
compaction and restarts; nothing is ever counted twice.
- **Per-complexity models inside a task rule**: each task is edited as rows of
"model + complexity" pairs — `[routing.tasks.code] low=["cheap"] high=["best"] any=[...]`.
The tier-specific pick leads its own tier, `any` (and legacy plain arrays) leads everywhere,
the global tier chain and `default` always remain as fallback, and high verdicts still
escalate over any-tier pins.
### Changed
- Benchmark hard probes are generated from wider interdisciplinary pools (6 families each) and
retry once with a fresh template — the router's novelty decay no longer flips them red.
### Fixed
- The `task-rule` badge is honest: it appears only when the rule actually chose the head of the
candidate chain, not whenever a rule merely exists for the task.
## [0.2.39] - 2026-08-07
### Added
- **Benchmark, redesigned into the console** (⏱ sidebar entry): two explicit modes. **Smart
routing** (default) sends every probe through `cortiq-auto` and judges the ROUTER — task label
vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge —
with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a
"requests went to" per-model distribution. **Individual models** compares pinned models
head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run ×
and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
- **Hard probes that exercise the high tier**: two route-judged tests (science / ops pools)
generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings
decay with the router's novelty signal. Pass = the router said `high` and the request escalated;
the tiny token budget keeps them at cents even on the priciest model.
- **One-paste harness setup** on the API-keys page: Claude Code (native Anthropic `/v1/messages`,
now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline —
each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses
already installed on its machine (✓ on the tab, "already configured for this gateway" status).
- **Stale-tab detection**: open consoles notice a gateway restart via the meta start marker and
show a reload banner instead of silently running an old interface.
- Log viewer Clear button (`DELETE /admin/api/logs`).
### Changed
- **High complexity escalates over task rules**: at tier `high` the tier chain leads and the task
rule demotes to fallback — a `code → cheap-model` rule no longer pins the hardest prompts to
the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
- **Honest stream costs**: the final cost rides a closing SSE event (`cortiq_cost_usd`) since the
initial header is written before generation; money renders with 4 decimals (last two smaller)
and grows precision below $0.0001 instead of collapsing to $0.0000.
- **Oracle-sized timeouts**: router default 800 ms → 15 s (hosted escalations take ~10 s);
provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall
cap so long generations and streams are never cut.
- Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-`<think>`),
router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on
reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
- New-key dialog simplified to account / rate limit / optional custom value.
### Fixed
- `cortiq bench` results showed "error" with newer cortiq releases that pretty-print `--json`:
the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers
steady decode speed (the old heuristic could report `allocs_per_token` as tok/s).
## [0.2.38] - 2026-08-07
### Added
Generated
+1 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "cortiq-gateway"
version = "0.2.38"
version = "0.2.41"
dependencies = [
"anyhow",
"arc-swap",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "cortiq-gateway"
version = "0.2.38"
version = "0.2.41"
edition = "2021"
rust-version = "1.75"
license = "Apache-2.0"
+5 -1
View File
@@ -71,7 +71,7 @@ openai_chat = true
openai_completions = true
openai_embeddings = true
openai_models = true
anthropic_messages = false
anthropic_messages = true
mcp = false
native_passthrough = true
@@ -119,6 +119,10 @@ enabled = true
file = "config/stats.jsonl" # empty = in-memory only
ring_size = 500 # how many recent requests to keep in the feed
retention = "7d" # aggregate/series depth (replayed on startup)
# stats.jsonl is compacted once it exceeds this size: old records fold into
# cumulative per-account totals (sidecar file), so per-key accounting survives
# while the disk stays bounded. 0 disables.
max_file_mb = 16
# ============================================================================
# Semantic cache — returns a cached answer for prompts semantically near a
+22 -1
View File
@@ -177,7 +177,28 @@ code = ["claude-sonnet", "gpt"] # code goes to the strongest coder first
A matching rule's models go **first** in the candidate chain; the complexity
chain follows as fallback, so a task pin never weakens failover. Tasks without
a rule route by complexity alone. The available labels come from the router
a rule route by complexity alone.
**Per-complexity models for a task.** A task rule also accepts a per-tier
map — "which model serves this task at each complexity":
```toml
[routing.tasks.code]
low = ["cheap-coder"] # routine code stays cheap
high = ["best-coder"] # hard code goes to the strongest coder
any = ["generic-coder"] # leads at every tier after the tier-specific pick
```
The tier-specific list leads its own tier; `any` (and the legacy plain-array
form) leads at every tier; the global tier chain and `default` always remain
as fallback, so a task rule never weakens failover. The admin console's
Routing page edits this as rows of “model + complexity” pairs per task.
**Exception — high complexity escalates.** When the router rates a request
`high` (and a `high` chain is configured), the tier chain leads and the task
rule demotes to fallback. Otherwise a rule like `code → cheap-model` would pin
even the hardest prompts to the cheap model and the tier system would never
get a say. Low/medium keep task-rule-first semantics. The available labels come from the router
(`GET /v1/taxonomies`) — the admin console's Routing page builds the editor
from your account's live taxonomy and shows each task's share of recent
traffic. When a rule supplied the head of the chain the response reports
+22 -1
View File
@@ -175,7 +175,28 @@ code = ["claude-sonnet", "gpt"] # код — сильнейшему ко
Модели подходящего правила идут **первыми** в цепочке кандидатов; цепочка по
сложности следует за ними как резерв — правило по задаче не ослабляет failover.
Задачи без правила маршрутизируются только по сложности. Список меток приходит
Задачи без правила маршрутизируются только по сложности.
**Модели по сложности внутри задачи.** Правило задачи принимает и карту по
уровням — «какая модель обслуживает задачу на какой сложности»:
```toml
[routing.tasks.code]
low = ["cheap-coder"] # рутинный код — дёшево
high = ["best-coder"] # сложный код — сильнейшему кодеру
any = ["generic-coder"] # ведёт на всех уровнях после уровневого выбора
```
Список конкретного уровня ведёт на своём уровне; `any` (и старая форма-массив)
ведёт на всех; глобальная цепочка уровня и `default` всегда остаются резервом —
правило задачи не ослабляет failover. На вкладке Роутинг это редактируется
строками пар «модель + сложность» у каждой задачи.
**Исключение — высокая сложность эскалирует.** Когда роутер оценил запрос как
`high` (и цепочка `high` настроена), цепочка уровня идёт первой, а правило
задачи становится резервом. Иначе правило вида `code → дешёвая-модель`
приковало бы к дешёвой модели даже самые сложные запросы, и система уровней
не имела бы права голоса. Для low/medium правило задачи ведёт, как и раньше. Список меток приходит
с роутера (`GET /v1/taxonomies`) — вкладка Роутинг строит редактор по живой
таксономии аккаунта и показывает долю каждой задачи в трафике. Когда цепочку
возглавило правило задачи, ответ сообщает `route_source: "router+task-rule"`.
+125 -23
View File
@@ -116,13 +116,7 @@ fn tiers_to_map(routing: &RoutingCfg) -> HashMap<String, Vec<String>> {
routing
.tiers
.iter()
.map(|(k, v)| {
let list = match v {
TierTargets::List(l) => l.clone(),
TierTargets::One(s) => vec![s.clone()],
};
(k.clone(), list)
})
.map(|(k, v)| (k.clone(), v.models()))
.collect()
}
@@ -160,6 +154,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
"/admin/api/protocols",
get(get_protocols).put(put_protocols),
)
.route("/admin/api/harnesses", get(get_harnesses))
.route("/admin/api/settings", get(get_settings).put(put_settings))
.route("/admin/api/keys", get(list_keys).post(create_key))
.route("/admin/api/keys/:key", axum::routing::delete(delete_key))
@@ -185,7 +180,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
.route("/admin/api/requests", get(get_requests))
.route("/admin/api/test", post(run_test))
.route("/admin/api/test/stream", post(run_test_stream))
.route("/admin/api/logs", get(get_logs))
.route("/admin/api/logs", get(get_logs).delete(clear_logs))
.route("/admin/api/system", get(get_system))
.route("/admin/api/restart", post(restart_gateway))
.route("/admin/api/bench", get(bench_list).post(bench_start))
@@ -599,8 +594,18 @@ async fn provider_models(
}
async fn meta(State(state): State<SharedState>) -> ApiResult {
// process start marker: open tabs poll this and offer a reload when the
// gateway (and thus the embedded SPA) has been restarted/updated
static STARTED: std::sync::OnceLock<u64> = std::sync::OnceLock::new();
let started = *STARTED.get_or_init(|| {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
});
ok(json!({
"version": env!("CARGO_PKG_VERSION"),
"started": started,
"update_available": state.gateway_update.lock().unwrap().clone(),
"admin_open_public": state
.admin_open_public
@@ -747,6 +752,13 @@ fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
}
for tt in routing.tiers.values_mut().chain(routing.tasks.values_mut()) {
match tt {
crate::config::TierTargets::Map(m) => {
for v in m.values_mut() {
if let crate::config::TaskTierVal::List(l) = v {
l.retain(|x| x != id);
}
}
}
crate::config::TierTargets::List(v) => v.retain(|x| x != id),
crate::config::TierTargets::One(s) if s == id => {
*tt = crate::config::TierTargets::List(Vec::new());
@@ -755,10 +767,7 @@ fn strip_id_from_routing(routing: &mut RoutingCfg, id: &str) {
}
}
// an emptied task rule means "no rule", not "no candidates"
routing.tasks.retain(|_, tt| match tt {
crate::config::TierTargets::List(v) => !v.is_empty(),
crate::config::TierTargets::One(s) => !s.is_empty(),
});
routing.tasks.retain(|_, tt| !tt.models().is_empty());
}
/// Whether any managed server or static model in `cfg` still uses `path` as its
@@ -899,23 +908,30 @@ async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -
async fn get_routing(State(state): State<SharedState>) -> ApiResult {
let live = state.live();
let tasks: HashMap<String, Vec<String>> = live
let tasks: HashMap<String, serde_json::Value> = live
.cfg
.routing
.tasks
.iter()
.map(|(k, v)| {
(
k.clone(),
match v {
TierTargets::List(l) => l.clone(),
TierTargets::One(s) => vec![s.clone()],
},
)
(k.clone(), {
let (any, per) = v.task_rule();
let mut o = serde_json::Map::new();
if !any.is_empty() {
o.insert("any".into(), json!(any));
}
for (tier, models) in per {
if !models.is_empty() {
o.insert(tier, json!(models));
}
}
serde_json::Value::Object(o)
})
})
.collect();
ok(json!({
"tiers": tiers_to_map(&live.cfg.routing),
"bands": live.cfg.routing.bands,
"tasks": tasks,
"default": live.cfg.routing.default,
"policy": live.cfg.routing.policy,
@@ -962,7 +978,9 @@ struct RoutingBody {
#[serde(default)]
tiers: HashMap<String, Vec<String>>,
#[serde(default)]
tasks: HashMap<String, Vec<String>>,
tasks: HashMap<String, TierTargets>,
#[serde(default)]
bands: Option<crate::config::TierBands>,
default: String,
#[serde(default)]
policy: RoutingPolicy,
@@ -979,9 +997,10 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
tasks: body
.tasks
.into_iter()
.filter(|(_, v)| !v.is_empty()) // empty rule = "по сложности", not a rule
.map(|(k, v)| (k, TierTargets::List(v)))
// empty rule (no models AND no forced tier) = "по сложности"
.filter(|(_, v)| !v.models().is_empty())
.collect(),
bands: body.bands,
default: body.default,
policy: body.policy,
};
@@ -989,6 +1008,84 @@ async fn put_routing(State(state): State<SharedState>, Json(body): Json<RoutingB
ok(json!({ "ok": true }))
}
// ──────────────────────────── harness detection ────────────────────────────
// Which coding harnesses are present on the machine the gateway runs on, so the
// API-keys view can mark tabs "installed" / "already configured for cortiq".
// Detection is best-effort: binaries in PATH + common install dirs, config
// files under the user's home. Works on Linux/macOS/Windows.
fn harness_home() -> Option<std::path::PathBuf> {
std::env::var_os("HOME")
.or_else(|| std::env::var_os("USERPROFILE"))
.map(std::path::PathBuf::from)
}
fn find_bin(name: &str) -> bool {
let exts: &[&str] = if cfg!(windows) {
&[".exe", ".cmd", ".bat", ""]
} else {
&[""]
};
let mut dirs: Vec<std::path::PathBuf> = std::env::var_os("PATH")
.map(|p| std::env::split_paths(&p).collect())
.unwrap_or_default();
if let Some(home) = harness_home() {
// GUI-launched processes often miss the shell PATH — add usual suspects
for extra in [
".local/bin",
".cargo/bin",
"bin",
".bun/bin",
".npm-global/bin",
"AppData/Roaming/npm",
] {
dirs.push(home.join(extra));
}
}
dirs.extend(["/usr/local/bin", "/opt/homebrew/bin"].map(std::path::PathBuf::from));
dirs.iter()
.any(|d| exts.iter().any(|e| d.join(format!("{name}{e}")).is_file()))
}
/// Does the harness config mention cortiq (i.e. was it pointed at this gateway)?
fn config_mentions_cortiq(rel: &str) -> bool {
harness_home()
.map(|h| h.join(rel))
.and_then(|p| std::fs::read_to_string(p).ok())
.map(|c| c.to_lowercase().contains("cortiq"))
.unwrap_or(false)
}
async fn get_harnesses() -> ApiResult {
let harnesses = tokio::task::spawn_blocking(|| {
let vscode_ext_glob = |marker: &str| -> bool {
let Some(home) = harness_home() else {
return false;
};
for extdir in [".vscode/extensions", ".vscode-server/extensions"] {
if let Ok(rd) = std::fs::read_dir(home.join(extdir)) {
for e in rd.flatten() {
if e.file_name().to_string_lossy().starts_with(marker) {
return true;
}
}
}
}
false
};
json!({
"Claude Code": { "installed": find_bin("claude"), "configured": serde_json::Value::Null },
"Codex": { "installed": find_bin("codex"), "configured": config_mentions_cortiq(".codex/config.toml") },
"OpenCode": { "installed": find_bin("opencode"), "configured": config_mentions_cortiq(".config/opencode/opencode.json") },
"Continue": { "installed": harness_home().map(|h| h.join(".continue").is_dir()).unwrap_or(false), "configured": config_mentions_cortiq(".continue/config.yaml") },
"Cline": { "installed": vscode_ext_glob("saoudrizwan.claude-dev"), "configured": serde_json::Value::Null },
})
})
.await
.unwrap_or_else(|_| json!({}));
ok(harnesses)
}
// ───────────────────────────── protocols ───────────────────────────────────
async fn get_protocols(State(state): State<SharedState>) -> ApiResult {
@@ -2113,6 +2210,11 @@ async fn get_logs(Query(q): Query<RecentQuery>) -> ApiResult {
ok(json!({ "lines": crate::logbuf::snapshot(limit) }))
}
async fn clear_logs() -> ApiResult {
crate::logbuf::clear();
ok(json!({ "ok": true }))
}
/// Host metrics snapshot: CPU, RAM, free disk at the models dir, gateway RSS.
async fn get_system(State(state): State<SharedState>) -> ApiResult {
let snap = state.sysmon.snapshot();
+71 -12
View File
@@ -32,11 +32,19 @@ fn now() -> u64 {
.as_secs()
}
/// tok/s out of the benchmark's JSON: first numeric field whose key mentions
/// tokens-per-second (the exact name has varied across cortiq releases).
/// tok/s out of the benchmark's JSON: known field names first (steady decode
/// speed is the honest number), then any `*tok_s*` field as a fallback. The
/// old loose "contains tok + per" heuristic could grab `allocs_per_token`.
fn extract_tok_s(v: &serde_json::Value) -> Option<f64> {
let obj = v.as_object()?;
for want in ["tok_s", "tok_per_s", "tokens_per_sec", "decode_tok_s"] {
for want in [
"decode_tok_s_steady",
"decode_tok_s_incl_prefill",
"decode_tok_s",
"tok_s",
"tok_per_s",
"tokens_per_sec",
] {
if let Some(n) = obj.get(want).and_then(|x| x.as_f64()) {
return Some(n);
}
@@ -44,12 +52,44 @@ fn extract_tok_s(v: &serde_json::Value) -> Option<f64> {
obj.iter()
.find(|(k, val)| {
let k = k.to_ascii_lowercase();
(k.contains("tok") && (k.contains("/s") || k.contains("_s") || k.contains("per")))
&& val.as_f64().is_some()
(k.contains("tok_s") || k.contains("tokens_per")) && val.as_f64().is_some()
})
.and_then(|(_, val)| val.as_f64())
}
/// Last parseable JSON object in mixed CLI output. Newer cortiq releases
/// pretty-print the `--json` result over many lines, older ones emit a single
/// line — this handles both, skipping any log noise around it.
fn last_json_object(text: &str) -> Option<serde_json::Value> {
let mut end = text.len();
while let Some(close) = text[..end].rfind('}') {
let mut depth = 0i32;
let mut start = None;
for (i, c) in text[..=close].char_indices().rev() {
match c {
'}' => depth += 1,
'{' => {
depth -= 1;
if depth == 0 {
start = Some(i);
break;
}
}
_ => {}
}
}
if let Some(st) = start {
if let Ok(v) = serde_json::from_str::<serde_json::Value>(&text[st..=close]) {
if v.is_object() {
return Some(v);
}
}
}
end = close;
}
None
}
impl BenchStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
@@ -125,13 +165,7 @@ pub fn start(
if n > 30 {
run.log.drain(0..n - 30);
}
// the --json contract: one JSON object on its own line
let parsed = text
.lines()
.rev()
.map(str::trim)
.find(|l| l.starts_with('{') && l.ends_with('}'))
.and_then(|l| serde_json::from_str::<serde_json::Value>(l).ok());
let parsed = last_json_object(&text);
match parsed {
Some(v) if o.status.success() => {
run.tok_s = extract_tok_s(&v);
@@ -163,4 +197,29 @@ mod tests {
let v = serde_json::json!({"decode_tokens_per_sec": 31.0});
assert_eq!(extract_tok_s(&v), Some(31.0));
}
#[test]
fn steady_decode_wins_and_allocs_never_matches() {
let v = serde_json::json!({
"allocs_per_token": 769.4,
"decode_tok_s_incl_prefill": 9.98,
"decode_tok_s_steady": 11.86,
"prefill_tok_s": 25.5,
});
assert_eq!(extract_tok_s(&v), Some(11.86));
let v = serde_json::json!({"allocs_per_token": 769.4, "seq_len": 140});
assert_eq!(extract_tok_s(&v), None);
}
#[test]
fn json_found_single_line_and_pretty() {
let single = "noise\n{\"tok_s\": 5.0}\nmore noise";
assert_eq!(last_json_object(single).unwrap()["tok_s"], 5.0);
let pretty = "INFO loading\n{\n \"decode_tok_s_steady\": 11.86,\n \"seq_len\": 140\n}\nINFO done {not json";
assert_eq!(
last_json_object(pretty).unwrap()["decode_tok_s_steady"],
11.86
);
assert!(last_json_object("no braces here").is_none());
}
}
+163 -8
View File
@@ -305,7 +305,9 @@ fn default_router_enabled() -> bool {
true
}
fn default_router_timeout() -> u64 {
800
// the hosted router escalates hard prompts to an oracle (~10 s);
// an 800 ms budget silently degraded every one of those to the fallback
15_000
}
fn default_verify_tls() -> bool {
true
@@ -402,17 +404,143 @@ pub struct RoutingCfg {
/// called "tasks" — serde extracts it before the flattened tier map.
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
pub tasks: HashMap<String, TierTargets>,
/// Local complexity bands: re-map the router's continuous score into
/// low/medium/high with YOUR OWN boundaries. The hosted router's bands
/// are global for all customers; these are per-gateway. `None` = keep
/// the router's tier verdict as delivered.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bands: Option<TierBands>,
}
/// Score boundaries: `score ≤ low` → low, `≤ medium` → medium, else high.
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
pub struct TierBands {
pub low: f32,
pub medium: f32,
}
impl TierBands {
pub fn tier_of(&self, score: f32) -> &'static str {
if score <= self.low {
"low"
} else if score <= self.medium {
"medium"
} else {
"high"
}
}
}
/// A routing tier is either a list of model_ids or the string `default`.
/// (`default` is handled by a separate field; only tier lists appear here.)
///
/// TASK rules additionally accept a per-complexity map — "which model serves
/// this task at each tier":
///
/// ```toml
/// [routing.tasks.code]
/// low = ["cheap-coder"]
/// high = ["big-model"]
/// any = ["fallback"] # leads at every tier after the tier-specific list
/// ```
///
/// The plain-array form stays valid and means "leads at any tier". The
/// transitional `{ models = [...], tier = "high" }` form is normalized into
/// the map on load. Untagged: map is tried first, arrays keep parsing as is.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum TierTargets {
Map(HashMap<String, TaskTierVal>),
List(Vec<String>),
One(String),
}
/// A value inside a task's per-tier map: a model list, a single model, or the
/// transitional string value of the legacy `tier = "high"` key.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum TaskTierVal {
List(Vec<String>),
Str(String),
}
impl TaskTierVal {
fn list(&self) -> Vec<String> {
match self {
TaskTierVal::List(v) => v.clone(),
TaskTierVal::Str(s) => vec![s.clone()],
}
}
}
pub const TASK_TIER_KEYS: [&str; 4] = ["any", "low", "medium", "high"];
impl TierTargets {
/// Every referenced model id, whatever the syntactic form (validation).
pub fn models(&self) -> Vec<String> {
match self {
TierTargets::Map(m) => {
let mut out = Vec::new();
for (k, v) in m {
if k == "tier" {
continue; // legacy marker, not a model list
}
out.extend(v.list());
}
out
}
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
}
}
/// Normalized task rule: `(any_tier_models, per_tier_models)`.
/// Legacy `{models, tier}` becomes `per[tier] = models`; a plain array
/// becomes `any`.
pub fn task_rule(&self) -> (Vec<String>, HashMap<String, Vec<String>>) {
match self {
TierTargets::Map(m) => {
let mut any: Vec<String> = Vec::new();
let mut per: HashMap<String, Vec<String>> = HashMap::new();
let legacy_tier = m.get("tier").and_then(|v| match v {
TaskTierVal::Str(s) => Some(s.clone()),
_ => None,
});
for (k, v) in m {
match k.as_str() {
"tier" => {}
"any" | "models" => any.extend(v.list()),
"low" | "medium" | "high" => {
per.entry(k.clone()).or_default().extend(v.list())
}
_ => {}
}
}
if let Some(tier) = legacy_tier {
// legacy pin: the models were meant for that tier
let moved = std::mem::take(&mut any);
per.entry(tier).or_default().extend(moved);
}
(any, per)
}
TierTargets::List(v) => (v.clone(), HashMap::new()),
TierTargets::One(s) => (vec![s.clone()], HashMap::new()),
}
}
/// Unknown keys inside a task map (validation).
pub fn invalid_task_keys(&self) -> Vec<String> {
match self {
TierTargets::Map(m) => m
.keys()
.filter(|k| !TASK_TIER_KEYS.contains(&k.as_str()) && *k != "models" && *k != "tier")
.cloned()
.collect(),
_ => Vec::new(),
}
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RoutingPolicy {
#[serde(default = "default_mode")]
@@ -585,6 +713,12 @@ pub struct StatsCfg {
/// Retention window for aggregates/time-series replayed on startup, e.g. `7d`.
#[serde(default = "default_stats_retention")]
pub retention: String,
/// Compact the JSONL file once it exceeds this size: records older than
/// `retention` fold into cumulative per-account totals (a small sidecar
/// file) and are removed, so per-key accounting survives while the disk
/// stays bounded. 0 disables compaction.
#[serde(default = "default_stats_max_file_mb")]
pub max_file_mb: u64,
}
impl Default for StatsCfg {
fn default() -> Self {
@@ -593,6 +727,7 @@ impl Default for StatsCfg {
file: default_stats_file(),
ring_size: default_stats_ring(),
retention: default_stats_retention(),
max_file_mb: default_stats_max_file_mb(),
}
}
}
@@ -608,6 +743,9 @@ fn default_stats_ring() -> usize {
fn default_stats_retention() -> String {
"7d".into()
}
fn default_stats_max_file_mb() -> u64 {
16
}
/// Self-warming shadow loop (docs/SELF_WARMING_GATEWAY.md). While a task
/// label is not yet promoted, the client is served the CLOUD answer; in
@@ -736,7 +874,7 @@ impl Default for Config {
router: RouterCfg {
enabled: true,
url: "https://138.226.222.209".to_string(),
timeout_ms: 12_000, // hard prompts escalate to an oracle (~10 s)
timeout_ms: 15_000, // hard prompts escalate to an oracle (~10 s)
verify_tls: false, // the hosted endpoint is IP-bound
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
taxonomy_id: Some("data-assistant".to_string()),
@@ -748,6 +886,7 @@ impl Default for Config {
default: String::new(),
policy: RoutingPolicy::default(),
tasks: HashMap::new(),
bands: None,
},
breaker: BreakerCfg::default(),
protocols: ProtocolsCfg {
@@ -755,7 +894,9 @@ impl Default for Config {
openai_completions: true,
openai_embeddings: true,
openai_models: true,
anthropic_messages: false,
// on by default: the API-keys view ships a Claude Code
// auto-setup snippet that targets /v1/messages
anthropic_messages: true,
mcp: false,
native_passthrough: true,
},
@@ -859,6 +1000,7 @@ impl Config {
}
for (tier, targets) in &self.routing.tiers {
let list = match targets {
TierTargets::Map(_) => targets.models(),
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
@@ -872,12 +1014,17 @@ impl Config {
}
}
}
if let Some(b) = &self.routing.bands {
if !(b.low > 0.0 && b.low < b.medium && b.medium < 1.0) {
anyhow::bail!(
"config: routing.bands must satisfy 0 < low < medium < 1 (got {} / {})",
b.low,
b.medium
);
}
}
for (task, targets) in &self.routing.tasks {
let list = match targets {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
};
for id in &list {
for id in &targets.models() {
if !ids.contains(id.as_str()) {
anyhow::bail!(
"config: routing.tasks.{} references unknown model id '{}'",
@@ -886,6 +1033,13 @@ impl Config {
);
}
}
if let Some(bad) = targets.invalid_task_keys().first() {
anyhow::bail!(
"config: routing.tasks.{}.{} is not a tier (use any/low/medium/high)",
task,
bad
);
}
}
Ok(())
}
@@ -977,6 +1131,7 @@ min_class = { low = "cheap", medium = "mid" }
let low = back.routing.tiers.get("low").expect("low tier present");
match low {
TierTargets::Map(_) => assert_eq!(low.models(), vec!["local-qwen".to_string()]),
TierTargets::List(v) => assert_eq!(v, &vec!["local-qwen".to_string()]),
TierTargets::One(s) => assert_eq!(s, "local-qwen"),
}
+5
View File
@@ -23,6 +23,11 @@ pub fn snapshot(limit: usize) -> Vec<String> {
.collect()
}
/// Drop everything in the ring (the admin console's "clear" button).
pub fn clear() {
ring().lock().unwrap().clear();
}
pub fn push_line(line: &str) {
let line = line.trim_end();
if line.is_empty() {
+37 -17
View File
@@ -231,6 +231,19 @@ impl Pipeline {
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
match live.router.route(&text, prof).await {
Ok(Some(dec)) => {
let mut dec = dec;
// per-gateway complexity bands: re-map the
// router's continuous score with the user's
// own boundaries (sliders on the Routing page)
if let Some(b) = &live.cfg.routing.bands {
if matches!(
dec.complexity_tier.as_str(),
"low" | "medium" | "high"
) {
dec.complexity_tier =
b.tier_of(dec.complexity_score).to_string();
}
}
let src = dec.source.clone();
let (cands, task_rule) = live
.routing
@@ -798,28 +811,35 @@ fn tap_stream(
// inject one final OpenAI-shaped usage event with the gateway's
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
// mistakes it for the tokenizer's truth.
if done_marker && !got_usage {
let est_completion = if delta_chunks > 0 {
delta_chunks
} else {
(delta_chars / 4) as u32
};
let est_prompt = (prompt_chars / 4) as u32;
let evt = serde_json::json!({
if done_marker {
// The initial X-Cortiq-Cost-USD header is always 0 — cost is
// only known after generation. Ship the final number (and the
// token estimate when the provider sent none) ahead of [DONE].
if !got_usage {
prompt_tokens = (prompt_chars / 4) as u32;
completion_tokens = if delta_chunks > 0 {
delta_chunks
} else {
(delta_chars / 4) as u32
};
}
let cost = provider.price(prompt_tokens, completion_tokens);
let mut evt = serde_json::json!({
"object": "chat.completion.chunk",
"model": info.selected_model,
"choices": [],
"usage": {
"prompt_tokens": est_prompt,
"completion_tokens": est_completion,
"total_tokens": est_prompt + est_completion,
},
"cortiq_estimated": true,
"cortiq_cost_usd": cost,
});
if !got_usage {
evt["usage"] = serde_json::json!({
"prompt_tokens": prompt_tokens,
"completion_tokens": completion_tokens,
"total_tokens": prompt_tokens + completion_tokens,
});
evt["cortiq_estimated"] = serde_json::json!(true);
got_usage = true;
}
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
prompt_tokens = est_prompt;
completion_tokens = est_completion;
got_usage = true;
}
yield item;
}
+7 -1
View File
@@ -32,7 +32,13 @@ impl AnthropicProvider {
caps: Caps::from_list(&cfg.caps),
price_in: cfg.price_in,
price_out: cfg.price_out,
http: reqwest::Client::new(),
// connect timeout: a dead host fails over in seconds instead of
// hanging the request; NO overall timeout — that would cut long
// generations and streams (OpenRouter can be slow legitimately)
http: reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_else(|_| reqwest::Client::new()),
})
}
+7 -1
View File
@@ -28,7 +28,13 @@ impl OpenAiProvider {
caps: Caps::from_list(&cfg.caps),
price_in: cfg.price_in,
price_out: cfg.price_out,
http: reqwest::Client::new(),
// connect timeout: a dead host fails over in seconds instead of
// hanging the request; NO overall timeout — that would cut long
// generations and streams (OpenRouter can be slow legitimately)
http: reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_else(|_| reqwest::Client::new()),
})
}
+147 -27
View File
@@ -4,16 +4,21 @@
use crate::config::{Config, TierTargets};
fn to_list(t: &TierTargets) -> Vec<String> {
match t {
TierTargets::List(v) => v.clone(),
TierTargets::One(s) => vec![s.clone()],
}
t.models()
}
/// A task rule, normalized: per-tier model lists ("which model serves this
/// task at each complexity") plus any-tier models that lead regardless.
#[derive(Clone, Default)]
pub struct TaskRule {
pub any: Vec<String>,
pub per: std::collections::HashMap<String, Vec<String>>,
}
pub struct RoutingTable {
tiers: std::collections::HashMap<String, Vec<String>>,
/// Per-task rules — a matching rule's models lead the candidate chain.
tasks: std::collections::HashMap<String, Vec<String>>,
tasks: std::collections::HashMap<String, TaskRule>,
default: String,
}
@@ -30,36 +35,78 @@ impl RoutingTable {
.routing
.tasks
.iter()
.map(|(task, targets)| (task.clone(), to_list(targets)))
.filter(|(_, v)| !v.is_empty())
.map(|(task, targets)| {
let (any, per) = targets.task_rule();
(task.clone(), TaskRule { any, per })
})
.filter(|(_, r)| !r.any.is_empty() || r.per.values().any(|v| !v.is_empty()))
.collect(),
default: cfg.routing.default.clone(),
}
}
/// Candidates for `(task, tier)`, in preference order, deduplicated:
/// the task rule's models first (when one exists), then the tier chain —
/// so a task pin adds preference without ever weakening failover.
/// Returns `(candidates, task_rule_matched)`.
/// Candidate chain for `(task, tier)` and whether a task rule shaped it.
///
/// Order of preference:
/// 1. the task's models FOR THIS TIER (the user said exactly which model
/// serves this task at this complexity);
/// 2. at tier=high — the global high chain (escalation: any-tier task
/// models must not undercut it), then the task's any-tier models;
/// otherwise the task's any-tier models, then the global tier chain;
/// 3. `default` as the last resort. Deduplicated, empties dropped.
pub fn candidates_for(&self, task: &str, tier: &str) -> (Vec<String>, bool) {
fn push(out: &mut Vec<String>, id: &str) {
if !id.is_empty() && !out.iter().any(|x| x == id) {
out.push(id.to_string());
}
}
let mut out: Vec<String> = Vec::new();
let matched = if let Some(rule) = self.tasks.get(task) {
out.extend(rule.iter().cloned());
true
} else {
false
};
let rule = self.tasks.get(task);
let tier_chain = self
.tiers
.get(tier)
.cloned()
.unwrap_or_else(|| vec![self.default.clone()]);
for id in tier_chain {
if !out.contains(&id) {
out.push(id);
if let Some(r) = rule {
if let Some(list) = r.per.get(tier) {
for id in list {
push(&mut out, id);
}
}
}
out.retain(|id| !id.is_empty());
let escalate = tier == "high" && self.tiers.contains_key("high");
if escalate {
// high verdicts escalate over any-tier task pins
for id in &tier_chain {
push(&mut out, id);
}
if let Some(r) = rule {
for id in &r.any {
push(&mut out, id);
}
}
} else {
if let Some(r) = rule {
for id in &r.any {
push(&mut out, id);
}
}
for id in &tier_chain {
push(&mut out, id);
}
}
let d = self.default.clone();
push(&mut out, &d);
// "task-rule" must mean "the rule chose the head of this chain" — a
// rule that exists but lost to the tier chain (math has a high pick,
// the request came in low) must not claim the credit.
let matched = match (rule, out.first()) {
(Some(r), Some(head)) => {
r.per.get(tier).map(|v| v.contains(head)).unwrap_or(false)
|| r.any.first() == Some(head)
}
_ => false,
};
(out, matched)
}
@@ -91,12 +138,33 @@ mod tests {
.collect(),
tasks: tasks
.iter()
.map(|(k, v)| (k.to_string(), v.iter().map(|s| s.to_string()).collect()))
.map(|(k, v)| {
(
k.to_string(),
TaskRule {
any: v.iter().map(|s| s.to_string()).collect(),
per: Default::default(),
},
)
})
.collect(),
default: default.to_string(),
}
}
fn with_task_tier_models(
mut t: RoutingTable,
task: &str,
tier: &str,
models: &[&str],
) -> RoutingTable {
t.tasks.entry(task.to_string()).or_default().per.insert(
tier.to_string(),
models.iter().map(|s| s.to_string()).collect(),
);
t
}
#[test]
fn task_rule_leads_tier_chain_follows() {
let t = table(
@@ -104,18 +172,70 @@ mod tests {
&[("low", &["cheap", "local"]), ("high", &["big"])],
"cheap",
);
let (c, matched) = t.candidates_for("translation", "high");
assert!(matched);
assert_eq!(c, vec!["local", "big"]);
// dedup: rule model already in the tier chain appears once, first
let (c, _) = t.candidates_for("translation", "low");
let (c, matched) = t.candidates_for("translation", "low");
assert!(matched);
assert_eq!(c, vec!["local", "cheap"]);
}
#[test]
fn high_tier_escalates_over_task_rule() {
let t = table(
&[("code", &["cheap-coder"])],
&[("low", &["cheap"]), ("high", &["big"])],
"cheap",
);
// medium (no chain defined → default): task rule still leads
let (c, m) = t.candidates_for("code", "medium");
assert!(m);
assert_eq!(c[0], "cheap-coder");
// high: tier chain leads, task rule follows as fallback
let (c, m) = t.candidates_for("code", "high");
// escalation put the global high chain first → the rule did not pick
// the head, so no task-rule credit
assert!(!m);
// default rides along as the very last fallback
assert_eq!(c, vec!["big", "cheap-coder", "cheap"]);
}
#[test]
fn per_tier_models_lead_their_tier() {
let base = table(
&[("code", &["generic-coder"])],
&[("low", &["cheap"]), ("high", &["big"])],
"cheap",
);
let t = with_task_tier_models(base, "code", "high", &["best-coder"]);
// at high: the task's HIGH pick leads, escalation chain next,
// any-tier task models after it, default last
let (c, m) = t.candidates_for("code", "high");
assert!(m);
assert_eq!(c, vec!["best-coder", "big", "generic-coder", "cheap"]);
// at low: any-tier task models lead as before (and the badge is
// honest — the head came from the rule)
let (c, m) = t.candidates_for("code", "low");
assert!(m);
assert_eq!(c, vec!["generic-coder", "cheap"]);
// a tier with an explicit pick but no any-models
let t2 = with_task_tier_models(
table(&[], &[("low", &["cheap"]), ("high", &["big"])], "cheap"),
"legal",
"high",
&["strong"],
);
let (c, m) = t2.candidates_for("legal", "high");
assert!(m);
assert_eq!(c, vec!["strong", "big", "cheap"]);
// the same rule at LOW did not shape the head → no task-rule credit
let (c, m) = t2.candidates_for("legal", "low");
assert!(!m);
assert_eq!(c, vec!["cheap"]);
}
#[test]
fn no_rule_falls_back_to_tier_then_default() {
let t = table(&[], &[("low", &["a"])], "d");
assert_eq!(t.candidates_for("qa", "low").0, vec!["a"]);
assert_eq!(t.candidates_for("qa", "low").0, vec!["a", "d"]);
let (c, matched) = t.candidates_for("qa", "unknown-tier");
assert!(!matched);
assert_eq!(c, vec!["d"]);
+154 -1
View File
@@ -55,7 +55,7 @@ pub struct RequestRecord {
pub error: Option<String>,
}
#[derive(Clone, Debug, Default, Serialize)]
#[derive(Clone, Debug, Default, Serialize, serde::Deserialize)]
pub struct Totals {
pub requests: u64,
pub ok: u64,
@@ -130,6 +130,15 @@ struct Inner {
/// Per-minute buckets — the single source for the windowed admin snapshot.
buckets: VecDeque<Bucket>,
recent: VecDeque<RequestRecord>,
/// Per-account totals over EVERY record still in the JSONL file (no
/// retention gate) — per-key accounting for the keys page.
by_account_alltime: HashMap<String, Totals>,
/// Per-account totals folded out of the file by compaction. alltime(key)
/// = folded + by_account_alltime, with no double counting: a record is
/// either in the file or folded, never both.
folded: HashMap<String, Totals>,
/// Appends since the last compaction size check.
writes_since_check: u32,
}
pub struct Stats {
@@ -137,6 +146,7 @@ pub struct Stats {
file: Option<String>,
ring_size: usize,
retention_secs: u64,
max_file_bytes: u64,
inner: Mutex<Inner>,
}
@@ -152,9 +162,12 @@ impl Stats {
file,
ring_size: cfg.ring_size.max(1),
retention_secs: parse_duration_secs(&cfg.retention).unwrap_or(7 * 86_400),
max_file_bytes: cfg.max_file_mb.saturating_mul(1024 * 1024),
inner: Mutex::new(Inner::default()),
};
s.load_folded();
s.replay();
s.maybe_compact(true);
std::sync::Arc::new(s)
}
@@ -175,6 +188,11 @@ impl Stats {
// push everything to the "recent" ring (it self-limits by size),
// but apply to aggregates/series only entries within the window.
Self::push_recent(&mut inner, &rec, self.ring_size);
inner
.by_account_alltime
.entry(rec.account.clone())
.or_default()
.apply(&rec);
if rec.ts >= cutoff {
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
}
@@ -233,6 +251,12 @@ impl Stats {
let mut inner = self.inner.lock().unwrap();
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
Self::push_recent(&mut inner, &rec, self.ring_size);
inner
.by_account_alltime
.entry(rec.account.clone())
.or_default()
.apply(&rec);
inner.writes_since_check += 1;
}
if let Some(path) = &self.file {
if let Ok(line) = serde_json::to_string(&rec) {
@@ -245,6 +269,112 @@ impl Stats {
}
}
}
self.maybe_compact(false);
}
fn totals_path(&self) -> Option<String> {
self.file.as_ref().map(|f| format!("{f}.totals.json"))
}
fn load_folded(&self) {
let Some(path) = self.totals_path() else {
return;
};
let Ok(content) = std::fs::read_to_string(&path) else {
return;
};
if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
self.inner.lock().unwrap().folded = m;
}
}
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
/// older than the retention window fold into the cumulative per-account
/// sidecar and are dropped from the file (tmp + rename on both files, so
/// a crash mid-compaction loses nothing). `force` checks size regardless
/// of the write counter (startup).
fn maybe_compact(&self, force: bool) {
if self.max_file_bytes == 0 {
return;
}
let Some(path) = &self.file else { return };
{
let mut inner = self.inner.lock().unwrap();
if !force {
if inner.writes_since_check < 256 {
return;
}
inner.writes_since_check = 0;
}
}
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
if size <= self.max_file_bytes {
return;
}
let Ok(content) = std::fs::read_to_string(path) else {
return;
};
let cutoff = now_secs().saturating_sub(self.retention_secs);
let mut kept = String::with_capacity(content.len() / 2);
let mut inner = self.inner.lock().unwrap();
for line in content.lines() {
let line = line.trim();
if line.is_empty() {
continue;
}
match serde_json::from_str::<RequestRecord>(line) {
Ok(rec) if rec.ts < cutoff => {
inner
.folded
.entry(rec.account.clone())
.or_default()
.apply(&rec);
// fold = leave by_account_alltime (record leaves the file,
// its share moves from "alltime" into "folded")
if let Some(t) = inner.by_account_alltime.get_mut(&rec.account) {
// subtract by rebuilding is costly; instead track the
// invariant additively: remove the record's share
t.requests = t.requests.saturating_sub(1);
if rec.outcome == "ok" {
t.ok = t.ok.saturating_sub(1);
} else {
t.errors = t.errors.saturating_sub(1);
}
if rec.failover {
t.failovers = t.failovers.saturating_sub(1);
}
t.prompt_tokens = t.prompt_tokens.saturating_sub(rec.prompt_tokens as u64);
t.completion_tokens = t
.completion_tokens
.saturating_sub(rec.completion_tokens as u64);
t.cost_usd = (t.cost_usd - rec.cost_usd).max(0.0);
t.latency_ms_sum = t.latency_ms_sum.saturating_sub(rec.latency_ms);
}
}
Ok(_) => {
kept.push_str(line);
kept.push('\n');
}
Err(_) => { /* unparseable line — drop */ }
}
}
// persist the folded totals FIRST, then swap the trimmed file in
if let Some(tpath) = self.totals_path() {
if let Ok(json) = serde_json::to_string(&inner.folded) {
let tmp = format!("{tpath}.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &tpath);
}
}
}
let tmp = format!("{path}.tmp");
if std::fs::write(&tmp, kept).is_ok() {
let _ = std::fs::rename(&tmp, path);
}
tracing::info!(
new_size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0),
"stats log compacted (older records folded into cumulative totals)"
);
}
/// Snapshot for the admin API: aggregates + time series for `range_secs` + breakdown by `groupby`.
@@ -303,7 +433,27 @@ impl Stats {
.cmp(&a["requests"].as_u64().unwrap_or(0))
});
let mut alltime: HashMap<String, Totals> = inner.folded.clone();
for (k, t) in &inner.by_account_alltime {
alltime.entry(k.clone()).or_default().merge(t);
}
let alltime_by_account: serde_json::Map<String, serde_json::Value> = alltime
.iter()
.map(|(k, t)| {
(
k.clone(),
serde_json::json!({
"requests": t.requests,
"prompt_tokens": t.prompt_tokens,
"completion_tokens": t.completion_tokens,
"cost_usd": t.cost_usd,
}),
)
})
.collect();
serde_json::json!({
"alltime_by_account": alltime_by_account,
"totals": {
"requests": win.requests,
"ok": win.ok,
@@ -327,6 +477,9 @@ impl Stats {
/// Reset all in-memory stats and truncate the JSONL log — the "clear logs"
/// admin action.
pub fn clear(&self) {
if let Some(tpath) = self.totals_path() {
let _ = std::fs::remove_file(tpath);
}
{
let mut inner = self.inner.lock().unwrap();
*inner = Inner::default();
+37
View File
@@ -585,6 +585,43 @@ body.modal-open { overflow: hidden; }
/* Featured gallery: the list grows as new models are published, so give it more
room than the search grid before it starts scrolling. */
.featured-section .hf-grid { max-height: min(62vh, 760px); }
/* Money: 4 decimals, last two de-emphasized */
.money-tail { font-size: .72em; opacity: .75; margin-left: 1px; }
.seg { flex-wrap: wrap; }
.ok { color: var(--good); }
.pairs { display: flex; flex-direction: column; gap: 6px; align-items: flex-start; }
.pair-row { display: flex; gap: 6px; align-items: center; }
.pair-model { min-width: 220px; max-width: 340px; padding: 4px 8px; font-size: 12.5px; }
.pair-tier { max-width: 150px; padding: 4px 8px; font-size: 12.5px; }
/* Complexity bands slider */
.bands-wrap { margin: 4px 0 16px; }
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--panel-2, #23252c); margin: 26px 8px 10px; }
.bands-track.disabled { opacity: .45; pointer-events: none; }
.bands-zone { position: absolute; top: 0; bottom: 0; }
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
background: var(--fg, #fff); border: 3px solid var(--accent, #6c6cf5); cursor: grab; touch-action: none; z-index: 2; }
.bands-handle:active { cursor: grabbing; }
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
font-family: var(--mono, monospace); color: var(--fg); background: var(--panel, #1b1d23); padding: 1px 6px;
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
.bands-val.raised { top: -46px; }
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
/* Benchmark view */
@keyframes bm-in { from { opacity: 0; transform: translateY(-3px); } to { opacity: 1; transform: none; } }
tbody tr:last-child { animation: bm-in .25s ease; }
.bm-sum { margin-bottom: 10px; }
.bm-cell { cursor: pointer; white-space: nowrap; }
.bm-cell:hover { background: var(--hover); }
.badge.ok { background: rgba(52,211,153,.15); color: var(--good); }
/* Routing: per-task rule rows */
.taskrow { align-items: flex-start; }
.taskname { min-width: 190px; line-height: 1.35; }
+5 -1
View File
@@ -73,6 +73,7 @@ export const api = {
// protocols
getProtocols: () => j("GET", "/protocols"),
harnesses: () => j("GET", "/harnesses"),
putProtocols: (b) => j("PUT", "/protocols", b),
// settings
@@ -130,6 +131,7 @@ export const api = {
// system / logs / lifecycle
system: () => j("GET", "/system"),
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
clearLogs: () => j("DELETE", "/logs"),
restart: () => j("POST", "/restart"),
benchStart: (model) => j("POST", "/bench", { model }),
benchList: () => j("GET", "/bench"),
@@ -190,6 +192,7 @@ export async function testStream(body, onDelta) {
let ttftMs = null; // time to first content token
let usage = null; // OpenAI-style final-chunk usage, when the provider sends it
let usageEstimated = false; // gateway-injected estimate rather than tokenizer truth
let costUsd = null; // final cost from the gateway's closing event (headers carry 0)
for (;;) {
const { done, value } = await reader.read();
if (done) break;
@@ -207,6 +210,7 @@ export async function testStream(body, onDelta) {
usage = chunk.usage;
usageEstimated = !!chunk.cortiq_estimated;
}
if (chunk.cortiq_cost_usd != null) costUsd = chunk.cortiq_cost_usd;
const d = chunk.choices?.[0]?.delta?.content;
if (d) {
if (ttftMs == null) ttftMs = Math.round(performance.now() - t0);
@@ -224,7 +228,7 @@ export async function testStream(body, onDelta) {
score: parseFloat(h.get("x-cortiq-complexity-score") || "0"),
selected_model: h.get("x-cortiq-selected-model") || "",
route_source: h.get("x-cortiq-route-source") || "",
cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"),
cost_usd: costUsd != null ? costUsd : parseFloat(h.get("x-cortiq-cost-usd") || "0"),
usage,
usage_estimated: usageEstimated,
ttft_ms: ttftMs,
+20
View File
@@ -12,6 +12,7 @@ import { renderKeys } from "./views/keys.js";
import { renderPlayground } from "./views/playground.js";
import { renderSettings } from "./views/settings.js";
import { renderLogs } from "./views/logs.js";
import { renderBenchmark } from "./views/benchmark.js";
import { renderOnboarding } from "./views/onboarding.js";
const ROUTES = [
@@ -22,6 +23,7 @@ const ROUTES = [
{ id: "protocols", icon: "⇆", view: renderProtocols },
{ id: "keys", icon: "🔑", view: renderKeys },
{ id: "playground", icon: "✦", view: renderPlayground },
{ id: "benchmark", icon: "⏱", view: renderBenchmark },
{ id: "logs", icon: "📜", view: renderLogs },
{ id: "settings", icon: "⚙", view: renderSettings },
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
@@ -254,12 +256,30 @@ function showLogin() {
mount(document.getElementById("app"), renderLogin(boot));
}
let staleNotified = false;
async function checkUiFresh() {
// the SPA lives in the tab until reload; if the gateway restarted (dev
// rebuild, update, config restart) the embedded UI may have changed —
// tell the user instead of letting them stare at stale views
try {
const m = await api.meta();
if (appState.meta?.started && m.started && m.started !== appState.meta.started && !staleNotified) {
staleNotified = true;
const bar = h("div", { class: "callout warn", style: "position:fixed; left:50%; transform:translateX(-50%); bottom:18px; z-index:99; box-shadow:0 8px 24px rgba(0,0,0,.35)" },
h("span", {}, "🔄 " + t("shell.uiStale")), " ",
h("button", { class: "btn sm primary", onClick: () => location.reload() }, t("shell.reload")));
document.body.append(bar);
}
} catch {}
}
async function refreshHealth() {
try {
appState.health = await api.health();
} catch {
appState.health = null;
}
checkUiFresh();
// re-render topbar only if shell present
const route = currentRoute();
const main = document.querySelector(".main");
+5
View File
@@ -48,6 +48,11 @@ export function onLangChange(fn) {
return () => listeners.delete(fn);
}
// Human name for a complexity tier; custom tier names pass through as-is.
export function trTier(tier) {
return ["low", "medium", "high"].includes(tier) ? t("tier." + tier) : tier;
}
export function t(key, vars) {
const d = DICTS[lang] || en;
let s = d[key] ?? en[key] ?? key;
+71
View File
@@ -563,4 +563,75 @@ export default {
"routing.tasks.builtin": "Router nicht erreichbar — eingebauter Labelsatz",
"routing.tasks.addCustom": "Aufgabe hinzufügen",
"routing.tasks.addPlaceholder": "eigenes Label…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Qualitäts- und Tempo-Proben durch die Live-Pipeline: Antworten per Schlüsselwort geprüft, Schleifen erkannt, tok/s und Zeit bis zum ersten Token je Modell.",
"bm.models": "Zu testende Modelle",
"bm.quickMode": "Schnelltest",
"bm.quickHint": "Die schnelle Teilmenge mit kurzen Antworten — etwa eine Minute pro lokalem Modell.",
"bm.suiteInfo": "Eingebaute Suite: {q} schnelle / {f} volle Tests, plus Ihre eigenen.",
"bm.run": "Starten",
"bm.pickModel": "Wählen Sie mindestens ein Modell.",
"bm.testOf": "Test {i} von {n}",
"bm.results": "Ergebnisse",
"bm.quick": "schnell",
"bm.test": "Test",
"bm.cellHint": "Klick auf eine Zelle zeigt die vollständige Antwort.",
"bm.history": "Frühere Läufe (in diesem Browser gespeichert)",
"bm.custom": "Eigene Tests",
"bm.customHint": "Prompt + kommagetrennte Schlüsselwörter, die die Antwort enthalten muss (| für Synonyme). Im Browser gespeichert.",
"bm.promptPh": "Prompt…",
"bm.checkPh": "Schlüsselwörter, kommagetrennt",
"bm.addTest": "Test hinzufügen",
"bm.emptyAnswer": "leere Antwort",
"bm.loopDetected": "Schleife erkannt",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Smartes Routing",
"bm.mode.models": "Einzelne Modelle",
"bm.mode.routingDesc": "Das Kernfeature des Produkts im Test: jede Probe läuft über cortiq-auto — Sie sehen, welche Aufgabe der Router erkannt hat, welches Modell er wählte, ob das den Erwartungen entspricht und was es gekostet hat.",
"bm.mode.modelsDesc": "Dieselbe Suite, an gewählte Modelle gepinnt — Qualität, Tempo und Kosten im direkten Vergleich, am Router vorbei.",
"bm.answersOk": "Antworten korrekt",
"bm.classAcc": "Aufgaben-Klassifikation",
"bm.modelDist": "Anfragen gingen an",
"bm.expectedLbl": "erwartet:",
"bm.answer": "Antwort",
"bm.liveRunning": "läuft…",
"bm.spent": "ausgegeben",
"bm.verdict": "🏁 Fertig: Antworten {p}, ausgegeben {s}",
"bm.clearHistory": "Alle löschen",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — ins Terminal einfügen (das Gateway spricht das Anthropic-Protokoll nativ):",
"keys.h.codex": "Codex — hängt den Provider an ~/.codex/config.toml an und startet:",
"keys.h.opencode": "OpenCode — schreibt ~/.config/opencode/opencode.json (überschreibt!) und startet:",
"keys.h.continue": "Continue (VS Code / JetBrains) — hängt das Modell an ~/.continue/config.yaml an:",
"keys.h.cline1": "Cline (VS Code) — in den Erweiterungseinstellungen wählen:",
"keys.h.cline2": "Einstellungen → API Provider → OpenAI Compatible, dann die Werte oben einfügen.",
"keys.h.note": "cortiq-auto = smartes Routing: der Router wählt das Modell pro Anfrage. Für ein festes Modell den exakten Namen angeben. Unter Windows die Skripte in Git Bash / WSL ausführen oder dieselben Variablen über die Systemumgebung setzen.",
"keys.h.protoOff": "Das Anthropic-Protokoll (/v1/messages) ist deaktiviert — Claude Code kann sich nicht verbinden.",
"keys.h.enableProto": "Aktivieren",
"keys.h.st.installed": "Auf der Gateway-Maschine installiert",
"keys.h.st.configured": "Bereits für dieses Gateway konfiguriert",
"keys.h.st.notFound": "Auf der Gateway-Maschine nicht gefunden",
"bm.degraded": "Router degradiert",
"bm.routerDegraded": "Der Router hat nicht rechtzeitig geantwortet — die Anfrage nutzte die Fallback-Kette; die Klassifikation wird nicht gewertet.",
"bm.cutByCap": "Budget ging fürs Denken drauf — keine Antwort mehr",
"shell.uiStale": "Das Gateway wurde aktualisiert — diese Seite zeigt eine ältere Oberfläche.",
"shell.reload": "Neu laden",
"keys.col.requests": "Anfragen",
"keys.col.tokens": "Tokens ein/aus",
"keys.col.costAll": "$ gesamt",
"routing.tasks.tier": "Komplexität",
"routing.tasks.tierAuto": "Auto (Router)",
"routing.tasks.tierHint": "Welches Modell diese Aufgabe bei welcher Komplexität bedient; \"jede\" führt auf allen Stufen. Reihenfolge = Fallback-Reihenfolge. High eskaliert weiterhin über \"jede\"-Einträge.",
"routing.tasks.tierAny": "jede Komplexität",
"routing.tasks.addModel": "Modell",
"routing.bands.enable": "Eigene Komplexitätsgrenzen",
"routing.bands.hint": "Ziehen Sie die Griffe: wo low endet und high beginnt — die Griffe schieben einander, die Zahlen aktualisieren sich live. Der kontinuierliche Router-Score wird mit IHREN Grenzen neu eingeteilt; ohne Häkchen gelten die Grenzen des Routers.",
"tier.low": "leicht",
"tier.medium": "mittel",
"tier.high": "schwer",
};
+71
View File
@@ -556,4 +556,75 @@ export default {
"routing.tasks.builtin": "router unreachable — built-in label set",
"routing.tasks.addCustom": "Add task",
"routing.tasks.addPlaceholder": "custom label…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Quality and speed probes through the live pipeline: keyword-checked answers, anti-loop detection, tok/s and time-to-first-token per model.",
"bm.models": "Models to test",
"bm.quickMode": "Quick test",
"bm.quickHint": "The fast subset with short answers — about a minute per local model.",
"bm.suiteInfo": "Built-in suite: {q} quick / {f} full tests, plus your custom ones.",
"bm.run": "Run",
"bm.pickModel": "Pick at least one model.",
"bm.testOf": "test {i} of {n}",
"bm.results": "Results",
"bm.quick": "quick",
"bm.test": "Test",
"bm.cellHint": "Click a cell to see the model's full answer.",
"bm.history": "Previous runs (stored in this browser)",
"bm.custom": "Custom tests",
"bm.customHint": "Prompt + comma-separated keywords the answer must contain (use | for synonyms). Stored in this browser.",
"bm.promptPh": "prompt…",
"bm.checkPh": "keywords, comma-separated",
"bm.addTest": "Add test",
"bm.emptyAnswer": "empty answer",
"bm.loopDetected": "loop detected",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Smart routing",
"bm.mode.models": "Individual models",
"bm.mode.routingDesc": "The product's core feature under test: every probe goes through cortiq-auto — you see which task the router detected, which model it picked, whether that matches expectations, and what it cost.",
"bm.mode.modelsDesc": "The same suite pinned to selected models — compare their quality, speed and cost head-to-head, bypassing the router.",
"bm.answersOk": "answers correct",
"bm.classAcc": "task classification",
"bm.modelDist": "Requests went to",
"bm.expectedLbl": "expected:",
"bm.answer": "Answer",
"bm.liveRunning": "running…",
"bm.spent": "spent",
"bm.verdict": "🏁 Done: answers {p}, spent {s}",
"bm.clearHistory": "Clear all",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — paste into the terminal (the gateway speaks the Anthropic protocol natively):",
"keys.h.codex": "Codex — appends the provider to ~/.codex/config.toml and starts:",
"keys.h.opencode": "OpenCode — writes ~/.config/opencode/opencode.json (overwrites!) and starts:",
"keys.h.continue": "Continue (VS Code / JetBrains) — appends the model to ~/.continue/config.yaml:",
"keys.h.cline1": "Cline (VS Code) — in extension settings pick:",
"keys.h.cline2": "Settings → API Provider → OpenAI Compatible, then paste the values above.",
"keys.h.note": "cortiq-auto = smart routing: the router picks the model per request. Use an exact model name to pin one. On Windows run the scripts in Git Bash / WSL, or set the same variables via the system environment settings.",
"keys.h.protoOff": "The Anthropic protocol (/v1/messages) is disabled — Claude Code will not connect.",
"keys.h.enableProto": "Enable",
"keys.h.st.installed": "Installed on the gateway machine",
"keys.h.st.configured": "Already configured for this gateway",
"keys.h.st.notFound": "Not found on the gateway machine",
"bm.degraded": "router degraded",
"bm.routerDegraded": "The router did not answer in time — the request used the fallback chain; classification is not judged.",
"bm.cutByCap": "budget spent on reasoning — answer never came",
"shell.uiStale": "The gateway was updated — this page is running an older interface.",
"shell.reload": "Reload",
"keys.col.requests": "Requests",
"keys.col.tokens": "Tokens in/out",
"keys.col.costAll": "$ all-time",
"routing.tasks.tier": "Complexity",
"routing.tasks.tierAuto": "Auto (router)",
"routing.tasks.tierHint": "Which model serves this task at which complexity; \"any\" leads at every tier. Order = fallback order. High verdicts still escalate over any-tier pins.",
"routing.tasks.tierAny": "any complexity",
"routing.tasks.addModel": "model",
"routing.bands.enable": "My complexity boundaries",
"routing.bands.hint": "Drag the handles to set where low ends and where high begins — the handles push each other, the numbers update live. The router's continuous score is re-mapped with YOUR boundaries; unchecked = the router's own bands.",
"tier.low": "low",
"tier.medium": "medium",
"tier.high": "high",
};
+71
View File
@@ -563,4 +563,75 @@ export default {
"routing.tasks.builtin": "router inaccesible — conjunto integrado",
"routing.tasks.addCustom": "Añadir tarea",
"routing.tasks.addPlaceholder": "etiqueta propia…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Pruebas de calidad y velocidad a través del pipeline en vivo: respuestas verificadas por palabras clave, detección de bucles, tok/s y tiempo del primer token por modelo.",
"bm.models": "Modelos a probar",
"bm.quickMode": "Prueba rápida",
"bm.quickHint": "El subconjunto rápido con respuestas cortas — alrededor de un minuto por modelo local.",
"bm.suiteInfo": "Suite integrada: {q} pruebas rápidas / {f} completas, más las tuyas.",
"bm.run": "Ejecutar",
"bm.pickModel": "Elige al menos un modelo.",
"bm.testOf": "prueba {i} de {n}",
"bm.results": "Resultados",
"bm.quick": "rápida",
"bm.test": "Prueba",
"bm.cellHint": "Haz clic en una celda para ver la respuesta completa.",
"bm.history": "Ejecuciones anteriores (guardadas en este navegador)",
"bm.custom": "Pruebas propias",
"bm.customHint": "Prompt + palabras clave separadas por comas que la respuesta debe contener (| para sinónimos). Se guardan en este navegador.",
"bm.promptPh": "prompt…",
"bm.checkPh": "palabras clave, separadas por comas",
"bm.addTest": "Añadir prueba",
"bm.emptyAnswer": "respuesta vacía",
"bm.loopDetected": "bucle detectado",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Enrutado inteligente",
"bm.mode.models": "Modelos individuales",
"bm.mode.routingDesc": "La función clave del producto a prueba: cada sonda pasa por cortiq-auto — ves qué tarea detectó el router, qué modelo eligió, si coincide con lo esperado y cuánto costó.",
"bm.mode.modelsDesc": "La misma suite fijada a los modelos elegidos — compara su calidad, velocidad y coste directamente, sin pasar por el router.",
"bm.answersOk": "respuestas correctas",
"bm.classAcc": "clasificación de tareas",
"bm.modelDist": "Las peticiones fueron a",
"bm.expectedLbl": "esperado:",
"bm.answer": "Respuesta",
"bm.liveRunning": "en curso…",
"bm.spent": "gastado",
"bm.verdict": "🏁 Listo: respuestas {p}, gastado {s}",
"bm.clearHistory": "Borrar todo",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — pega en la terminal (la pasarela habla el protocolo Anthropic de forma nativa):",
"keys.h.codex": "Codex — añade el proveedor a ~/.codex/config.toml y arranca:",
"keys.h.opencode": "OpenCode — escribe ~/.config/opencode/opencode.json (¡sobrescribe!) y arranca:",
"keys.h.continue": "Continue (VS Code / JetBrains) — añade el modelo a ~/.continue/config.yaml:",
"keys.h.cline1": "Cline (VS Code) — en los ajustes de la extensión elige:",
"keys.h.cline2": "Ajustes → API Provider → OpenAI Compatible, luego pega los valores de arriba.",
"keys.h.note": "cortiq-auto = enrutado inteligente: el router elige el modelo en cada petición. Para fijar uno, indica su nombre exacto. En Windows ejecuta los scripts en Git Bash / WSL o define las mismas variables en el entorno del sistema.",
"keys.h.protoOff": "El protocolo Anthropic (/v1/messages) está desactivado — Claude Code no podrá conectarse.",
"keys.h.enableProto": "Activar",
"keys.h.st.installed": "Instalado en la máquina de la pasarela",
"keys.h.st.configured": "Ya configurado para esta pasarela",
"keys.h.st.notFound": "No encontrado en la máquina de la pasarela",
"bm.degraded": "router degradado",
"bm.routerDegraded": "El router no respondió a tiempo — la petición usó la cadena de reserva; la clasificación no se cuenta.",
"bm.cutByCap": "presupuesto gastado en razonar — sin respuesta final",
"shell.uiStale": "La pasarela se actualizó — esta página usa una interfaz antigua.",
"shell.reload": "Recargar",
"keys.col.requests": "Peticiones",
"keys.col.tokens": "Tokens ent/sal",
"keys.col.costAll": "$ total",
"routing.tasks.tier": "Complejidad",
"routing.tasks.tierAuto": "Auto (router)",
"routing.tasks.tierHint": "Qué modelo atiende esta tarea en qué complejidad; «cualquiera» lidera en todos los niveles. El orden = orden de respaldo. Un veredicto high sigue escalando sobre las entradas «cualquiera».",
"routing.tasks.tierAny": "cualquier complejidad",
"routing.tasks.addModel": "modelo",
"routing.bands.enable": "Mis límites de complejidad",
"routing.bands.hint": "Arrastra los tiradores: dónde termina low y dónde empieza high — se empujan entre sí y las cifras se actualizan en vivo. La puntuación continua del router se re-mapea con TUS límites; sin marcar = las bandas del propio router.",
"tier.low": "fácil",
"tier.medium": "medio",
"tier.high": "difícil",
};
+71
View File
@@ -563,4 +563,75 @@ export default {
"routing.tasks.builtin": "routeur injoignable — jeu d'étiquettes intégré",
"routing.tasks.addCustom": "Ajouter une tâche",
"routing.tasks.addPlaceholder": "étiquette personnalisée…",
// benchmark view
"nav.benchmark": "Benchmark",
"bm.title": "Benchmark",
"bm.subtitle": "Sondes de qualité et de vitesse via le pipeline en direct : réponses vérifiées par mots-clés, boucles détectées, tok/s et temps du premier token par modèle.",
"bm.models": "Modèles à tester",
"bm.quickMode": "Test rapide",
"bm.quickHint": "Le sous-ensemble rapide aux réponses courtes — environ une minute par modèle local.",
"bm.suiteInfo": "Suite intégrée : {q} tests rapides / {f} complets, plus les vôtres.",
"bm.run": "Lancer",
"bm.pickModel": "Choisissez au moins un modèle.",
"bm.testOf": "test {i} sur {n}",
"bm.results": "Résultats",
"bm.quick": "rapide",
"bm.test": "Test",
"bm.cellHint": "Cliquez sur une cellule pour voir la réponse complète.",
"bm.history": "Exécutions précédentes (stockées dans ce navigateur)",
"bm.custom": "Tests personnalisés",
"bm.customHint": "Prompt + mots-clés séparés par des virgules que la réponse doit contenir (| pour les synonymes). Stockés dans ce navigateur.",
"bm.promptPh": "prompt…",
"bm.checkPh": "mots-clés, séparés par des virgules",
"bm.addTest": "Ajouter un test",
"bm.emptyAnswer": "réponse vide",
"bm.loopDetected": "boucle détectée",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Routage intelligent",
"bm.mode.models": "Modèles individuels",
"bm.mode.routingDesc": "La fonctionnalité clé du produit à l'épreuve : chaque sonde passe par cortiq-auto — vous voyez quelle tâche le routeur a détectée, quel modèle il a choisi, si cela correspond aux attentes et combien cela a coûté.",
"bm.mode.modelsDesc": "La même suite épinglée aux modèles choisis — comparez leur qualité, vitesse et coût en direct, sans passer par le routeur.",
"bm.answersOk": "réponses correctes",
"bm.classAcc": "classification des tâches",
"bm.modelDist": "Requêtes envoyées vers",
"bm.expectedLbl": "attendu :",
"bm.answer": "Réponse",
"bm.liveRunning": "en cours…",
"bm.spent": "dépensé",
"bm.verdict": "🏁 Terminé : réponses {p}, dépensé {s}",
"bm.clearHistory": "Tout effacer",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — collez dans le terminal (la passerelle parle nativement le protocole Anthropic) :",
"keys.h.codex": "Codex — ajoute le fournisseur à ~/.codex/config.toml et démarre :",
"keys.h.opencode": "OpenCode — écrit ~/.config/opencode/opencode.json (écrase !) et démarre :",
"keys.h.continue": "Continue (VS Code / JetBrains) — ajoute le modèle à ~/.continue/config.yaml :",
"keys.h.cline1": "Cline (VS Code) — dans les réglages de l'extension, choisissez :",
"keys.h.cline2": "Réglages → API Provider → OpenAI Compatible, puis collez les valeurs ci-dessus.",
"keys.h.note": "cortiq-auto = routage intelligent : le routeur choisit le modèle à chaque requête. Pour en fixer un, indiquez son nom exact. Sous Windows, lancez les scripts dans Git Bash / WSL ou définissez les mêmes variables via l'environnement système.",
"keys.h.protoOff": "Le protocole Anthropic (/v1/messages) est désactivé — Claude Code ne pourra pas se connecter.",
"keys.h.enableProto": "Activer",
"keys.h.st.installed": "Installé sur la machine de la passerelle",
"keys.h.st.configured": "Déjà configuré pour cette passerelle",
"keys.h.st.notFound": "Introuvable sur la machine de la passerelle",
"bm.degraded": "routeur dégradé",
"bm.routerDegraded": "Le routeur n'a pas répondu à temps — la requête a suivi la chaîne de secours ; la classification n'est pas comptée.",
"bm.cutByCap": "budget épuisé en raisonnement — pas de réponse",
"shell.uiStale": "La passerelle a été mise à jour — cette page utilise une ancienne interface.",
"shell.reload": "Recharger",
"keys.col.requests": "Requêtes",
"keys.col.tokens": "Tokens in/out",
"keys.col.costAll": "$ au total",
"routing.tasks.tier": "Complexité",
"routing.tasks.tierAuto": "Auto (routeur)",
"routing.tasks.tierHint": "Quel modèle sert cette tâche à quelle complexité ; « toute » mène à tous les niveaux. L'ordre = ordre de repli. Un verdict high escalade toujours au-dessus des entrées « toute ».",
"routing.tasks.tierAny": "toute complexité",
"routing.tasks.addModel": "modèle",
"routing.bands.enable": "Mes limites de complexité",
"routing.bands.hint": "Faites glisser les poignées : où finit low et où commence high — elles se poussent l'une l'autre, les chiffres se mettent à jour en direct. Le score continu du routeur est re-découpé avec VOS limites ; décoché = les bandes du routeur.",
"tier.low": "facile",
"tier.medium": "moyen",
"tier.high": "difficile",
};
+71
View File
@@ -555,4 +555,75 @@ export default {
"routing.tasks.builtin": "роутер недоступен — встроенный набор меток",
"routing.tasks.addCustom": "Добавить задачу",
"routing.tasks.addPlaceholder": "своя метка…",
// benchmark view
"nav.benchmark": "Бенчмарк",
"bm.title": "Бенчмарк",
"bm.subtitle": "Проверки качества и скорости через живой пайплайн: ответы сверяются по ключевым словам, ловится зацикливание, замеряются tok/s и время первого токена.",
"bm.models": "Какие модели тестировать",
"bm.quickMode": "Быстрый тест",
"bm.quickHint": "Короткое подмножество с краткими ответами — около минуты на локальную модель.",
"bm.suiteInfo": "Встроенный набор: {q} быстрых / {f} полных тестов, плюс ваши собственные.",
"bm.run": "Запустить",
"bm.pickModel": "Выберите хотя бы одну модель.",
"bm.testOf": "тест {i} из {n}",
"bm.results": "Результаты",
"bm.quick": "быстрый",
"bm.test": "Тест",
"bm.cellHint": "Клик по ячейке — полный ответ модели.",
"bm.history": "Прошлые прогоны (хранятся в этом браузере)",
"bm.custom": "Свои тесты",
"bm.customHint": "Промпт + ключевые слова через запятую, которые должны быть в ответе (| — синонимы). Хранятся в этом браузере.",
"bm.promptPh": "промпт…",
"bm.checkPh": "ключевые слова через запятую",
"bm.addTest": "Добавить тест",
"bm.emptyAnswer": "пустой ответ",
"bm.loopDetected": "зацикливание",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Умный роутинг",
"bm.mode.models": "Отдельные модели",
"bm.mode.routingDesc": "Проверка главной фишки продукта: каждый тест идёт через cortiq-auto — видно, какую задачу распознал роутер, какую модель выбрал, совпало ли это с ожиданием и сколько это стоило.",
"bm.mode.modelsDesc": "Тот же набор тестов, но закреплённый за выбранными моделями — сравнение их качества, скорости и стоимости напрямую, мимо роутера.",
"bm.answersOk": "ответы верны",
"bm.classAcc": "классификация задач",
"bm.modelDist": "Запросы ушли в",
"bm.expectedLbl": "ожидалось:",
"bm.answer": "Ответ",
"bm.liveRunning": "идёт…",
"bm.spent": "потрачено",
"bm.verdict": "🏁 Готово: ответы {p}, потрачено {s}",
"bm.clearHistory": "Очистить всё",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — вставьте в терминал (шлюз говорит на протоколе Anthropic нативно):",
"keys.h.codex": "Codex — добавит провайдера в ~/.codex/config.toml и запустит:",
"keys.h.opencode": "OpenCode — запишет ~/.config/opencode/opencode.json (перезапишет!) и запустит:",
"keys.h.continue": "Continue (VS Code / JetBrains) — добавит модель в ~/.continue/config.yaml:",
"keys.h.cline1": "Cline (VS Code) — в настройках расширения укажите:",
"keys.h.cline2": "Настройки → API Provider → OpenAI Compatible, затем вставьте значения выше.",
"keys.h.note": "cortiq-auto — умный роутинг: роутер сам выбирает модель под каждый запрос. Чтобы закрепить конкретную, укажите её точное имя. На Windows запускайте скрипты в Git Bash / WSL или задайте те же переменные через настройки окружения системы.",
"keys.h.protoOff": "Протокол Anthropic (/v1/messages) выключен — Claude Code не подключится.",
"keys.h.enableProto": "Включить",
"keys.h.st.installed": "Установлен на машине шлюза",
"keys.h.st.configured": "Уже настроен на этот шлюз",
"keys.h.st.notFound": "Не найден на машине шлюза",
"bm.degraded": "роутер деградировал",
"bm.routerDegraded": "Роутер не успел ответить — запрос ушёл по резервной цепочке; классификация не засчитывается.",
"bm.cutByCap": "бюджет ушёл на размышления — ответ не успел",
"shell.uiStale": "Шлюз обновился — на этой странице старый интерфейс.",
"shell.reload": "Перезагрузить",
"keys.col.requests": "Запросы",
"keys.col.tokens": "Токены вх/исх",
"keys.col.costAll": "$ за всё время",
"routing.tasks.tier": "Сложность",
"routing.tasks.tierAuto": "Авто (по роутеру)",
"routing.tasks.tierHint": "Какая модель обслуживает задачу на какой сложности; «любая» ведёт на всех уровнях. Порядок = порядок фолбэка. Вердикт high по-прежнему эскалирует поверх «любых».",
"routing.tasks.tierAny": "любая сложность",
"routing.tasks.addModel": "модель",
"routing.bands.enable": "Мои границы сложности",
"routing.bands.hint": "Тяните ручки: где кончается low и где начинается high — ручки толкают друг друга, цифры обновляются на лету. Непрерывный score роутера пересчитывается ВАШИМИ границами; галочка снята — действуют пороги самого роутера.",
"tier.low": "лёгкий",
"tier.medium": "средний",
"tier.high": "сложный",
};
+71
View File
@@ -563,4 +563,75 @@ export default {
"routing.tasks.builtin": "yönlendiriciye ulaşılamıyor — yerleşik etiket seti",
"routing.tasks.addCustom": "Görev ekle",
"routing.tasks.addPlaceholder": "özel etiket…",
// benchmark view
"nav.benchmark": "Kıyaslama",
"bm.title": "Kıyaslama",
"bm.subtitle": "Canlı hat üzerinden kalite ve hız probları: yanıtlar anahtar kelimeyle doğrulanır, döngü tespit edilir, model başına tok/s ve ilk token süresi ölçülür.",
"bm.models": "Test edilecek modeller",
"bm.quickMode": "Hızlı test",
"bm.quickHint": "Kısa yanıtlı hızlı alt küme — yerel model başına yaklaşık bir dakika.",
"bm.suiteInfo": "Yerleşik küme: {q} hızlı / {f} tam test, artı kendi testleriniz.",
"bm.run": "Çalıştır",
"bm.pickModel": "En az bir model seçin.",
"bm.testOf": "{n} testten {i}.",
"bm.results": "Sonuçlar",
"bm.quick": "hızlı",
"bm.test": "Test",
"bm.cellHint": "Modelin tam yanıtı için hücreye tıklayın.",
"bm.history": "Önceki çalıştırmalar (bu tarayıcıda saklanır)",
"bm.custom": "Özel testler",
"bm.customHint": "İstem + yanıtın içermesi gereken virgülle ayrılmış anahtar kelimeler (| eşanlamlılar için). Bu tarayıcıda saklanır.",
"bm.promptPh": "istem…",
"bm.checkPh": "anahtar kelimeler, virgülle",
"bm.addTest": "Test ekle",
"bm.emptyAnswer": "boş yanıt",
"bm.loopDetected": "döngü tespit edildi",
// benchmark: routing mode, cost, live
"bm.mode.routing": "Akıllı yönlendirme",
"bm.mode.models": "Tek tek modeller",
"bm.mode.routingDesc": "Ürünün temel özelliği testte: her prob cortiq-auto üzerinden gider — yönlendiricinin hangi görevi algıladığını, hangi modeli seçtiğini, bunun beklentiyle örtüşüp örtüşmediğini ve maliyetini görürsünüz.",
"bm.mode.modelsDesc": "Aynı küme seçilen modellere sabitlenir — yönlendiriciyi atlayarak kalite, hız ve maliyeti doğrudan karşılaştırın.",
"bm.answersOk": "yanıtlar doğru",
"bm.classAcc": "görev sınıflandırması",
"bm.modelDist": "İstekler şuraya gitti",
"bm.expectedLbl": "beklenen:",
"bm.answer": "Yanıt",
"bm.liveRunning": "sürüyor…",
"bm.spent": "harcanan",
"bm.verdict": "🏁 Bitti: yanıtlar {p}, harcanan {s}",
"bm.clearHistory": "Tümünü temizle",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — terminale yapıştırın (ağ geçidi Anthropic protokolünü doğal olarak konuşur):",
"keys.h.codex": "Codex — sağlayıcıyı ~/.codex/config.toml dosyasına ekler ve başlatır:",
"keys.h.opencode": "OpenCode — ~/.config/opencode/opencode.json dosyasını yazar (üzerine yazar!) ve başlatır:",
"keys.h.continue": "Continue (VS Code / JetBrains) — modeli ~/.continue/config.yaml dosyasına ekler:",
"keys.h.cline1": "Cline (VS Code) — eklenti ayarlarında şunları seçin:",
"keys.h.cline2": "Ayarlar → API Provider → OpenAI Compatible, ardından yukarıdaki değerleri yapıştırın.",
"keys.h.note": "cortiq-auto = akıllı yönlendirme: yönlendirici her istek için modeli kendisi seçer. Belirli bir model için tam adını yazın. Windows'ta betikleri Git Bash / WSL içinde çalıştırın veya aynı değişkenleri sistem ortamından ayarlayın.",
"keys.h.protoOff": "Anthropic protokolü (/v1/messages) kapalı — Claude Code bağlanamaz.",
"keys.h.enableProto": "Etkinleştir",
"keys.h.st.installed": "Ağ geçidi makinesinde kurulu",
"keys.h.st.configured": "Bu ağ geçidi için zaten yapılandırılmış",
"keys.h.st.notFound": "Ağ geçidi makinesinde bulunamadı",
"bm.degraded": "yönlendirici yedeğe düştü",
"bm.routerDegraded": "Yönlendirici zamanında yanıt vermedi — istek yedek zincire gitti; sınıflandırma sayılmaz.",
"bm.cutByCap": "bütçe düşünmeye gitti — yanıt gelemedi",
"shell.uiStale": "Ağ geçidi güncellendi — bu sayfa eski arayüzü çalıştırıyor.",
"shell.reload": "Yeniden yükle",
"keys.col.requests": "İstekler",
"keys.col.tokens": "Token giriş/çıkış",
"keys.col.costAll": "$ toplam",
"routing.tasks.tier": "Karmaşıklık",
"routing.tasks.tierAuto": "Otomatik (yönlendirici)",
"routing.tasks.tierHint": "Bu görevi hangi karmaşıklıkta hangi model karşılar; \"her\" tüm katmanlarda önde gider. Sıra = yedekleme sırası. High kararı \"her\" girdilerinin üzerine yine tırmanır.",
"routing.tasks.tierAny": "her karmaşıklık",
"routing.tasks.addModel": "model",
"routing.bands.enable": "Kendi karmaşıklık sınırlarım",
"routing.bands.hint": "Tutamaçları sürükleyin: low nerede biter, high nerede başlar — tutamaçlar birbirini iter, sayılar anlık güncellenir. Yönlendiricinin sürekli puanı SİZİN sınırlarınızla yeniden bölümlenir; işaret kaldırılırsa yönlendiricinin kendi bantları geçerli.",
"tier.low": "kolay",
"tier.medium": "orta",
"tier.high": "zor",
};
+71
View File
@@ -563,4 +563,75 @@ export default {
"routing.tasks.builtin": "路由器不可达 — 使用内置标签集",
"routing.tasks.addCustom": "添加任务",
"routing.tasks.addPlaceholder": "自定义标签……",
// benchmark view
"nav.benchmark": "基准测试",
"bm.title": "基准测试",
"bm.subtitle": "通过实时管线做质量与速度探测:按关键词核对答案、检测循环输出,并按模型测量 tok/s 与首个 token 时间。",
"bm.models": "要测试的模型",
"bm.quickMode": "快速测试",
"bm.quickHint": "答案简短的快速子集 — 每个本地模型约一分钟。",
"bm.suiteInfo": "内置套件:{q} 个快速 / {f} 个完整测试,另加你自己的。",
"bm.run": "运行",
"bm.pickModel": "请至少选择一个模型。",
"bm.testOf": "第 {i}/{n} 个测试",
"bm.results": "结果",
"bm.quick": "快速",
"bm.test": "测试",
"bm.cellHint": "点击单元格查看模型的完整回答。",
"bm.history": "历史运行(保存在本浏览器)",
"bm.custom": "自定义测试",
"bm.customHint": "提示词 + 逗号分隔的关键词,回答必须包含它们(| 表示同义词)。保存在本浏览器。",
"bm.promptPh": "提示词……",
"bm.checkPh": "关键词,逗号分隔",
"bm.addTest": "添加测试",
"bm.emptyAnswer": "回答为空",
"bm.loopDetected": "检测到循环",
// benchmark: routing mode, cost, live
"bm.mode.routing": "智能路由",
"bm.mode.models": "单独模型",
"bm.mode.routingDesc": "测试产品核心能力:每个探测都经过 cortiq-auto — 可以看到路由器识别的任务、选中的模型、是否符合预期,以及花费了多少。",
"bm.mode.modelsDesc": "同一套测试固定在所选模型上 — 绕过路由器,直接对比它们的质量、速度与成本。",
"bm.answersOk": "回答正确",
"bm.classAcc": "任务分类",
"bm.modelDist": "请求流向",
"bm.expectedLbl": "预期:",
"bm.answer": "回答",
"bm.liveRunning": "进行中……",
"bm.spent": "花费",
"bm.verdict": "🏁 完成:回答 {p},花费 {s}",
"bm.clearHistory": "全部清除",
// harness auto-setup (API keys view)
"keys.h.claude": "Claude Code — 粘贴到终端(网关原生支持 Anthropic 协议):",
"keys.h.codex": "Codex — 将提供商追加到 ~/.codex/config.toml 并启动:",
"keys.h.opencode": "OpenCode — 写入 ~/.config/opencode/opencode.json(会覆盖!)并启动:",
"keys.h.continue": "Continue(VS Code / JetBrains)— 将模型追加到 ~/.continue/config.yaml:",
"keys.h.cline1": "Cline(VS Code)— 在扩展设置中选择:",
"keys.h.cline2": "设置 → API Provider → OpenAI Compatible,然后粘贴上面的值。",
"keys.h.note": "cortiq-auto = 智能路由:路由器为每个请求选择模型。要固定某个模型,请填写其精确名称。Windows 上请在 Git Bash / WSL 中运行脚本,或通过系统环境变量设置相同的值。",
"keys.h.protoOff": "Anthropic 协议(/v1/messages)已关闭 — Claude Code 无法连接。",
"keys.h.enableProto": "启用",
"keys.h.st.installed": "已安装在网关所在机器上",
"keys.h.st.configured": "已配置为使用此网关",
"keys.h.st.notFound": "网关所在机器上未找到",
"bm.degraded": "路由器降级",
"bm.routerDegraded": "路由器未及时响应 — 请求走了备用链路;分类不计入统计。",
"bm.cutByCap": "预算全用于思考——未能给出答案",
"shell.uiStale": "网关已更新 — 此页面运行的是旧界面。",
"shell.reload": "重新加载",
"keys.col.requests": "请求数",
"keys.col.tokens": "Token 入/出",
"keys.col.costAll": "$ 累计",
"routing.tasks.tier": "复杂度",
"routing.tasks.tierAuto": "自动(路由器)",
"routing.tasks.tierHint": "哪个模型在哪个复杂度上处理该任务;「任意」在所有层级领先。顺序即回退顺序。high 判定仍会越过「任意」条目升级。",
"routing.tasks.tierAny": "任意复杂度",
"routing.tasks.addModel": "模型",
"routing.bands.enable": "自定义复杂度边界",
"routing.bands.hint": "拖动手柄设定 low 到哪里结束、high 从哪里开始 — 手柄会相互推动,数字实时更新。路由器的连续得分将按你的边界重新划分;取消勾选则使用路由器自身的边界。",
"tier.low": "简单",
"tier.medium": "中等",
"tier.high": "困难",
};
+16 -2
View File
@@ -95,11 +95,25 @@ export function confirmDialog(message) {
}
// number formatting
// Money as a DOM node: 4 decimals, the last two rendered smaller — tiny
// per-request prices stay readable without shouting six zeros.
export function moneyEl(n) {
if (n == null) return document.createTextNode("—");
if (n === 0) return document.createTextNode("$0");
// sub-0.0001 costs (cheap flash models) would collapse to $0.0000 — grow
// precision until two significant digits survive
const s = Math.abs(n) >= 0.0001 ? n.toFixed(4) : n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
return h("span", { class: "money" }, "$" + s.slice(0, -2), h("small", { class: "money-tail" }, s.slice(-2)));
}
export function money(n) {
if (n == null) return "—";
if (n === 0) return "$0";
if (Math.abs(n) < 0.01) return "$" + n.toFixed(6);
return "$" + n.toFixed(4);
if (Math.abs(n) > 0 && Math.abs(n) < 0.0001) return "$" + n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
// up to 4 decimals, trailing zeros trimmed (but keep money-like 2 minimum)
let s = n.toFixed(4).replace(/0+$/, "").replace(/\.$/, "");
if (/\.\d$/.test(s)) s += "0";
return "$" + s;
}
export function ms(n) {
return n == null ? "—" : `${Math.round(Number(n))} ms`;
+525
View File
@@ -0,0 +1,525 @@
// Benchmark — two modes over the live pipeline:
// 🧭 Smart routing (default): the product's core promise under test — every
// probe goes through cortiq-auto and the router's task/tier decision and
// chosen model are judged alongside the answer itself.
// 🎯 Individual models: the same suite pinned to selected models — compare
// quality and speed head-to-head.
// A redesign of the old standalone benchmark.html: same scoring ideas
// (keyword checks with | synonyms, think-tag stripping, anti-loop detection,
// pass/partial/fail), now embedded, localized and CDN-free.
import { h, mount, toast, ms, money, moneyEl } from "../ui.js";
import { t, trTier } from "../i18n.js";
import { api, testStream } from "../api.js";
// ── built-in suite ──────────────────────────────────────────────────────────
// Checks are language-neutral where possible (digits, identifiers, emails);
// `check` entries may hold |-separated synonyms — one hit per entry counts.
// `quick: true` marks the fast subset.
// `label`: router task labels accepted as a correct classification (null = not
// judged — the prompt has no natural label in the taxonomy).
const SUITE = [
{ id: "math-mult", cat: "math", label: ["math"], quick: true, max: 96, prompt: "What is 17 * 23? Answer with the number only.", check: ["391"] },
{ id: "math-eq", cat: "math", label: ["math"], max: 128, prompt: "Solve 2x + 6 = 14. Answer with the value of x only.", check: ["4"] },
{ id: "logic-syll", cat: "logic", label: ["qa", "math"], max: 96, prompt: "If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? Answer yes or no.", check: ["yes|да|oui|ja|sí|si|evet|是"] },
{ id: "count-letters", cat: "logic", label: ["qa", "math"], quick: true, max: 64, prompt: "How many letters are in the word 'cat'? Answer with a digit.", check: ["3"] },
{ id: "apples", cat: "logic", label: ["math", "qa"], quick: true, max: 96, prompt: "John has 3 apples, buys 2 more, then eats 1. How many apples does he have? Answer with a digit.", check: ["4"] },
{ id: "code-fact", cat: "code", label: ["code"], max: 256, prompt: "Write a Python function factorial(n) that returns n!.", check: ["def", "factorial", "return"] },
{ id: "qa-capital", cat: "qa", label: ["qa"], quick: true, max: 48, prompt: "What is the capital of France? One word.", check: ["paris|париж"] },
{ id: "primes", cat: "qa", label: ["math", "qa"], max: 96, prompt: "List the first five prime numbers, comma-separated.", check: ["2", "3", "5", "7", "11"] },
{ id: "instr-ok", cat: "instruction", label: null, quick: true, max: 32, prompt: "Reply with exactly the word OK and nothing else.", check: ["ok"] },
{ id: "extract-email", cat: "extraction", label: ["extraction"], quick: true, max: 64, prompt: "Extract the email address from: 'Contact us at info@example.com today.' Reply with the address only.", check: ["info@example.com"] },
{ id: "translate", cat: "translation", label: ["translation"], max: 64, prompt: "Translate to English: «Доброе утро». Reply with the translation only.", check: ["good morning"] },
{ id: "format-json", cat: "instruction", label: null, max: 96, prompt: 'Return a JSON object with a single key "n" whose value is 7. JSON only.', check: ['"n"', "7"] },
{ id: "summarize", cat: "summarization", label: ["summarization"], max: 96, prompt: "Summarize in one short sentence: 'The cat sat on the warm mat by the window. After a while it curled up and fell asleep in the sun.'", check: ["cat|кот|кошк"] },
{ id: "creative", cat: "creative-writing", label: ["creative-writing"], max: 512, prompt: "Write a two-line poem about the sea.", check: ["sea|море|ocean"] },
{ id: "chitchat", cat: "chitchat", label: ["chitchat"], quick: true, max: 48, prompt: "Hi! How are you today?", check: [] },
// hard probes: dense enough that the router's complexity model rates them
// high tier — this is where tier-based routing actually earns its keep.
// Prompts are GENERATED fresh each run (domain + numbers rotate): the
// router deliberately scores familiar text as less novel, so any fixed
// wording would decay to medium/low over repeated runs.
// label: null — ambiguity between task labels is exactly what makes these high
{ id: "hard-science", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "science" },
{ id: "hard-ops", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "ops" },
];
// Fresh hard prompts per run. Two findings drive the design (measured against
// the live router): ambiguity carries the biggest weight in the complexity
// score, so INTERDISCIPLINARY prompts (science + engineering + planning) rate
// high reliably, while single-domain ones classify too confidently and drop
// to medium; and familiar text loses novelty on every repeat, so domains and
// numbers rotate to keep each run lexically fresh.
function hardPrompt(kind) {
const pick = (a) => a[(Math.random() * a.length) | 0];
const n = (lo, hi) => lo + ((Math.random() * (hi - lo)) | 0);
const science = [
() => `Architect a protein-folding inference pipeline for a wet lab: ensemble MSA generation with taxonomy-aware subsampling, template selection under a ${n(30, 60)} percent identity ceiling, distributed attention across ${n(4, 16)} GPUs with activation checkpointing, confidence calibration against experimental B-factors, and an active-learning loop that decides which of ${n(300, 2000)} candidate mutants to synthesize next week.`,
() => `Schedule observations for a network of ${n(8, 30)} robotic telescopes chasing gravitational-wave counterparts: probability skymaps updated every ${n(5, 20)} minutes, slew-time-aware target assignment with decaying rewards, cloud-cover forecasts folded in as stochastic availability, spectroscopy triggered only above a ${n(40, 85)} percent kilonova likelihood, and a plan to validate the scheduler against the one historical multi-messenger event.`,
() => `Run a closed-loop materials-discovery campaign for solid-state electrolytes: DFT screening of ${n(2000, 9000)} candidates, Bayesian optimization driving ${n(3, 12)} parallel synthesis robots, CNN-based XRD phase identification, an ionic-conductivity target of ${n(2, 25)} mS/cm, and an explicit decision policy for when simulation is trusted over synthesis.`,
() => `Design a surface-code error-correction pipeline for a ${n(400, 3000)}-qubit superconducting processor: choose a code distance for a logical error target of 1e-${n(9, 14)} at physical error rate 1e-3, syndrome extraction avoiding hook errors, a real-time decoder under a ${n(1, 4)} microsecond budget, and the classical co-processor layout with feedback for lattice surgery.`,
() => `Design a monitoring-reporting-verification pipeline for soil-carbon credits across ${n(500, 5000)} farms: satellite spectral indices calibrated against ${n(800, 4000)} physical core samples, hierarchical Bayesian estimation with shrinkage, additionality argued against county-level counterfactuals, uncertainty discounts priced into issuance, and an audit sampling plan catching over-reporting with ${n(90, 99)} percent power.`,
() => `Plan a cryo-EM structure-determination campaign for a ${n(300, 900)} kDa membrane complex: grid-screening strategy across ${n(4, 12)} detergent conditions, particle picking with a distilled transformer at ${n(2, 9)} million particles, ab-initio reconstruction ambiguity resolved by tilt-pair validation, local-resolution-aware model building, and a decision rule for when to stop collecting and publish.`,
];
const ops = [
() => `Model a dengue outbreak across ${n(20, 90)} districts by coupling an SEIR compartment model to vaccine cold-chain optimization: transmission parameters estimated from noisy case counts with a particle filter, uncertainty propagated into weekly demand forecasts, refrigerated routing under a ${n(120, 400)}-dose-per-crate constraint, and a defended ethical weighting for which ${n(3, 12)} districts get priority when supply covers ${n(20, 60)} percent of demand.`,
() => `Co-optimize a regional power grid's day-ahead market with a fleet of ${n(200, 2000)} grid-scale batteries: bid curves learned from ${n(2, 6)} years of price data, unit commitment as mixed-integer programming with ramp constraints, degradation amortized into marginal costs, N-1 contingency screening after every dispatch, and a mechanism-design argument that the bidding rules cannot be gamed by a coalition controlling ${n(10, 30)} percent of storage.`,
() => `Couple regional climate downscaling at ${n(2, 12)} km resolution to catastrophe-bond pricing: bias-correct ${n(10, 40)} CMIP ensemble members, generate stochastic hurricane tracks, propagate landfall distributions into a portfolio of ${n(30, 200)} bonds, price the attachment probabilities, and defend the discount rate applied to century-scale tail risk.`,
() => `Co-design a hospital emergency department's triage policy with its nurse staffing schedule: arrivals as a time-varying Poisson process fitted to ${n(2, 6)} years of data, acuity drift as a hidden Markov chain, staffing as integer programming over ${n(8, 12)}-hour shifts, a queueing bound on door-to-doctor time at the ${n(90, 99)}th percentile, and a fairness constraint preventing systematic delays for low-acuity elderly patients.`,
() => `Optimize a container port's berth allocation jointly with tidal windows and rail departures: vessel ETAs with heavy-tailed uncertainty, berth-crane assignment as a mixed-integer program re-solved hourly, tide-constrained draft limits for the ${n(3, 8)} deepest berths, a queueing model of the truck gate coupled to city traffic, and a robustness argument for surviving a ${n(24, 72)}-hour crane outage.`,
() => `Schedule observations for a fleet of ${n(6, 40)} agricultural drones across ${n(200, 2000)} fields: multispectral revisit deadlines driven by crop growth-stage models, battery-swap logistics as a vehicle-routing problem with time windows, rain forecasts as stochastic field availability, disease-outbreak alerts pre-empting the plan, and a proof the replanner never starves a field past its agronomic deadline.`,
];
return pick(kind === "science" ? science : ops)();
}
const LS_CUSTOM = "allaigate_bench_custom";
const LS_HISTORY = "allaigate_bench_history";
function customTests() {
try {
return JSON.parse(localStorage.getItem(LS_CUSTOM) || "[]");
} catch {
return [];
}
}
function saveCustom(list) {
localStorage.setItem(LS_CUSTOM, JSON.stringify(list));
}
function history() {
try {
return JSON.parse(localStorage.getItem(LS_HISTORY) || "[]");
} catch {
return [];
}
}
function pushHistory(run) {
const l = history();
l.unshift(run);
localStorage.setItem(LS_HISTORY, JSON.stringify(l.slice(0, 12)));
}
function saveHistory(l) {
localStorage.setItem(LS_HISTORY, JSON.stringify(l));
}
// ── evaluation (ported from the old benchmark, trimmed) ─────────────────────
function stripThink(text) {
return text.replace(/<(?:\|)?(?:think|thought)(?:\|)?>[\s\S]*?(?:<\/(?:\|)?(?:think|thought)(?:\|)?>|$)/gi, "").trim();
}
// tail loop: the same 12+ char chunk repeated 3+ times back to back
function looksLooped(text) {
const tail = text.slice(-360);
for (let len = 12; len <= 60; len += 8) {
const probe = tail.slice(-len);
if (probe && tail.split(probe).length - 1 >= 3) return true;
}
return false;
}
function evaluate(raw, test) {
let text = stripThink(raw);
if (!text && raw) text = raw; // max_tokens cut inside <think> — judge the raw
const resp = text.toLowerCase();
if (!resp.trim()) return { status: "fail", score: 0, note: t("bm.emptyAnswer") };
if (looksLooped(resp)) return { status: "fail", score: 0, note: t("bm.loopDetected") };
if (!test.check.length) return { status: "pass", score: 1, note: "✓" };
let hit = 0;
for (const cond of test.check) {
const opts = cond.toLowerCase().split("|").map((s) => s.trim());
if (opts.some((o) => o && resp.includes(o))) hit++;
}
const frac = hit / test.check.length;
const status = frac >= 1 ? "pass" : frac >= 0.5 ? "partial" : "fail";
return { status, score: frac, note: `${hit}/${test.check.length}` };
}
export async function renderBenchmark() {
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
const root = h("div");
let running = false;
let stopFlag = false;
let mode = "routing"; // routing (cortiq-auto, the product's feature) | models
const modelCbs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox", checked: ids.length === 1 ? true : null }) }));
const quickCb = h("input", { type: "checkbox", checked: true });
const resultsHost = h("div", {});
const progressHost = h("div", {});
const historyHost = h("div", {});
const modeDesc = h("p", { class: "muted", style: "margin:8px 0 0" }, t("bm.mode.routingDesc"));
const modelsPick = h("div", { class: "flex wrap", style: "gap:10px; margin-top:12px; display:none" },
...modelCbs.map(({ id, cb }) => h("label", { class: "check" }, cb, h("span", { class: "mono" }, id))));
const modeSeg = h("div", { class: "seg" },
h("button", { class: "active" }, "🧭 " + t("bm.mode.routing")),
h("button", {}, "🎯 " + t("bm.mode.models")));
modeSeg.querySelectorAll("button").forEach((b, i) =>
b.addEventListener("click", () => {
mode = i === 0 ? "routing" : "models";
modeSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === i));
modelsPick.style.display = mode === "models" ? "" : "none";
modeDesc.textContent = t(mode === "routing" ? "bm.mode.routingDesc" : "bm.mode.modelsDesc");
}));
function suite() {
const base = SUITE.concat(customTests());
return quickCb.checked ? base.filter((x) => x.quick || x.custom) : base;
}
const runBtn = h("button", { class: "btn primary" }, "▶ " + t("bm.run"));
const stopBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
stopBtn.addEventListener("click", () => {
stopFlag = true;
stopBtn.disabled = true;
});
runBtn.addEventListener("click", async () => {
const models = mode === "routing" ? ["cortiq-auto"] : modelCbs.filter((m) => m.cb.checked).map((m) => m.id);
if (!models.length) {
toast(t("bm.pickModel"), "warn");
return;
}
running = true;
stopFlag = false;
runBtn.disabled = true;
stopBtn.style.display = "";
stopBtn.disabled = false;
const tests = suite().filter((x) => mode === "routing" || x.judge !== "route");
const run = { ts: Date.now(), mode, quick: quickCb.checked, models: {}, nTests: tests.length };
for (const model of models) {
const rec = { results: [], score: 0, tokS: [], ttft: [], failed: false };
run.models[model] = rec;
for (let i = 0; i < tests.length; i++) {
if (stopFlag) break;
const test = tests[i];
const prompt = test.gen ? hardPrompt(test.gen) : test.prompt;
mount(
progressHost,
h("div", { class: "card" },
h("div", { class: "flex" },
h("span", { class: "spinner" }), " ",
h("b", { class: "mono" }, mode === "routing" ? t("bm.mode.routing") : model), " · ",
t("bm.testOf", { i: i + 1, n: tests.length }), " · ",
h("span", { class: "muted" }, test.id)),
h("div", { class: "muted small", style: "margin-top:6px; font-style:italic" }, "«" + prompt.slice(0, 110) + (prompt.length > 110 ? "…»" : "»")),
h("div", { class: "pbar", style: "margin-top:8px" },
h("div", { class: "pbar-fill det", style: `width:${Math.round((i / tests.length) * 100)}%` })))
);
const started = performance.now();
let text = "";
try {
const info = await testStream(
// +256: headroom for reasoning models — a tight cap cuts them
// mid-<think> and turns a correct model into a false ❌
{ model, messages: [{ role: "user", content: prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
(d) => (text += d)
);
const latency = performance.now() - started;
let ev;
if (test.judge === "route") {
// the test asserts the ROUTER's decision: did it rate this high?
// Two honest retries. (1) The SAME prompt after a pause: the
// router judges difficulty in the background — the first-ever
// occurrence routes by prior, the judgement lands in the cache a
// couple of seconds later, so the second occurrence is the real
// steady-state answer. (2) A fresh template, in case this one's
// wording clustered into a burnt region.
if (info.tier !== "high" && test.gen) {
const attempts = [
{ prompt, delayMs: 4000 },
{ prompt: hardPrompt(test.gen), delayMs: 0 },
];
for (const a of attempts) {
if (info.tier === "high") break;
if (a.delayMs) await new Promise((r) => setTimeout(r, a.delayMs));
try {
const again = await testStream(
{ model, messages: [{ role: "user", content: a.prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
() => {}
);
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
if (again.tier === "high") {
info.tier = again.tier;
info.selected_model = again.selected_model;
info.task_label = again.task_label;
info.route_source = again.route_source;
}
} catch { /* keep previous attempt */ }
}
}
ev = info.tier === "high"
? { status: "pass", score: 1, note: "high ✓" }
: { status: "fail", score: 0, note: (info.tier || "—") + " ≠ high" };
} else {
ev = evaluate(text, test);
}
const compl = info.usage?.completion_tokens;
if (test.judge !== "route" && ev.status === "fail" && !stripThink(text).trim() && compl >= (test.max || 128) + 250) {
ev.note = t("bm.cutByCap"); // the whole budget went into <think>
}
if (compl && latency > 0) {
const gen = info.ttft_ms != null ? Math.max(1, latency - info.ttft_ms) : latency;
rec.tokS.push(compl / (gen / 1000));
}
if (info.ttft_ms != null) rec.ttft.push(info.ttft_ms);
const row = { id: test.id, cat: test.cat, ...ev, latency: Math.round(latency), cost: info.cost_usd || 0, answer: stripThink(text).slice(0, 300) };
if (mode === "routing") {
// router timed out / fell back — there was no classification to
// judge, so it must not count against accuracy
row.degraded = !info.task_label
|| info.task_label === "degraded"
|| String(info.route_source || "").includes("degraded");
row.task = info.task_label || "—";
row.tier = info.tier || "—";
row.chosen = info.selected_model || "—";
row.source = info.route_source || "—";
row.expected = test.label; // null = classification not judged
row.taskOk = test.label && !row.degraded ? test.label.includes(info.task_label) : null;
}
rec.results.push(row);
} catch (e) {
rec.results.push({ id: test.id, cat: test.cat, status: "fail", score: 0, note: String(e.message || e).slice(0, 80), latency: 0, answer: "" });
rec.failed = true;
}
renderResults(run, true); // live: строка появляется сразу после теста
}
if (stopFlag) break;
}
mount(progressHost);
running = false;
runBtn.disabled = false;
stopBtn.style.display = "none";
if (!stopFlag) {
pushHistory(run);
// финальный аккорд: короткий вердикт с процентами и деньгами
const first = run.models[Object.keys(run.models)[0]];
if (first) {
const st = stats(first);
const judged = first.results.filter((r) => r.taskOk != null);
const cls = judged.length ? ` · ${judged.filter((r) => r.taskOk).length}/${judged.length}` : "";
const total = Object.values(run.models).reduce((a, rec) => a + stats(rec).cost, 0);
toast(t("bm.verdict", { p: Math.round(st.score * 100) + "%" + cls, s: money(total) }), "ok", 6000);
}
}
renderResults(run);
renderHistory();
});
function statusIcon(s) {
return s === "pass" ? "✅" : s === "partial" ? "◐" : "❌";
}
function stats(rec) {
const n = rec.results.length || 1;
return {
score: rec.results.reduce((a, r) => a + r.score, 0) / n,
tokS: rec.tokS.length ? rec.tokS.reduce((a, b) => a + b, 0) / rec.tokS.length : null,
ttft: rec.ttft.length ? rec.ttft.reduce((a, b) => a + b, 0) / rec.ttft.length : null,
cost: rec.results.reduce((a, r) => a + (r.cost || 0), 0),
};
}
function renderResults(run, live) {
const models = Object.keys(run.models);
if (!models.length) {
mount(resultsHost);
return;
}
const routing = run.mode === "routing";
const detail = h("div", {});
let summary;
if (routing) {
const rec = run.models[models[0]];
const st = stats(rec);
const judged = rec.results.filter((r) => r.taskOk != null);
const okCount = judged.filter((r) => r.taskOk).length;
// какие модели реально выбирал роутер
const dist = {};
for (const r of rec.results) {
if (!r.chosen) continue;
dist[r.chosen] = dist[r.chosen] || { n: 0, cost: 0 };
dist[r.chosen].n++;
dist[r.chosen].cost += r.cost || 0;
}
const pct = Math.round(st.score * 100);
summary = h("div", {},
h("div", { class: "kpis", style: "grid-template-columns: repeat(auto-fit, minmax(150px, 1fr))" },
h("div", { class: "kpi" }, h("div", { class: "big" }, pct + "%"), h("div", { class: "lbl" }, t("bm.answersOk"))),
h("div", { class: "kpi" }, h("div", { class: "big" }, judged.length ? `${okCount}/${judged.length}` : "—"), h("div", { class: "lbl" }, t("bm.classAcc"))),
h("div", { class: "kpi" }, h("div", { class: "big" }, st.ttft != null ? ms(Math.round(st.ttft)) : "—"), h("div", { class: "lbl" }, t("pg.ttft"))),
h("div", { class: "kpi" }, h("div", { class: "big" }, st.tokS != null ? st.tokS.toFixed(1) : "—"), h("div", { class: "lbl" }, "tok/s")),
h("div", { class: "kpi" }, h("div", { class: "big" }, moneyEl(st.cost)), h("div", { class: "lbl" }, t("bm.spent")))),
h("div", { class: "flex wrap", style: "gap:6px; margin-top:4px" },
h("span", { class: "muted small" }, t("bm.modelDist") + ":"),
...Object.entries(dist).map(([m, d]) =>
h("span", { class: "badge" }, `${m} × ${d.n}`, d.cost > 0 ? " · " : "", d.cost > 0 ? moneyEl(d.cost) : ""))));
} else {
const sorted = models.slice().sort((a, b) => stats(run.models[b]).score - stats(run.models[a]).score);
summary = h("div", {},
...sorted.map((m) => {
const st = stats(run.models[m]);
const pct = Math.round(st.score * 100);
return h("div", { class: "bm-sum" },
h("div", { class: "flex" },
h("b", { class: "mono grow" }, m),
h("span", { class: "badge " + (pct >= 80 ? "ok" : pct >= 50 ? "" : "bad") }, pct + "%"),
st.tokS != null ? h("span", { class: "muted small", style: "margin-left:8px" }, st.tokS.toFixed(1) + " tok/s") : null,
st.ttft != null ? h("span", { class: "muted small", style: "margin-left:8px" }, "TTFT " + ms(Math.round(st.ttft))) : null,
st.cost > 0 ? h("span", { class: "muted small", style: "margin-left:8px" }, t("bm.spent") + " ", moneyEl(st.cost)) : null),
h("div", { class: "sysbar" }, h("i", { class: pct < 50 ? "bad" : "", style: `width:${pct}%` })));
}));
}
function answerCell(r, m) {
const cell = h("td", { class: "bm-cell", title: r.answer || r.note },
statusIcon(r.status) + " ",
h("span", { class: "muted small" }, r.note + (r.latency ? ` · ${(r.latency / 1000).toFixed(1)}s` : "")));
cell.addEventListener("click", () => {
mount(detail, h("div", { class: "card" },
h("div", { class: "card-title" }, `${r.id} · ${m} · ${statusIcon(r.status)}`),
h("pre", { class: "conv-log", style: "max-height:200px" }, r.answer || "—")));
});
return cell;
}
let header, rows;
if (routing) {
const rec = run.models[models[0]];
header = h("tr", {},
h("th", {}, t("bm.test")),
h("th", {}, t("pg.task")),
h("th", {}, t("pg.tier")),
h("th", {}, t("pg.model")),
h("th", {}, t("bm.answer")));
rows = rec.results.map((r) =>
h("tr", {},
h("td", {}, h("div", {}, r.id), h("div", { class: "muted small" }, r.cat)),
h("td", {},
r.degraded
? h("span", { class: "muted", title: t("bm.routerDegraded") }, "⚠ " + t("bm.degraded"))
: r.taskOk == null
? h("span", { class: "muted" }, r.task || "—")
: h("span", { class: r.taskOk ? "" : "err" },
(r.taskOk ? "✓ " : "✗ ") + r.task,
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—"),
h("td", {},
h("span", { class: "mono small" }, r.chosen || "—"),
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
r.source && r.source.includes("task-rule") ? h("div", { class: "badge", style: "margin-top:2px" }, "task-rule") : null),
answerCell(r, r.chosen || "cortiq-auto")));
} else {
header = h("tr", {}, h("th", {}, t("bm.test")), ...models.map((m) => h("th", { class: "mono" }, m)));
rows = (run.models[models[0]].results || []).map((_, i) =>
h("tr", {},
h("td", {},
h("div", {}, run.models[models[0]].results[i].id),
h("div", { class: "muted small" }, run.models[models[0]].results[i].cat)),
...models.map((m) => {
const r = run.models[m].results[i];
return r ? answerCell(r, m) : h("td", {}, "—");
})));
}
mount(resultsHost,
h("div", { class: "card" },
h("div", { class: "card-head" },
h("h3", {}, (routing ? "🧭 " : "🎯 ") + t("bm.results")),
h("span", { class: "card-head sub" },
(live ? t("bm.liveRunning") + " · " : "") + new Date(run.ts).toLocaleString() + (run.quick ? " · " + t("bm.quick") : ""))),
summary,
h("div", { class: "table-wrap", style: "margin-top:14px" }, h("table", {}, h("thead", {}, header), h("tbody", {}, ...rows))),
h("div", { class: "muted small", style: "margin-top:6px" }, t("bm.cellHint"))),
detail);
}
function renderHistory() {
const l = history();
if (!l.length) {
mount(historyHost);
return;
}
const clearAll = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("bm.clearHistory"));
clearAll.addEventListener("click", () => {
saveHistory([]);
renderHistory();
});
mount(historyHost, h("div", { class: "card" },
h("div", { class: "card-head" },
h("h3", {}, t("bm.history")),
h("div", { class: "right" }, clearAll)),
...l.map((run, idx) => {
const models = Object.keys(run.models);
const scoreOf = (m) => {
const rs = run.models[m].results || [];
return rs.length ? rs.reduce((a, r) => a + r.score, 0) / rs.length : 0;
};
const best = models.slice().sort((a, b) => scoreOf(b) - scoreOf(a))[0];
const row = h("div", { class: "flex bench-row", style: "cursor:pointer" },
h("span", {}, run.mode === "routing" ? "🧭" : "🎯"),
h("span", { class: "muted small" }, new Date(run.ts).toLocaleString()),
h("span", { class: "grow mono small" }, run.mode === "routing" ? t("bm.mode.routing") : models.join(", ")),
run.quick ? h("span", { class: "badge" }, t("bm.quick")) : null,
best ? h("b", {}, Math.round(scoreOf(best) * 100) + "%") : null,
h("button", { class: "btn sm ghost danger", title: t("common.delete"), onClick: (e) => {
e.stopPropagation();
l.splice(idx, 1);
saveHistory(l);
renderHistory();
} }, "×"));
row.addEventListener("click", () => renderResults(run));
return row;
})));
}
// ── custom tests (stored in the browser) ──
function customCard() {
const host = h("div", { class: "card" });
function paint() {
const list = customTests();
const promptIn = h("input", { placeholder: t("bm.promptPh") });
const checkIn = h("input", { placeholder: t("bm.checkPh") });
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("bm.addTest"));
addBtn.addEventListener("click", () => {
const prompt = promptIn.value.trim();
const check = checkIn.value.split(",").map((s) => s.trim()).filter(Boolean);
if (!prompt || !check.length) return;
list.push({ id: "custom-" + (list.length + 1), cat: "custom", custom: true, max: 128, prompt, check });
saveCustom(list);
paint();
});
mount(host,
h("div", { class: "card-title" }, t("bm.custom")),
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bm.customHint")),
...list.map((tst, i) =>
h("div", { class: "flex bench-row" },
h("span", { class: "grow small" }, tst.prompt.slice(0, 90)),
h("span", { class: "muted small mono" }, tst.check.join(", ").slice(0, 40)),
h("button", { class: "btn sm ghost danger", onClick: () => { list.splice(i, 1); saveCustom(list); paint(); } }, "×"))),
h("div", { class: "flex", style: "gap:8px; margin-top:8px" }, promptIn, checkIn, addBtn));
}
paint();
return host;
}
mount(root,
h("div", { class: "page-head" },
h("h2", {}, t("bm.title")),
h("p", {}, t("bm.subtitle"))),
ids.length === 0 ? h("div", { class: "callout" }, t("routing.empty")) : null,
h("div", { class: "card" },
modeSeg,
modeDesc,
modelsPick,
h("div", { class: "flex", style: "justify-content:space-between; margin-top:14px" },
h("label", { class: "check", style: "margin:0", title: t("bm.quickHint") }, quickCb, "⚡ " + t("bm.quickMode")),
h("div", { class: "flex", style: "gap:8px" }, stopBtn, runBtn)),
h("div", { class: "muted small", style: "margin-top:6px" },
t("bm.suiteInfo", { q: SUITE.filter((x) => x.quick).length, f: SUITE.length }))),
progressHost,
resultsHost,
customCard(),
historyHost);
renderHistory();
return root;
}
+2 -2
View File
@@ -1,5 +1,5 @@
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
import { h, mount, money, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
@@ -161,7 +161,7 @@ export async function renderDashboard() {
{ class: "kpis" },
kpi(num(tot.requests), t("dash.kpi.requests")),
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
kpi(money(tot.cost_usd), t("dash.kpi.cost")),
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
kpi(pct(tot.success_rate), t("dash.kpi.success")),
kpi(num(tot.failovers), t("dash.kpi.failovers"))
+138 -19
View File
@@ -1,30 +1,21 @@
// API keys — list, create (with copy-once), revoke.
import { h, mount, modal, toast, confirmDialog } from "../ui.js";
import { h, mount, modal, toast, confirmDialog, moneyEl, num } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
export async function renderKeys() {
const modelsData = await api.listModels();
const ids = (modelsData.models || []).map((m) => m.id);
const root = h("div");
let usageRange = "7d";
function openCreate(reload) {
const accountIn = h("input", { placeholder: "team-a" });
const rateIn = h("input", { type: "number", value: 600 });
const customIn = h("input", { placeholder: "sk-gw-…" });
const allowInputs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox" }) }));
const body = h(
"div",
{},
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
h(
"label",
{ class: "field" },
h("span", {}, t("keys.form.allow")),
h("div", { class: "flex wrap" }, ...allowInputs.map(({ id, cb }) => h("label", { class: "check" }, cb, id))),
h("div", { class: "hint" }, t("keys.form.allowHint"))
),
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
);
modal(t("keys.form.title"), body, async () => {
@@ -33,7 +24,7 @@ export async function renderKeys() {
key: customIn.value.trim(),
account: accountIn.value.trim() || "default",
rate_per_min: parseInt(rateIn.value) || 0,
allow_models: allowInputs.filter(({ cb }) => cb.checked).map(({ id }) => id),
allow_models: [],
});
reload();
showCreated(r.key);
@@ -58,9 +49,64 @@ export async function renderKeys() {
// Keys are stored hashed-masked in the list, so the key itself is a
// placeholder unless one was just created on this page.
function snippetsCard(openMode) {
const base = location.origin + "/v1";
const origin = location.origin;
const base = origin + "/v1";
const key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
const snips = {
"Claude Code": `# ${t("keys.h.claude")}
export ANTHROPIC_BASE_URL="${origin}"
export ANTHROPIC_AUTH_TOKEN="${key}"
export ANTHROPIC_MODEL="cortiq-auto"
claude`,
Codex: `# ${t("keys.h.codex")}
mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
model = "cortiq-auto"
model_provider = "cortiq"
[model_providers.cortiq]
name = "Cortiq Gateway"
base_url = "${base}"
env_key = "CORTIQ_API_KEY"
wire_api = "chat"
EOF
export CORTIQ_API_KEY="${key}"
codex`,
OpenCode: `# ${t("keys.h.opencode")}
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"cortiq": {
"npm": "@ai-sdk/openai-compatible",
"name": "Cortiq Gateway",
"options": { "baseURL": "${base}", "apiKey": "${key}" },
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
}
}
}
EOF
opencode`,
Continue: `# ${t("keys.h.continue")}
mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
models:
- name: Cortiq Auto
provider: openai
model: cortiq-auto
apiBase: ${base}
apiKey: ${key}
EOF`,
Cline: `${t("keys.h.cline1")}
API Provider: OpenAI Compatible
Base URL: ${base}
API Key: ${key}
Model ID: cortiq-auto
${t("keys.h.cline2")}`,
curl: `curl ${base}/chat/completions \\
-H "Authorization: Bearer ${key}" \\
-H "Content-Type: application/json" \\
@@ -82,20 +128,65 @@ const resp = await client.chat.completions.create({
});
console.log(resp.choices[0].message.content);`,
};
let cur = "curl";
let cur = "Claude Code";
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
const seg = h(
"div",
{ class: "seg" },
...Object.keys(snips).map((k, i) =>
h("button", { class: i === 0 ? "active" : "", onClick: (e) => {
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
cur = k;
pre.textContent = snips[k];
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
paintStatus();
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
e.target.classList.add("active");
e.currentTarget.classList.add("active");
} }, k)
)
);
// Claude Code needs the anthropic_messages protocol — offer a one-click
// enable when it is off, right where the user will hit the problem.
const protoWarn = h("div", { style: "display:none; margin-top:8px" });
api.getProtocols().then((prot) => {
if (prot.anthropic_messages) return;
const fixBtn = h("button", { class: "btn sm primary" }, t("keys.h.enableProto"));
fixBtn.addEventListener("click", async () => {
try {
await api.putProtocols({ ...prot, anthropic_messages: true });
protoWarn.style.display = "none";
toast(t("common.saved"), "ok");
} catch (e) {
toast(String(e.message || e), "err");
}
});
mount(protoWarn, h("div", { class: "callout warn" }, h("span", {}, "⚠ " + t("keys.h.protoOff")), " ", fixBtn));
if (cur === "Claude Code") protoWarn.style.display = "";
}).catch(() => {});
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
// status line under the snippet ("installed" / "already configured").
let detected = {};
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
function paintStatus() {
const d = detected[cur];
if (!d) {
statusHost.textContent = "";
return;
}
statusHost.innerHTML = "";
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
else mount(statusHost, h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")));
}
api.harnesses().then((d) => {
detected = d || {};
seg.querySelectorAll("button").forEach((b) => {
const st = detected[b.dataset.snip];
if (st && st.installed) b.append(" ✓");
});
paintStatus();
}).catch(() => {});
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
copy.addEventListener("click", async () => {
await navigator.clipboard.writeText(pre.textContent);
@@ -107,14 +198,26 @@ console.log(resp.choices[0].message.content);`,
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
h("div", { class: "muted small", style: "margin-bottom:8px" },
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px" }, seg, h("div", { class: "grow" }), copy),
pre
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), copy),
pre,
statusHost,
protoWarn,
h("div", { class: "muted small", style: "margin-top:6px" }, t("keys.h.note"))
);
}
async function reload() {
const data = await api.listKeys();
const keys = data.keys || [];
// per-account usage: window breakdown + cumulative-forever totals — the
// "who spends what" view for departments/developers sharing one gateway
let usage = {};
let alltime = {};
try {
const st = await api.stats(usageRange, "account");
for (const b of st.breakdown || []) usage[b.key] = b;
alltime = st.alltime_by_account || {};
} catch { /* stats disabled — columns show dashes */ }
mount(
root,
h(
@@ -124,6 +227,9 @@ console.log(resp.choices[0].message.content);`,
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("keys.title")), h("p", {}, t("keys.subtitle"))),
h("div", { class: "seg", style: "margin-right:10px" },
...["24h", "7d", "30d"].map((r) =>
h("button", { class: r === usageRange ? "active" : "", onClick: () => { usageRange = r; reload(); } }, r))),
h("button", { class: "btn primary", onClick: () => openCreate(reload) }, "+ " + t("keys.add"))
)
),
@@ -142,7 +248,16 @@ console.log(resp.choices[0].message.content);`,
h(
"thead",
{},
h("tr", {}, h("th", {}, t("keys.col.key")), h("th", {}, t("keys.col.account")), h("th", {}, t("keys.col.rate")), h("th", {}, t("keys.col.models")), h("th", { class: "right-align" }, t("common.actions")))
h("tr", {},
h("th", {}, t("keys.col.key")),
h("th", {}, t("keys.col.account")),
h("th", {}, t("keys.col.rate")),
h("th", {}, t("keys.col.models")),
h("th", { class: "right-align" }, t("keys.col.requests") + " (" + usageRange + ")"),
h("th", { class: "right-align" }, t("keys.col.tokens") + " (" + usageRange + ")"),
h("th", { class: "right-align" }, "$ (" + usageRange + ")"),
h("th", { class: "right-align" }, t("keys.col.costAll")),
h("th", { class: "right-align" }, t("common.actions")))
),
h(
"tbody",
@@ -155,6 +270,10 @@ console.log(resp.choices[0].message.content);`,
h("td", {}, k.account),
h("td", { class: "mono" }, k.rate_per_min || "—"),
h("td", {}, (k.allow_models && k.allow_models.length) ? k.allow_models.join(", ") : h("span", { class: "muted" }, t("keys.allowAll"))),
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
h("td", { class: "right-align" }, alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—"),
h(
"td",
{ class: "right-align" },
+12 -1
View File
@@ -21,6 +21,17 @@ export function renderLogs() {
);
const pauseBtn = h("button", { class: "btn ghost sm" }, "⏸ " + t("logs.pause"));
const copyBtn = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
const clearBtn = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("dash.clear"));
clearBtn.addEventListener("click", async () => {
try {
await api.clearLogs();
lines = [];
paint();
toast(t("dash.cleared"), "ok");
} catch (e) {
toast(String(e.message || e), "err");
}
});
let lines = [];
function paint() {
@@ -76,7 +87,7 @@ export function renderLogs() {
h(
"div",
{ class: "card" },
h("div", { class: "flex", style: "gap:8px; flex-wrap:wrap; margin-bottom:10px" }, filterIn, levelSel, pauseBtn, copyBtn),
h("div", { class: "flex", style: "gap:8px; flex-wrap:wrap; margin-bottom:10px" }, filterIn, levelSel, pauseBtn, copyBtn, clearBtn),
pre
)
);
+4 -4
View File
@@ -4,8 +4,8 @@
// video → `cortiq animate` over a local text-to-video .cmf (MiniMax-H3)
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
import { h, mount, toast, money, ms } from "../ui.js";
import { t } from "../i18n.js";
import { h, mount, toast, money, moneyEl, ms } from "../ui.js";
import { t, trTier } from "../i18n.js";
import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
import { appState } from "../app.js";
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
@@ -206,7 +206,7 @@ export async function renderPlayground() {
decision,
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
kv(t("pg.task"), c.task_label || "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, trTier(c.complexity.tier)) : "—"),
h(
"div",
{ class: "kv", style: "display:block" },
@@ -218,7 +218,7 @@ export async function renderPlayground() {
kv(t("pg.tokens"), haveUsage ? `${r.usage_estimated ? "≈ " : ""}${u.prompt_tokens ?? 0} / ${u.completion_tokens ?? 0}` : "—"),
kv(t("pg.tokSpeed"), tokS != null ? tokS + " tok/s" : "—"),
kv(t("pg.ttft"), r.ttft_ms != null ? ms(r.ttft_ms) : "—"),
kv(t("pg.cost"), money(c.cost_usd)),
kv(t("pg.cost"), moneyEl(c.cost_usd)),
kv(t("pg.latency"), ms(r.latency_ms)),
kv(t("pg.failover"), c.failover ? "yes" : "no")
);
+153 -10
View File
@@ -1,6 +1,6 @@
// Routing — tier editor (ordered model chips), default model, and policy.
import { h, mount, toast } from "../ui.js";
import { t } from "../i18n.js";
import { t, trTier } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
@@ -54,10 +54,24 @@ export async function renderRouting() {
// local editable state — ensure standard tiers exist for a friendly editor
const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {});
const tasks = {};
for (const l of taskLabels) tasks[l] = (routing.tasks || {})[l] || [];
// A task rule is edited as ordered PAIRS (model, tier): "this model serves
// this task at this complexity" (tier "any" = at every complexity).
// GET returns {any:[], low:[], medium:[], high:[]}; legacy arrays = any.
const taskPairs = {};
for (const l of taskLabels) {
const v = (routing.tasks || {})[l];
const pairs = [];
if (Array.isArray(v)) for (const m of v) pairs.push({ model: m, tier: "any" });
else if (v && typeof v === "object") {
for (const tier of ["low", "medium", "high", "any"])
for (const m of v[tier] || []) pairs.push({ model: m, tier });
}
taskPairs[l] = pairs;
}
const st = {
tiers,
tasks,
taskPairs,
bands: routing.bands || null, // null = пороги роутера как есть
default: routing.default || ids[0] || "",
policy: routing.policy || { mode: "fixed_table", min_class: {} },
};
@@ -107,11 +121,97 @@ export async function renderRouting() {
);
}
// Linked boundary sliders: one track, two handles (low|medium boundaries),
// three colored zones. Handles push each other, numbers update live —
// nobody types score ranges by hand.
function bandsCard() {
const DEF = { low: 0.33, medium: 0.48 };
const enabled = !!st.bands;
const b = st.bands || { ...DEF };
const MIN_GAP = 0.02;
const track = h("div", { class: "bands-track" });
const zoneLow = h("div", { class: "bands-zone z-low" });
const zoneMed = h("div", { class: "bands-zone z-med" });
const zoneHigh = h("div", { class: "bands-zone z-high" });
const hLow = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
const hMed = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
const legend = h("div", { class: "bands-legend" });
track.append(zoneLow, zoneMed, zoneHigh, hLow, hMed);
function paint() {
const lo = b.low, me = b.medium;
zoneLow.style.width = lo * 100 + "%";
zoneMed.style.left = lo * 100 + "%";
zoneMed.style.width = (me - lo) * 100 + "%";
zoneHigh.style.left = me * 100 + "%";
zoneHigh.style.width = (1 - me) * 100 + "%";
hLow.style.left = lo * 100 + "%";
hMed.style.left = me * 100 + "%";
hLow.querySelector(".bands-val").textContent = lo.toFixed(2);
hMed.querySelector(".bands-val").textContent = me.toFixed(2);
// подписи не наезжают, когда ручки рядом
hMed.querySelector(".bands-val").classList.toggle("raised", me - lo < 0.09);
mount(legend,
h("span", { class: "badge tier-low" }, `${trTier("low")} ≤ ${lo.toFixed(2)}`),
h("span", { class: "badge tier-medium" }, `${trTier("medium")} ≤ ${me.toFixed(2)}`),
h("span", { class: "badge tier-high" }, `${trTier("high")} > ${me.toFixed(2)}`));
track.classList.toggle("disabled", !st.bands);
}
function drag(handle, which) {
handle.addEventListener("pointerdown", (e) => {
if (!st.bands) return;
handle.setPointerCapture(e.pointerId);
const rect = track.getBoundingClientRect();
const move = (ev) => {
let v = (ev.clientX - rect.left) / rect.width;
v = Math.min(0.98, Math.max(0.02, Math.round(v * 100) / 100));
if (which === "low") {
b.low = v;
// связанность: low толкает medium перед собой
if (b.medium < b.low + MIN_GAP) b.medium = Math.min(0.99, b.low + MIN_GAP);
} else {
b.medium = v;
// и medium толкает low назад
if (b.low > b.medium - MIN_GAP) b.low = Math.max(0.01, b.medium - MIN_GAP);
}
st.bands = { low: b.low, medium: b.medium };
paint();
};
const up = () => {
handle.removeEventListener("pointermove", move);
handle.removeEventListener("pointerup", up);
};
handle.addEventListener("pointermove", move);
handle.addEventListener("pointerup", up);
});
}
drag(hLow, "low");
drag(hMed, "medium");
const toggle = h("input", { type: "checkbox", checked: enabled ? "" : null });
toggle.checked = enabled;
toggle.addEventListener("change", () => {
st.bands = toggle.checked ? { low: b.low, medium: b.medium } : null;
paint();
});
paint();
return h(
"div",
{ class: "bands-wrap" },
h("label", { class: "check", style: "margin-bottom:8px" }, toggle, t("routing.bands.enable")),
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.bands.hint")),
track,
legend
);
}
function tierRow(tier) {
return h(
"div",
{ class: "tierrow" },
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, trTier(tier))),
chipEditor(st.tiers, tier)
);
}
@@ -130,17 +230,52 @@ export async function renderRouting() {
h("div", { class: "muted small mono" }, label),
share != null ? h("div", { class: "muted small" }, t("routing.tasks.share", { p: Math.round(share * 100) })) : null
),
chipEditor(st.tasks, label, t("routing.tasks.inherit"))
pairsEditor(label)
);
}
// Pairs editor: each row is [model ▾][complexity ▾][×]; "any" serves every
// tier. Order matters — it is the fallback order inside the tier.
function pairsEditor(label) {
const pairs = st.taskPairs[label];
const wrap = h("div", { class: "pairs grow" });
function paint() {
mount(
wrap,
...pairs.map((p, i) => {
const modelSel = h(
"select",
{ class: "pair-model" },
...ids.map((id) => h("option", { value: id }, id))
);
modelSel.value = p.model;
modelSel.addEventListener("change", () => { p.model = modelSel.value; });
const tierSel = h(
"select",
{ class: "pair-tier", title: t("routing.tasks.tierHint") },
h("option", { value: "any" }, t("routing.tasks.tierAny")),
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, trTier(tr)))
);
tierSel.value = p.tier;
tierSel.addEventListener("change", () => { p.tier = tierSel.value; });
const del = h("button", { class: "btn sm ghost danger", onClick: () => { pairs.splice(i, 1); paint(); } }, "×");
return h("div", { class: "pair-row" }, modelSel, tierSel, del);
}),
pairs.length === 0 ? h("div", { class: "muted small", style: "align-self:center" }, t("routing.tasks.inherit")) : null,
h("button", { class: "btn ghost sm", onClick: () => { pairs.push({ model: ids[0] || "", tier: "any" }); paint(); } }, "+ " + t("routing.tasks.addModel"))
);
}
paint();
return wrap;
}
function tasksCard() {
const addIn = h("input", { placeholder: t("routing.tasks.addPlaceholder"), style: "max-width:220px" });
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("routing.tasks.addCustom"));
addBtn.addEventListener("click", () => {
const v = addIn.value.trim().toLowerCase();
if (!v || st.tasks[v]) return;
st.tasks[v] = [];
if (!v || st.taskPairs[v]) return;
st.taskPairs[v] = [];
render();
});
return h(
@@ -151,7 +286,7 @@ export async function renderRouting() {
h("span", { class: "card-head sub" },
labelsSource === "router" ? t("routing.tasks.fromRouter") : t("routing.tasks.builtin"))),
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.tasks.hint")),
...Object.keys(st.tasks).map(taskRow),
...Object.keys(st.taskPairs).map(taskRow),
h("div", { class: "flex", style: "gap:8px; margin-top:10px" }, addIn, addBtn)
);
}
@@ -201,8 +336,15 @@ export async function renderRouting() {
saveBtn.disabled = true;
try {
const tasks = {};
for (const [k, v] of Object.entries(st.tasks)) if (v.length) tasks[k] = v;
await api.putRouting({ tiers: st.tiers, tasks, default: st.default, policy: st.policy });
for (const [k, pairs] of Object.entries(st.taskPairs)) {
const rule = {};
for (const p of pairs) {
if (!p.model) continue;
(rule[p.tier] = rule[p.tier] || []).push(p.model);
}
if (Object.keys(rule).length) tasks[k] = rule;
}
await api.putRouting({ tiers: st.tiers, tasks, bands: st.bands, default: st.default, policy: st.policy });
toast(t("toast.saved"), "good");
} catch (e) {
toast(String(e.message || e), "bad");
@@ -227,6 +369,7 @@ export async function renderRouting() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
bandsCard(),
...Object.keys(st.tiers).map(tierRow)
),
tasksCard(),