Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b339c2b622 | ||
|
|
34093330f6 | ||
|
|
faddf945d7 | ||
|
|
255d989e83 | ||
|
|
0d9d24125a | ||
|
|
1c59906b49 | ||
|
|
19f666e7b1 | ||
|
|
dbcd9827f6 | ||
|
|
b689180717 | ||
|
|
f385c2254b | ||
|
|
9607a38792 | ||
|
|
239ddc767c | ||
|
|
057abaa95a | ||
|
|
dd864eed22 | ||
|
|
c1ff675d80 | ||
|
|
9ef1af3ae1 | ||
|
|
6f4d83ce13 | ||
|
|
0040bfd7e9 | ||
|
|
10b5101757 | ||
|
|
788c87c565 | ||
|
|
179c414a0e | ||
|
|
115b04f20e | ||
|
|
b24d4cdbab | ||
|
|
06813b3e97 | ||
|
|
95e560dc72 | ||
|
|
ef28a6633d | ||
|
|
b4d36334bf | ||
|
|
dd9ccaea94 | ||
|
|
ff8415d7d4 | ||
|
|
7479cc51c2 | ||
|
|
b1e0dc76ec | ||
|
|
c3dd251cce | ||
|
|
9af51b679e | ||
|
|
1bba427042 | ||
|
|
54b73f1365 | ||
|
|
ec92bdddb7 | ||
|
|
1120d8f11a | ||
|
|
914ff42fc5 | ||
|
|
2ede0ee452 | ||
|
|
15de36be3b | ||
|
|
a77beeabab | ||
|
|
32ce2f7ed7 | ||
|
|
9ec8861464 | ||
|
|
ad438a5dcb | ||
|
|
45def5c6b6 | ||
|
|
5d8455b4a0 | ||
|
|
b255c60b0b | ||
|
|
a75cacd0ec | ||
|
|
850a54acdf | ||
|
|
22110ac524 | ||
|
|
b3b37d27f4 | ||
|
|
a7c8ed72c7 | ||
|
|
f2e6047bff | ||
|
|
0b28aa3d59 | ||
|
|
1414ad1b52 | ||
|
|
94c8551206 | ||
|
|
6ecc709fd2 | ||
|
|
1661c19550 | ||
|
|
66ee32b233 | ||
|
|
f6ee045989 | ||
|
|
59e2eb3215 | ||
|
|
5b743a4691 | ||
|
|
c59ed9c949 | ||
|
|
b55e72789f | ||
|
|
0d1283c3fa | ||
|
|
0105a43ef5 | ||
|
|
b8d48c31f7 | ||
|
|
196ff52f0f | ||
|
|
e25814a54c | ||
|
|
0c20eb34f0 | ||
|
|
f8c1bb0914 | ||
|
|
0f299b77e6 | ||
|
|
7c49510c46 | ||
|
|
31da84cc45 | ||
|
|
749a47ce9b | ||
|
|
d2b1d50eb8 | ||
|
|
d48349a144 | ||
|
|
f05d26bfbb | ||
|
|
39f1fcb600 | ||
|
|
42377a75d4 | ||
|
|
6cf2608fc8 | ||
|
|
e6d1a630c9 | ||
|
|
8dd77ca7ff | ||
|
|
630149d141 | ||
|
|
85fc508f04 | ||
|
|
010171e05c | ||
|
|
39c56145ed | ||
|
|
17c8186383 | ||
|
|
e773e18efd | ||
|
|
6466006de6 | ||
|
|
6a25d8ac08 | ||
|
|
e486617e51 | ||
|
|
729a8bfbba | ||
|
|
5759de153b | ||
|
|
bf8534f13d | ||
|
|
70d8cde258 | ||
|
|
3d71b03788 | ||
|
|
492953d4c2 | ||
|
|
3b0231b32d | ||
|
|
fd2f2a165b | ||
|
|
cbc385f6c7 |
@@ -1,4 +1,6 @@
|
||||
target
|
||||
models
|
||||
*.cmf
|
||||
.git
|
||||
.github
|
||||
docs
|
||||
|
||||
@@ -21,9 +21,20 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
# Skip when this version is already on crates.io — a version can never be
|
||||
# re-uploaded, so publishing one that exists (released by hand, or a re-run
|
||||
# of this job) is a guaranteed failure rather than a real problem.
|
||||
- name: Publish to crates.io
|
||||
if: ${{ env.CARGO_REGISTRY_TOKEN != '' }}
|
||||
run: cargo publish
|
||||
run: |
|
||||
name=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].name')
|
||||
version=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].version')
|
||||
if curl -sf -H 'User-Agent: cortiq-gateway-release' \
|
||||
"https://crates.io/api/v1/crates/$name/$version" >/dev/null; then
|
||||
echo "$name $version is already published — skipping"
|
||||
else
|
||||
cargo publish
|
||||
fi
|
||||
|
||||
# --- prebuilt binaries + GitHub Release -----------------------------------
|
||||
release:
|
||||
|
||||
+351
@@ -3,6 +3,357 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.45] - 2026-08-21
|
||||
|
||||
### Fixed
|
||||
- **Open WebUI (and every strict OpenAI client) can finally chat with the gateway.**
|
||||
`POST /v1/chat/completions` rejected perfectly ordinary payloads with a bare HTTP 422 —
|
||||
a plain-text body no OpenAI client can parse, which is why the chat window just spun with
|
||||
no answer and no error. The request parser now accepts every shape real clients send:
|
||||
`content` as a string, `null`, or an array of content parts (text parts concatenated,
|
||||
images skipped), `tools: null`, `stream: null`, a string `reasoning_effort`
|
||||
(`low`/`medium`/`high`) as well as a numeric budget, `max_completion_tokens` as an alias
|
||||
for `max_tokens`, `messages[].name` and `messages[].tool_call_id` proxied through, and a
|
||||
missing `Content-Type`. A body that genuinely cannot be read now returns an OpenAI error
|
||||
envelope with HTTP 400.
|
||||
- **Strict OpenAI response JSON, no optional `null`s.** Absent tool calls no longer emit
|
||||
`"tool_calls": null` (the key is omitted entirely); `finish_reason` is mapped to the
|
||||
spec's closed set, so `cortiq serve`'s `max_tokens` reaches clients as `length`; the
|
||||
response always carries a non-empty `id`.
|
||||
- **`model` echoes the id the client requested** (`cortiq-auto` stays `cortiq-auto`, not the
|
||||
upstream's `cortiq`), matching OpenAI's own behaviour. The model that actually answered
|
||||
stays in `cortiq.selected_model` and `X-Cortiq-Selected-Model`. Applies to
|
||||
`/v1/chat/completions`, `/v1/completions` and `/v1/messages`.
|
||||
- **Streaming is re-normalised instead of forwarded verbatim.** Every chunk is reissued as a
|
||||
strict `chat.completion.chunk`: the stream opens with a `role` delta, `message`-shaped
|
||||
chunks are converted to `delta`, empty deltas are dropped, and it always closes with one
|
||||
terminal `finish_reason` and `data: [DONE]` — even when the upstream stops early or dies
|
||||
mid-flight, in which case the failure arrives as an SSE `error` event rather than a client
|
||||
waiting forever. SSE responses also carry `X-Accel-Buffering: no` so nginx/Traefik cannot
|
||||
buffer the stream into a hang.
|
||||
- **Client bookkeeping keys are no longer proxied upstream.** `chat_id`, `session_id`,
|
||||
`metadata`, `features`, `background_tasks` and friends (plus a non-string `user`) are
|
||||
stripped before the provider call, so strict backends stop rejecting the request.
|
||||
|
||||
### Added
|
||||
- **`GET /v1/models` lists managed local CMF models.** Every `[[cmf.servers]]` entry now
|
||||
appears alongside `cortiq-auto` and the `[[models]]` pool, with live `running`/`healthy`
|
||||
state in a non-standard `cortiq` field, so Open WebUI and other clients can pin a local
|
||||
`.cmf` directly. Entries also carry `created`.
|
||||
|
||||
### Changed
|
||||
- Dropped leftover `[to_wire]` debug prints from the OpenAI provider — they wrote to stderr
|
||||
on every single request.
|
||||
|
||||
## [0.2.44] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Power controls in Settings**: «Stop everything & power off» stops every managed local
|
||||
model server and exits the gateway (the panel shows a clear "gateway is off" screen);
|
||||
«Restart everything» (relabeled) re-execs the gateway with its local servers. Both behind
|
||||
confirm dialogs, translated in all seven languages.
|
||||
|
||||
### Fixed
|
||||
- `cortiq-gateway` typed in any directory picks up your configuration. Resolution order:
|
||||
`--config` → a readable `./config/gateway.toml` → `$CORTIQ_GATEWAY_HOME` →
|
||||
`~/.cortiq-gateway/gateway.toml`. An unrelated project that merely contains `Cargo.toml`
|
||||
and a `config/` folder no longer hijacks the config path ("cannot read config
|
||||
config/gateway.toml").
|
||||
|
||||
## [0.2.43] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
|
||||
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
|
||||
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
|
||||
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
|
||||
configured = open mode, as before.
|
||||
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
|
||||
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
|
||||
when the server becomes healthy, so a local model whose chat template can drive tool
|
||||
calling serves agent (tools) traffic with zero manual configuration. Manual override:
|
||||
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
|
||||
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
|
||||
remaining credit is a KPI card, red under $5.
|
||||
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
|
||||
per harness, an install command chip when the harness is missing, and a paste-your-key
|
||||
field that substitutes the real key into every snippet.
|
||||
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
|
||||
Caddy TLS fronting, security checklist, backup list.
|
||||
|
||||
### Fixed
|
||||
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
|
||||
route's chain has no model with the required capabilities (tools), the gateway escalates
|
||||
through medium→high chains and then the rest of the pool instead of returning
|
||||
"No healthy model available". Explicit pins still fail loudly.
|
||||
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
|
||||
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
|
||||
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
|
||||
no history-expansion `!`), and the install chips are readable in the light theme.
|
||||
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
|
||||
into the sidecar totals file — per-account all-time and month spend survive compaction
|
||||
and restarts.
|
||||
|
||||
### Breaking
|
||||
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
|
||||
will now be rejected with 401 — either remove the keys from the config or start sending
|
||||
`Authorization: Bearer <key>`.
|
||||
|
||||
## [0.2.42] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **The continuous complexity score is visible**: the benchmark's routing table and the
|
||||
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
|
||||
see what different prompts actually weigh and tune the boundary sliders from real traffic.
|
||||
|
||||
### Changed
|
||||
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
|
||||
«Рабочий стол».
|
||||
|
||||
## [0.2.41] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
|
||||
(the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
|
||||
other, the numbers ride on the handles and the legend updates live — no hand-typed score
|
||||
ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
|
||||
(`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
|
||||
stats; unchecked = the router's own bands, untouched behavior.
|
||||
|
||||
### Fixed
|
||||
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
|
||||
selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
|
||||
facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
|
||||
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
|
||||
fresh template — difficulty is judged in the background, so the second occurrence is the
|
||||
honest steady-state answer.
|
||||
|
||||
## [0.2.40] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Per-key spend accounting** on the API-keys page: requests, tokens in/out and cost per key
|
||||
for a selectable window (24h / 7d / 30d) plus a cumulative all-time cost column — "which
|
||||
department spends what" at a glance.
|
||||
- **Bounded stats on disk**: once `stats.jsonl` outgrows `stats.max_file_mb` (default 16, 0
|
||||
disables), records older than the retention window fold into cumulative per-account totals in
|
||||
a small sidecar file (tmp+rename on both) and leave the log. All-time accounting survives
|
||||
compaction and restarts; nothing is ever counted twice.
|
||||
- **Per-complexity models inside a task rule**: each task is edited as rows of
|
||||
"model + complexity" pairs — `[routing.tasks.code] low=["cheap"] high=["best"] any=[...]`.
|
||||
The tier-specific pick leads its own tier, `any` (and legacy plain arrays) leads everywhere,
|
||||
the global tier chain and `default` always remain as fallback, and high verdicts still
|
||||
escalate over any-tier pins.
|
||||
|
||||
### Changed
|
||||
- Benchmark hard probes are generated from wider interdisciplinary pools (6 families each) and
|
||||
retry once with a fresh template — the router's novelty decay no longer flips them red.
|
||||
|
||||
### Fixed
|
||||
- The `task-rule` badge is honest: it appears only when the rule actually chose the head of the
|
||||
candidate chain, not whenever a rule merely exists for the task.
|
||||
|
||||
## [0.2.39] - 2026-08-07
|
||||
|
||||
### Added
|
||||
- **Benchmark, redesigned into the console** (⏱ sidebar entry): two explicit modes. **Smart
|
||||
routing** (default) sends every probe through `cortiq-auto` and judges the ROUTER — task label
|
||||
vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge —
|
||||
with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a
|
||||
"requests went to" per-model distribution. **Individual models** compares pinned models
|
||||
head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run ×
|
||||
and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
|
||||
- **Hard probes that exercise the high tier**: two route-judged tests (science / ops pools)
|
||||
generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings
|
||||
decay with the router's novelty signal. Pass = the router said `high` and the request escalated;
|
||||
the tiny token budget keeps them at cents even on the priciest model.
|
||||
- **One-paste harness setup** on the API-keys page: Claude Code (native Anthropic `/v1/messages`,
|
||||
now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline —
|
||||
each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses
|
||||
already installed on its machine (✓ on the tab, "already configured for this gateway" status).
|
||||
- **Stale-tab detection**: open consoles notice a gateway restart via the meta start marker and
|
||||
show a reload banner instead of silently running an old interface.
|
||||
- Log viewer Clear button (`DELETE /admin/api/logs`).
|
||||
|
||||
### Changed
|
||||
- **High complexity escalates over task rules**: at tier `high` the tier chain leads and the task
|
||||
rule demotes to fallback — a `code → cheap-model` rule no longer pins the hardest prompts to
|
||||
the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
|
||||
- **Honest stream costs**: the final cost rides a closing SSE event (`cortiq_cost_usd`) since the
|
||||
initial header is written before generation; money renders with 4 decimals (last two smaller)
|
||||
and grows precision below $0.0001 instead of collapsing to $0.0000.
|
||||
- **Oracle-sized timeouts**: router default 800 ms → 15 s (hosted escalations take ~10 s);
|
||||
provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall
|
||||
cap so long generations and streams are never cut.
|
||||
- Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-`<think>`),
|
||||
router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on
|
||||
reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
|
||||
- New-key dialog simplified to account / rate limit / optional custom value.
|
||||
|
||||
### Fixed
|
||||
- `cortiq bench` results showed "error" with newer cortiq releases that pretty-print `--json`:
|
||||
the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers
|
||||
steady decode speed (the old heuristic could report `allocs_per_token` as tok/s).
|
||||
|
||||
## [0.2.38] - 2026-08-07
|
||||
|
||||
### Added
|
||||
- **Image & video generation in the Playground**: three tabs now — Chat, Image, Video. Image runs
|
||||
`cortiq imagine` over a local text-to-image `.cmf` (Lumina) with width/height/steps/CFG/seed,
|
||||
live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser
|
||||
with one-click PNG export. Video runs `cortiq animate` (MiniMax-H3) with frames/steps/quality
|
||||
and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded
|
||||
in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens
|
||||
opportunistically when ffmpeg exists). Local `.cmf` files are classified chat/image/video via
|
||||
`cortiq info`; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56.
|
||||
- **Task-based routing**: `[routing.tasks]` maps a router task label to an ordered model list.
|
||||
A matching rule's models lead the candidate chain and the complexity chain follows as fallback,
|
||||
so a task pin never weakens failover. The Routing page builds the editor from the account's
|
||||
live taxonomy (`GET /v1/taxonomies`), shows each task's share of 7-day traffic, and supports
|
||||
custom labels. Responses whose chain was headed by a rule report `route_source "router+task-rule"`.
|
||||
- **Generation gallery** under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete),
|
||||
and generations **queue up** (with position shown) instead of being rejected.
|
||||
- **Chat is a real dialogue**: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live
|
||||
metrics: token counts, tok/s and time-to-first-token.
|
||||
- **Exact stream token counts** when the model server provides usage; the gateway fallback counts
|
||||
content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate.
|
||||
- **Jobs survive restarts**: import and media job lists persist to `<data>/jobs-*.json` and are
|
||||
reconciled on start — the Register button no longer vanishes because the gateway restarted.
|
||||
- **Admin console, ops batch**: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy);
|
||||
a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at
|
||||
the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download;
|
||||
a Restart-gateway button; an update-available notice (crates.io checked twice a day); config
|
||||
export as the real `gateway.toml` and TOML import (validated, hot-reloaded); "Connect your app"
|
||||
snippets (curl/Python/JS) on the keys page; one-click `cortiq bench --json` on the Models page.
|
||||
- **Docker ships the cortiq binary** (built in the builder stage, plus `libvulkan1`): local `.cmf`
|
||||
chat models and media generation now work in containers, where auto-install never could.
|
||||
- **Security**: an open admin console listening on 0.0.0.0 outside Docker warns in the log and
|
||||
shows a dismissible banner.
|
||||
|
||||
### Fixed
|
||||
- The default config pointed at a placeholder `localhost:8080` router with a nonexistent taxonomy,
|
||||
leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now
|
||||
targets the hosted allaigate router and degrades gracefully until a key is set.
|
||||
- The wizard's Finish lands on the Playground instead of the Models list.
|
||||
|
||||
## [0.2.37] - 2026-08-06
|
||||
|
||||
### Added
|
||||
- **Zero-argument startup**: `cortiq-gateway` now runs with no flags. The config is created on
|
||||
first launch (repo `config/gateway.toml` when run from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml`, or `$CORTIQ_GATEWAY_HOME`), the browser opens by itself,
|
||||
and a busy port is swapped for the next free one. `--no-browser` / `--no-port-search` opt out;
|
||||
`--config` still takes an explicit path.
|
||||
- **No admin token required by default**: the console opens without auth. Set `--admin-token`,
|
||||
`[admin].token` or `[admin].token_env` to protect it.
|
||||
- **First-run wizard** (`/admin#/onboarding`): pick a language, download a ready-made `.cmf`
|
||||
model, and finish — Finish connects every completed download as a local model, starts its
|
||||
`cortiq serve` and points `routing.default` at it. Nothing to edit by hand.
|
||||
- **Ready-made models are read live** from the `infosave` HuggingFace account in both the wizard
|
||||
and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a
|
||||
delete button. Newly published models appear without a gateway update.
|
||||
- **Windows-friendly data dir**: `%APPDATA%` / `%LOCALAPPDATA%` / `%USERPROFILE%` are used before
|
||||
falling back, so running the exe from `Downloads` never writes into `Program Files`.
|
||||
- **Full localization**: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr).
|
||||
Import, Settings → Local models, the model form and the whole first-run wizard were English-only
|
||||
in de/fr/es/zh/tr before this release.
|
||||
|
||||
### Fixed
|
||||
- **Download progress jumped straight to 100% and then hung**: with `-L`, curl prints a complete
|
||||
`0→100%` progress bar for the redirect hop before the transfer starts, and that bar was being
|
||||
parsed as the download's progress. Progress is now derived from bytes on disk against the size
|
||||
reported by the HuggingFace tree (or a `HEAD` request), so the bar and the reported
|
||||
`done / total` bytes track the real transfer.
|
||||
- **Re-downloading over an existing file reported 100% instantly**: the previous copy is now
|
||||
removed before the transfer starts, so the size poller cannot read stale bytes.
|
||||
- **Downloaded models were not recognised**: the admin UI derived the output file name
|
||||
independently of the gateway, so a finished download did not mark its card as downloaded and the
|
||||
delete button never appeared. The gateway now returns the exact output stem (`local_name`) and
|
||||
the UI sends it back verbatim for download, progress and delete.
|
||||
- **Registering a local model produced an unusable pool entry**: with `manage_server = false` the
|
||||
model was added to the static pool pointing at `cortiq_server_url`, where nothing listens —
|
||||
every call failed with `upstream unavailable`. Registration now always creates a managed server
|
||||
the gateway runs itself, and repairs a stale static entry for the same file.
|
||||
- **Cancel did nothing during a sharded download**: cancellation was only checked between part
|
||||
files, so a multi-GB part had to finish first. The running transfer is now aborted immediately
|
||||
and the partial output removed.
|
||||
- **Deleting a model left the config broken**: removing a `.cmf` now also unregisters the model
|
||||
and stops its managed server instead of leaving a pool entry pointing at a missing file.
|
||||
- **Two variants of one repo collapsed into a single job row**; jobs are now keyed by repo + variant.
|
||||
- Curl's stderr is captured into the job log, so a failed download explains itself instead of
|
||||
showing a bare exit code.
|
||||
|
||||
## [0.2.36] - 2026-07-25
|
||||
|
||||
### Fixed
|
||||
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
|
||||
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
|
||||
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
|
||||
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.
|
||||
|
||||
## [0.2.35] - 2026-07-23
|
||||
|
||||
### Fixed
|
||||
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
|
||||
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.
|
||||
|
||||
## [0.2.34] - 2026-07-23
|
||||
|
||||
### Added
|
||||
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
|
||||
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.
|
||||
|
||||
### Fixed
|
||||
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
|
||||
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.
|
||||
|
||||
## [0.2.31] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
|
||||
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.
|
||||
|
||||
### Fixed
|
||||
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
|
||||
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.
|
||||
|
||||
## [0.2.30] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
|
||||
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.
|
||||
|
||||
|
||||
### Added
|
||||
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
|
||||
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
### Added
|
||||
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
|
||||
gateway + a local Ollama model in one command. GHCR badge in the READMEs.
|
||||
|
||||
## [0.2.2] - 2026-07-01
|
||||
|
||||
### Added
|
||||
|
||||
Generated
+249
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.2"
|
||||
version = "0.2.45"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
@@ -297,10 +297,12 @@ dependencies = [
|
||||
"futures",
|
||||
"getrandom 0.2.17",
|
||||
"mime_guess",
|
||||
"open",
|
||||
"reqwest",
|
||||
"rust-embed",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sysinfo",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tokio-stream",
|
||||
@@ -340,6 +342,16 @@ dependencies = [
|
||||
"crypto-common",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dispatch2"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "displaydoc"
|
||||
version = "0.2.6"
|
||||
@@ -743,6 +755,25 @@ version = "2.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
|
||||
|
||||
[[package]]
|
||||
name = "is-docker"
|
||||
version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "928bae27f42bc99b60d9ac7334e3a21d10ad8f1835a4e12ec3ec0464765ed1b3"
|
||||
dependencies = [
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is-wsl"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "173609498df190136aa7dea1a91db051746d339e18476eed5ca40521f02d7aa5"
|
||||
dependencies = [
|
||||
"is-docker",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is_terminal_polyfill"
|
||||
version = "1.70.2"
|
||||
@@ -853,6 +884,15 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ntapi"
|
||||
version = "0.4.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae"
|
||||
dependencies = [
|
||||
"winapi",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nu-ansi-term"
|
||||
version = "0.50.3"
|
||||
@@ -862,6 +902,63 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2"
|
||||
version = "0.6.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3a12a8ed07aefc768292f076dc3ac8c48f3781c8f2d5851dd3d98950e8c5a89f"
|
||||
dependencies = [
|
||||
"objc2-encode",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-core-foundation"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"dispatch2",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-encode"
|
||||
version = "4.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33"
|
||||
|
||||
[[package]]
|
||||
name = "objc2-foundation"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-io-kit"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"objc2-core-foundation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-open-directory"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bb82bed227edf5201dfedf072bba4015a33d3d4a98519837295a90f0a23f676d"
|
||||
dependencies = [
|
||||
"objc2",
|
||||
"objc2-core-foundation",
|
||||
"objc2-foundation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
@@ -874,6 +971,16 @@ version = "1.70.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
||||
|
||||
[[package]]
|
||||
name = "open"
|
||||
version = "5.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f9cfef937e9c486488c7e3d949ae31c0f1d06bdacd75b99c086cb35356e30408"
|
||||
dependencies = [
|
||||
"is-wsl",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot"
|
||||
version = "0.12.5"
|
||||
@@ -1400,6 +1507,21 @@ dependencies = [
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sysinfo"
|
||||
version = "0.39.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d2071df9448915b71c4fe6d25deaf1c22f12bd234f01540b77312bb8e41361e6"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"memchr",
|
||||
"ntapi",
|
||||
"objc2-core-foundation",
|
||||
"objc2-io-kit",
|
||||
"objc2-open-directory",
|
||||
"windows",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "thiserror"
|
||||
version = "1.0.69"
|
||||
@@ -1884,6 +2006,22 @@ dependencies = [
|
||||
"rustls-pki-types",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winapi"
|
||||
version = "0.3.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
|
||||
dependencies = [
|
||||
"winapi-i686-pc-windows-gnu",
|
||||
"winapi-x86_64-pc-windows-gnu",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winapi-i686-pc-windows-gnu"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
|
||||
|
||||
[[package]]
|
||||
name = "winapi-util"
|
||||
version = "0.1.11"
|
||||
@@ -1893,12 +2031,113 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winapi-x86_64-pc-windows-gnu"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
|
||||
|
||||
[[package]]
|
||||
name = "windows"
|
||||
version = "0.62.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580"
|
||||
dependencies = [
|
||||
"windows-collections",
|
||||
"windows-core",
|
||||
"windows-future",
|
||||
"windows-numerics",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-collections"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-core"
|
||||
version = "0.62.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
|
||||
dependencies = [
|
||||
"windows-implement",
|
||||
"windows-interface",
|
||||
"windows-link",
|
||||
"windows-result",
|
||||
"windows-strings",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-future"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-link",
|
||||
"windows-threading",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-implement"
|
||||
version = "0.60.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-interface"
|
||||
version = "0.59.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-link"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
|
||||
|
||||
[[package]]
|
||||
name = "windows-numerics"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-result"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-strings"
|
||||
version = "0.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.52.0"
|
||||
@@ -1959,6 +2198,15 @@ dependencies = [
|
||||
"windows_x86_64_msvc 0.53.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-threading"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_gnullvm"
|
||||
version = "0.52.6"
|
||||
|
||||
+18
-5
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.2"
|
||||
version = "0.2.45"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
@@ -10,12 +10,16 @@ homepage = "https://api.allaigate.com"
|
||||
readme = "README.md"
|
||||
keywords = ["llm", "gateway", "proxy", "openai", "routing"]
|
||||
categories = ["web-programming::http-server", "command-line-utilities"]
|
||||
# Files shipped in the published crate. The embedded admin SPA (web/) MUST be
|
||||
# included because rust-embed compiles it in; docs/screenshots are excluded to
|
||||
# keep the crate lean.
|
||||
# Files shipped in the published crate. The embedded admin SPA MUST be included
|
||||
# because rust-embed compiles it in; docs/screenshots are excluded to keep the
|
||||
# crate lean. The web/ subfolders are listed one by one rather than as web/**/*
|
||||
# so stray scratch files dropped next to the SPA never end up published.
|
||||
include = [
|
||||
"src/**/*",
|
||||
"web/**/*",
|
||||
"web/index.html",
|
||||
"web/favicon.svg",
|
||||
"web/css/**/*",
|
||||
"web/js/**/*",
|
||||
"config/gateway.example.toml",
|
||||
"README.md",
|
||||
"CHANGELOG.md",
|
||||
@@ -54,6 +58,11 @@ tokio-stream = "0.1"
|
||||
|
||||
# CLI / config
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
# open browser on first run (optional)
|
||||
open = { version = "5", optional = true }
|
||||
|
||||
# host metrics for the admin dashboard (CPU / RAM / disk)
|
||||
sysinfo = "0.39"
|
||||
|
||||
# errors / logging / metrics
|
||||
thiserror = "1"
|
||||
@@ -61,6 +70,10 @@ anyhow = "1"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
|
||||
[features]
|
||||
default = ["open-browser"]
|
||||
open-browser = ["open"]
|
||||
|
||||
[profile.release]
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
|
||||
+19
-5
@@ -1,6 +1,12 @@
|
||||
# ---- build stage ------------------------------------------------------------
|
||||
FROM rust:1-slim AS builder
|
||||
# pin builder and runtime to the same Debian release — a floating `rust:1-slim`
|
||||
# drifts to a newer glibc than the runtime image and the binary refuses to start
|
||||
FROM rust:1-slim-trixie AS builder
|
||||
WORKDIR /app
|
||||
# the cortiq runtime CANNOT be auto-installed inside the container (no cargo in
|
||||
# the runtime image), so it is compiled here and shipped alongside the gateway —
|
||||
# local .cmf chat models and image/video generation work in Docker out of the box
|
||||
RUN cargo install cortiq-cli --locked
|
||||
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY src ./src
|
||||
@@ -8,18 +14,26 @@ COPY web ./web
|
||||
RUN cargo build --release
|
||||
|
||||
# ---- runtime stage ----------------------------------------------------------
|
||||
FROM debian:bookworm-slim
|
||||
FROM debian:trixie-slim
|
||||
# curl — model imports download ready .cmf files with it;
|
||||
# libvulkan1 — optional GPU offload for cortiq (falls back to CPU without a GPU)
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl libvulkan1 \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& useradd -r -u 10001 cortiq \
|
||||
&& mkdir -p /app/config
|
||||
&& mkdir -p /app/config /app/data/models
|
||||
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
|
||||
COPY --from=builder /usr/local/cargo/bin/cortiq /usr/local/bin/cortiq
|
||||
# ship the example as the default config so `docker run` works out of the box
|
||||
COPY config/gateway.example.toml /app/config/gateway.toml
|
||||
RUN chown -R 10001:10001 /app
|
||||
WORKDIR /app
|
||||
USER cortiq
|
||||
# all gateway state (downloaded models, stats, secrets) lives under /app/data —
|
||||
# mount a volume there to keep it across container recreates
|
||||
ENV CORTIQ_GATEWAY_HOME=/app/data
|
||||
# runs as root so writes to bind-mounted host folders (Docker Desktop on
|
||||
# Windows/macOS) don't fail with permission errors (curl exit code 23);
|
||||
# add `--user 10001` to docker run to harden when not using host bind mounts
|
||||
EXPOSE 9000
|
||||
ENTRYPOINT ["cortiq-gateway"]
|
||||
CMD ["--config", "/app/config/gateway.toml"]
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
**English** · [Русский](https://github.com/infosave2007/cortiq-gateway/blob/master/README.ru.md)
|
||||
|
||||
[](https://crates.io/crates/cortiq-gateway)
|
||||
[](https://github.com/infosave2007/cortiq-gateway/pkgs/container/cortiq-gateway)
|
||||
[](https://github.com/infosave2007/cortiq-gateway/actions/workflows/ci.yml)
|
||||
[](https://github.com/infosave2007/cortiq-gateway/blob/master/LICENSE)
|
||||

|
||||
@@ -35,10 +36,12 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Zero-config start.** `cortiq-gateway` with no flags: the config is created for you, a busy port is swapped for a free one, the browser opens on a setup wizard that downloads a ready-made local model and wires it into routing — no TOML, no token, no separate model server.
|
||||
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
|
||||
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
|
||||
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down.
|
||||
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. Plus **per-task rules** (`[routing.tasks]`): pin translations to a local model and code to your strongest coder — the complexity chain stays as fallback ([docs](docs/ROUTING.md)).
|
||||
- **Image & video generation.** The Playground drives local text-to-image (Lumina) and text-to-video (MiniMax-H3) `.cmf` models via `cortiq imagine`/`animate`: live progress with ETA, a generation gallery, PNG export, in-browser AVI playback — no ffmpeg anywhere.
|
||||
- **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard).
|
||||
- **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes.
|
||||
- **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint.
|
||||
@@ -53,11 +56,13 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
```bash
|
||||
# Docker (GitHub Container Registry)
|
||||
docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
-v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
-v "$PWD/config:/app/config" -v cortiq-data:/app/data \
|
||||
ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
# → admin console at http://localhost:9000/admin
|
||||
|
||||
# from crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # no flags, no config file, no token — see Quick start
|
||||
|
||||
# or build from source
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -66,7 +71,23 @@ cargo build --release # ./target/release/cortiq-gateway
|
||||
```
|
||||
|
||||
The image ships a default config and runs out of the box; mount `/app/config` (with a
|
||||
`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
|
||||
`gateway.toml`) to use your own. Downloaded models and stats live in `/app/data` — mount
|
||||
a volume there so they survive container recreates (a host folder works too, e.g.
|
||||
`-v "$PWD/models:/app/data/models"`). Tags: `:latest`, `:X.Y.Z`, `:edge`
|
||||
(linux/amd64 + arm64).
|
||||
|
||||
**Docker Compose** — gateway + a local Ollama model in one command:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
|
||||
# → OpenAI API at http://localhost:9000/v1 · admin at /admin (token: change-me)
|
||||
```
|
||||
|
||||
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
|
||||
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
|
||||
|
||||
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
|
||||
|
||||
---
|
||||
|
||||
@@ -86,32 +107,54 @@ The gateway ships with an embedded web console at **`/admin`**:
|
||||
|  |
|
||||
| test the live pipeline and see the routing decision |
|
||||
|
||||
The console is **open by default** — no token needed. To protect it, set any one of
|
||||
`--admin-token <TOKEN>`, `[admin].token`, or `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <YOUR_TOKEN>
|
||||
cortiq-gateway --admin-token <YOUR_TOKEN>
|
||||
# open http://localhost:9000/admin?token=<YOUR_TOKEN>
|
||||
```
|
||||
|
||||
If `--admin-token` / `[admin].token_env` are not set, a token is generated at startup
|
||||
and printed to the log. All `/admin/api/*` endpoints require a Bearer admin token;
|
||||
secret values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
Once a token is set, every `/admin/api/*` endpoint requires it as a Bearer token. Secret
|
||||
values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick start
|
||||
|
||||
```bash
|
||||
# 1. start a router (the hosted allaigate router, or a local cortiq-router)
|
||||
# it listens e.g. on http://localhost:8080 (or https://api.allaigate.com)
|
||||
|
||||
# 2. describe your model pool
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. run the gateway
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
# Gateway listens on 0.0.0.0:9000 and serves the admin console at /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
That is the whole command. On first run the gateway:
|
||||
|
||||
1. **creates its config** — `config/gateway.toml` when started from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml` (override with `$CORTIQ_GATEWAY_HOME` or `--config`);
|
||||
2. **picks a free port** if 9000 is taken, and writes the choice back to the config;
|
||||
3. **opens your browser** on the setup wizard at `/admin#/onboarding`.
|
||||
|
||||
The wizard takes you from nothing to a working local model:
|
||||
|
||||
| Step | What happens |
|
||||
|---|---|
|
||||
| **Language** | 7 languages; changeable later from the header |
|
||||
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
|
||||
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
|
||||
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, `routing.default` points at it, and you land in the Playground |
|
||||
|
||||
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
|
||||
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
|
||||
both still work, and changes hot-reload without a restart.
|
||||
|
||||
```bash
|
||||
# explicit config / port / no browser — all optional
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
For smart routing across a pool, point `[router]` at the hosted allaigate router or a local
|
||||
`cortiq-router` (see [docs/ROUTER.md](docs/ROUTER.md)). Without a router the gateway degrades
|
||||
gracefully to the default model, so a local-only setup works out of the box.
|
||||
|
||||
Now any OpenAI client works through the gateway:
|
||||
|
||||
```python
|
||||
|
||||
+55
-15
@@ -2,6 +2,8 @@
|
||||
|
||||
[English](README.md) · **Русский**
|
||||
|
||||
[](https://crates.io/crates/cortiq-gateway)
|
||||
[](https://github.com/infosave2007/cortiq-gateway/pkgs/container/cortiq-gateway)
|
||||
[](LICENSE)
|
||||

|
||||

|
||||
@@ -34,9 +36,11 @@
|
||||
|
||||
## ✨ Ключевое
|
||||
|
||||
- **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели.
|
||||
- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
|
||||
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
|
||||
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. Плюс **правила по задачам** (`[routing.tasks]`): переводы — в локальную модель, код — сильнейшему кодеру; цепочка по сложности остаётся резервом ([доки](docs/ROUTING.ru.md)).
|
||||
- **Генерация картинок и видео.** Песочница запускает локальные text-to-image (Lumina) и text-to-video (MiniMax-H3) `.cmf` модели через `cortiq imagine`/`animate`: живой прогресс с ETA, галерея генераций, экспорт в PNG, проигрывание AVI прямо в браузере — без ffmpeg вообще.
|
||||
- **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
|
||||
- **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
|
||||
- **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга.
|
||||
@@ -55,6 +59,7 @@ docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
|
||||
# с crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # без флагов, без конфига, без токена — см. Быстрый старт
|
||||
|
||||
# или сборка из исходников
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -66,6 +71,19 @@ cargo build --release # ./target/release/cortiq-gateway
|
||||
(с `gateway.toml`), чтобы использовать свой. Теги: `:latest`, `:X.Y.Z`, `:edge`
|
||||
(linux/amd64 + arm64).
|
||||
|
||||
**Docker Compose** — шлюз + локальная модель Ollama одной командой:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скачать модель
|
||||
# → OpenAI API на http://localhost:9000/v1 · панель на /admin (токен: change-me)
|
||||
```
|
||||
|
||||
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
|
||||
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
|
||||
|
||||
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Панель управления
|
||||
@@ -81,32 +99,54 @@ cargo build --release # ./target/release/cortiq-gateway
|
||||
|  |  |
|
||||
| **Playground** — тест живого пайплайна с решением роутинга | **Dashboard** — трафик, стоимость, латентность |
|
||||
|
||||
Панель **по умолчанию открыта** — токен не нужен. Чтобы закрыть её, задайте любое из
|
||||
`--admin-token <ТОКЕН>`, `[admin].token` или `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <ВАШ_ТОКЕН>
|
||||
cortiq-gateway --admin-token <ВАШ_ТОКЕН>
|
||||
# откройте http://localhost:9000/admin?token=<ВАШ_ТОКЕН>
|
||||
```
|
||||
|
||||
Если `--admin-token` / `[admin].token_env` не заданы — токен генерируется при старте и
|
||||
печатается в лог. Все эндпоинты `/admin/api/*` требуют Bearer admin-токен; значения
|
||||
секретов API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
Когда токен задан, все эндпоинты `/admin/api/*` требуют его как Bearer. Значения секретов
|
||||
API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Быстрый старт
|
||||
|
||||
```bash
|
||||
# 1. поднимите роутер (хостовый allaigate или локальный cortiq-router)
|
||||
# он слушает, например, http://localhost:8080 (или https://api.allaigate.com)
|
||||
|
||||
# 2. опишите свой пул моделей
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. запустите шлюз
|
||||
cargo run --release -- --config config/gateway.toml
|
||||
# Шлюз слушает 0.0.0.0:9000 и отдаёт панель на /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
Это вся команда. При первом запуске шлюз:
|
||||
|
||||
1. **создаёт конфиг** — `config/gateway.toml`, если запущен из репозитория, иначе
|
||||
`~/.cortiq-gateway/gateway.toml` (переопределяется через `$CORTIQ_GATEWAY_HOME` или `--config`);
|
||||
2. **подбирает свободный порт**, если 9000 занят, и записывает выбор в конфиг;
|
||||
3. **открывает браузер** на мастере настройки `/admin#/onboarding`.
|
||||
|
||||
Мастер доводит от нуля до работающей локальной модели:
|
||||
|
||||
| Шаг | Что происходит |
|
||||
|---|---|
|
||||
| **Язык** | 7 языков; потом меняется в шапке |
|
||||
| **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки |
|
||||
| **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` |
|
||||
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё — и вы попадаете в Песочницу |
|
||||
|
||||
Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели.
|
||||
Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг
|
||||
руками — оба пути работают, изменения применяются на лету без перезапуска.
|
||||
|
||||
```bash
|
||||
# явный конфиг / порт / без браузера — всё опционально
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
Для умного роутинга по пулу направьте `[router]` на хостовый allaigate или локальный
|
||||
`cortiq-router` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)). Без роутера шлюз мягко
|
||||
деградирует до модели по умолчанию, поэтому чисто локальная установка работает сразу.
|
||||
|
||||
Теперь любой OpenAI-клиент работает через шлюз:
|
||||
|
||||
```python
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# Config used by docker-compose.yml — routes to the Ollama service in the same
|
||||
# compose network. Send model="local" (pinned) or "cortiq-auto" (falls back to the
|
||||
# default model when no router is configured). Add a [router] key to enable smart routing.
|
||||
listen = "0.0.0.0:9000"
|
||||
|
||||
[router]
|
||||
url = "http://localhost:8080" # optional; pinned/default works without a router
|
||||
timeout_ms = 800
|
||||
|
||||
[[models]]
|
||||
id = "local"
|
||||
provider = "openai"
|
||||
base_url = "http://ollama:11434/v1" # the Ollama service (see docker-compose.yml)
|
||||
model = "qwen2.5:0.5b"
|
||||
cost_tier = "cheap"
|
||||
|
||||
[routing]
|
||||
default = "local"
|
||||
|
||||
[protocols]
|
||||
openai_chat = true
|
||||
|
||||
[admin]
|
||||
enabled = true
|
||||
|
||||
[telemetry]
|
||||
metrics = true
|
||||
+46
-55
@@ -14,11 +14,11 @@ listen = "0.0.0.0:9000"
|
||||
# NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the
|
||||
# hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default.
|
||||
[router]
|
||||
url = "http://localhost:8080"
|
||||
url = "https://138.226.222.209" # hosted allaigate router (or your own cortiq-router URL)
|
||||
api_key_env = "CORTIQ_ROUTER_KEY" # name of the env var holding the router key
|
||||
timeout_ms = 800 # timeout for the /v1/route call (12000+ for oracle)
|
||||
taxonomy_id = "general.task-type" # for allaigate: "data-assistant"
|
||||
verify_tls = true # false if the router uses an IP / self-signed cert
|
||||
timeout_ms = 12000 # hard prompts escalate to an oracle (~10 s)
|
||||
taxonomy_id = "data-assistant" # the hosted router's task taxonomy
|
||||
verify_tls = false # the hosted endpoint is IP-bound; true for your own domain
|
||||
|
||||
# --- how to pick the text to classify from the conversation -----------------
|
||||
[route]
|
||||
@@ -28,60 +28,29 @@ cache_ttl = "60s"
|
||||
profile = "balanced" # cost-saver | balanced | quality-first
|
||||
|
||||
# ============================================================================
|
||||
# Model pool: mix cheap local models and expensive cloud ones.
|
||||
# provider = "openai" covers OpenAI, OpenRouter, Together, Groq, and also
|
||||
# LOCAL servers vLLM / llama.cpp / LM Studio / Ollama (/v1).
|
||||
# Model pool — EMPTY by default. A fresh gateway starts with no models and an
|
||||
# empty pool; add them in the admin console (/admin → Models), or run a local
|
||||
# .cmf via the [cmf] section below. To add one by hand, copy this shape:
|
||||
#
|
||||
# [[models]]
|
||||
# id = "my-model"
|
||||
# provider = "openai" # openai | anthropic (openai also covers
|
||||
# base_url = "http://127.0.0.1:8081/v1" # OpenRouter/Together/Groq/vLLM/Ollama)
|
||||
# model = "my-model"
|
||||
# # api_key_env = "OPENAI_API_KEY" # for hosted providers (never inline a key)
|
||||
# cost_tier = "local"
|
||||
# caps = ["tools"]
|
||||
# ============================================================================
|
||||
|
||||
[[models]]
|
||||
id = "local-qwen"
|
||||
provider = "openai"
|
||||
base_url = "http://localhost:8000/v1"
|
||||
model = "qwen2.5-7b-instruct"
|
||||
cost_tier = "cheap"
|
||||
price_in = 0.0
|
||||
price_out = 0.0
|
||||
caps = ["tools"]
|
||||
|
||||
[[models]]
|
||||
id = "gpt-4o-mini"
|
||||
provider = "openai"
|
||||
base_url = "https://api.openai.com/v1"
|
||||
model = "gpt-4o-mini"
|
||||
api_key_env = "OPENAI_API_KEY"
|
||||
cost_tier = "mid"
|
||||
price_in = 0.15
|
||||
price_out = 0.60
|
||||
caps = ["tools", "vision"]
|
||||
|
||||
[[models]]
|
||||
id = "claude-opus"
|
||||
provider = "anthropic"
|
||||
base_url = "https://api.anthropic.com"
|
||||
model = "claude-opus-4"
|
||||
api_key_env = "ANTHROPIC_API_KEY"
|
||||
cost_tier = "expensive"
|
||||
price_in = 15.0
|
||||
price_out = 75.0
|
||||
caps = ["tools", "vision"]
|
||||
|
||||
# --- embedding model (for /v1/embeddings in provider mode) ------------------
|
||||
[[models]]
|
||||
id = "local-embed"
|
||||
provider = "openai"
|
||||
base_url = "http://localhost:8000/v1"
|
||||
model = "nomic-embed-text"
|
||||
cost_tier = "cheap"
|
||||
kind = "embedding"
|
||||
|
||||
# ============================================================================
|
||||
# Routing table: complexity tier → ordered pool (priority = fallback chain)
|
||||
# Routing table: complexity tier → ordered pool (priority = fallback chain).
|
||||
# Leave default empty for a fresh install; set it to a model id once you add one.
|
||||
# ============================================================================
|
||||
[routing]
|
||||
low = ["local-qwen"]
|
||||
medium = ["gpt-4o-mini", "local-qwen"]
|
||||
high = ["claude-opus", "gpt-4o-mini"]
|
||||
default = "gpt-4o-mini" # used when the router is unreachable
|
||||
low = []
|
||||
medium = []
|
||||
high = []
|
||||
default = "" # set to a model id once you add one
|
||||
|
||||
[routing.policy]
|
||||
mode = "fixed_table" # fixed_table | cost_aware
|
||||
@@ -102,7 +71,7 @@ openai_chat = true
|
||||
openai_completions = true
|
||||
openai_embeddings = true
|
||||
openai_models = true
|
||||
anthropic_messages = false
|
||||
anthropic_messages = true
|
||||
mcp = false
|
||||
native_passthrough = true
|
||||
|
||||
@@ -118,7 +87,7 @@ echo = true # add a "cortiq" field to the response body (X-Cortiq-* headers
|
||||
key = "sk-gw-team-a"
|
||||
account = "team-a"
|
||||
rate_per_min = 600
|
||||
# allow_models = ["cortiq-auto", "local-qwen"] # restrict available models (optional)
|
||||
# allow_models = ["cortiq-auto", "cmf-local"] # restrict available models (optional)
|
||||
|
||||
# --- idempotency and logging ------------------------------------------------
|
||||
[idempotency]
|
||||
@@ -150,6 +119,10 @@ enabled = true
|
||||
file = "config/stats.jsonl" # empty = in-memory only
|
||||
ring_size = 500 # how many recent requests to keep in the feed
|
||||
retention = "7d" # aggregate/series depth (replayed on startup)
|
||||
# stats.jsonl is compacted once it exceeds this size: old records fold into
|
||||
# cumulative per-account totals (sidecar file), so per-key accounting survives
|
||||
# while the disk stays bounded. 0 disables.
|
||||
max_file_mb = 16
|
||||
|
||||
# ============================================================================
|
||||
# Semantic cache — returns a cached answer for prompts semantically near a
|
||||
@@ -162,3 +135,21 @@ threshold = 0.92 # cosine similarity for a hit (0..1)
|
||||
ttl = "1h"
|
||||
max_entries = 1000
|
||||
# embed_model = "local-embed" # defaults to the first kind = "embedding" model
|
||||
|
||||
# ============================================================================
|
||||
# Local CMF model server. The gateway can install/run a local .cmf via cortiq
|
||||
# (github.com/infosave2007/cmf) and register it as the "cmf-local" model above,
|
||||
# so it works even when the router and cloud providers are unavailable.
|
||||
# Changes to manage_server / local_model / threads / gpu apply on restart.
|
||||
# ============================================================================
|
||||
[cmf]
|
||||
manage_server = false # true → gateway spawns `cortiq serve` at startup
|
||||
local_model = "models/model.cmf" # path to the .cmf to serve
|
||||
local_host = "127.0.0.1"
|
||||
local_port = 8081 # confirm it's free in Settings → "Check port" (8081 often clashes with Docker/Apache)
|
||||
model_id = "cmf-local" # id it registers under (matches [[models]])
|
||||
threads = 8 # decode worker threads (CMF_THREADS); ~3x vs serial
|
||||
gpu = false # offload matvecs to GPU (Metal/wgpu); mainly a prefill win
|
||||
cortiq_bin = "cortiq" # the cortiq CLI (cargo install cortiq-cli)
|
||||
auto_install = false # install cortiq-cli from crates.io if the binary is missing
|
||||
local_only = false # true → route every request to the local model
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# Quick start: Cortiq Gateway + a local model (Ollama).
|
||||
#
|
||||
# docker compose up -d
|
||||
# docker compose exec ollama ollama pull qwen2.5:0.5b # pull the model once
|
||||
# # → OpenAI-compatible API at http://localhost:9000/v1
|
||||
# # → admin console at http://localhost:9000/admin (token: change-me)
|
||||
#
|
||||
# For smart routing, add a [router] key to config/gateway.compose.toml
|
||||
# (or set CORTIQ_ROUTER_KEY and point [router] at allaigate — see docs/ROUTER.md).
|
||||
services:
|
||||
gateway:
|
||||
image: ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
ports:
|
||||
- "9000:9000"
|
||||
environment:
|
||||
CORTIQ_ADMIN_TOKEN: change-me
|
||||
CORTIQ_ROUTER_KEY: ${CORTIQ_ROUTER_KEY:-}
|
||||
volumes:
|
||||
- ./config/gateway.compose.toml:/app/config/gateway.toml:ro
|
||||
# downloaded .cmf models + stats persist here across recreates; swap the
|
||||
# named volume for a host folder (e.g. ./models:/app/data/models) if you
|
||||
# want the files visible next to the compose file
|
||||
- cortiq-data:/app/data
|
||||
depends_on:
|
||||
- ollama
|
||||
restart: unless-stopped
|
||||
|
||||
ollama:
|
||||
image: ollama/ollama
|
||||
ports:
|
||||
- "11434:11434"
|
||||
volumes:
|
||||
- ollama:/root/.ollama
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
cortiq-data:
|
||||
ollama:
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
# Deploying cortiq-gateway for a team / office
|
||||
|
||||
One gateway on a small server, API keys per department, local models for the
|
||||
cheap tiers, cloud models for the hard ones. This page covers the ops side.
|
||||
|
||||
## 1. Run as a service
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Task Scheduler → Create Task → "Run whether user is logged on or not",
|
||||
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
|
||||
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS and the outside world
|
||||
|
||||
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
|
||||
and bind the gateway to localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE streaming
|
||||
proxy_read_timeout 3600s; # long generations
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
|
||||
same with automatic certificates.)
|
||||
|
||||
## 3. Security checklist
|
||||
|
||||
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
|
||||
the console warns that it is open to the network.
|
||||
- **API keys**: create one per department/developer on the keys page. With at
|
||||
least one key configured the model APIs require Bearer / x-api-key —
|
||||
requests without a valid key get 401.
|
||||
- **Rate limits**: per-key requests/min → 429 with Retry-After.
|
||||
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
|
||||
account's month-to-date spend crosses it.
|
||||
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
|
||||
|
||||
## 4. Backups
|
||||
|
||||
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
|
||||
- `secrets.toml` — provider API keys (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
|
||||
cumulative per-key totals and month-to-date budget spend.
|
||||
|
||||
Copy those three and you can rebuild the instance anywhere.
|
||||
|
||||
## 5. Observability
|
||||
|
||||
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
|
||||
OpenRouter balance (red under $5).
|
||||
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
|
||||
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Развёртывание cortiq-gateway для команды / офиса
|
||||
|
||||
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
|
||||
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
|
||||
|
||||
## 1. Запуск как сервис
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
|
||||
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
|
||||
--no-port-search`. Либо [NSSM](https://nssm.cc):
|
||||
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS и внешний мир
|
||||
|
||||
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
|
||||
привяжите к localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE-стриминг
|
||||
proxy_read_timeout 3600s; # долгие генерации
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
|
||||
с автоматическими сертификатами.)
|
||||
|
||||
## 3. Чек-лист безопасности
|
||||
|
||||
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
|
||||
него консоль предупреждает, что открыта сети.
|
||||
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
|
||||
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
|
||||
запросы без валидного ключа получают 401.
|
||||
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
|
||||
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
|
||||
траты аккаунта за месяц превысят лимит.
|
||||
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
|
||||
|
||||
## 4. Бэкапы
|
||||
|
||||
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
|
||||
- `secrets.toml` — ключи провайдеров (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
|
||||
тоталы по ключам и месячные траты для бюджетов.
|
||||
|
||||
Скопируйте эти три файла — и инстанс поднимается где угодно.
|
||||
|
||||
## 5. Наблюдаемость
|
||||
|
||||
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
|
||||
сложности, баланс OpenRouter (красный ниже $5).
|
||||
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
|
||||
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
|
||||
+40
-9
@@ -53,7 +53,17 @@ Routing rule common to all protocols:
|
||||
```
|
||||
|
||||
Standard OpenAI fields are supported; unknown fields are proxied to the provider unchanged
|
||||
(forward-compatible).
|
||||
(forward-compatible), minus the bookkeeping keys chat UIs attach to the payload
|
||||
(`chat_id`, `session_id`, `metadata`, `features`, …), which are dropped so strict upstreams
|
||||
do not reject the call.
|
||||
|
||||
The parser is deliberately lenient about the shapes real clients send:
|
||||
`content` may be a string, `null`, or an array of content parts (text parts are
|
||||
concatenated, images and other non-text parts are skipped); `tools` may be `null`;
|
||||
`reasoning_effort` may be a string (`low` | `medium` | `high`) or a token budget;
|
||||
`max_completion_tokens` is accepted as an alias for `max_tokens`. A body that still
|
||||
cannot be read comes back as an OpenAI error envelope with HTTP 400 — never a bare 422,
|
||||
which OpenAI clients cannot parse (they show no answer at all).
|
||||
|
||||
### Response (non-streaming)
|
||||
|
||||
@@ -62,12 +72,12 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
|
||||
"id": "chatcmpl-9f...",
|
||||
"object": "chat.completion",
|
||||
"created": 1718800000,
|
||||
"model": "local-qwen", // the ACTUALLY selected model
|
||||
"model": "cortiq-auto", // the model the CLIENT asked for, echoed back
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": { "role": "assistant", "content": "x = 2 or x = 3" },
|
||||
"finish_reason": "stop"
|
||||
"finish_reason": "stop" // always one of stop|length|tool_calls|content_filter
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
@@ -86,10 +96,24 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
|
||||
}
|
||||
```
|
||||
|
||||
> `model` echoes the id the client requested, exactly as OpenAI does — clients match the
|
||||
> reply against the id they sent. The model that actually answered is in
|
||||
> `cortiq.selected_model` and in the `X-Cortiq-Selected-Model` header.
|
||||
|
||||
> Optional fields are omitted rather than sent as `null`: absent tool calls mean no
|
||||
> `tool_calls` key at all. `"tool_calls": null` is not in the OpenAI schema and trips
|
||||
> strict clients.
|
||||
|
||||
> The `cortiq` field is an optional extension. OpenAI SDKs ignore it. If the client does
|
||||
> not want extra fields, disable `cortiq.echo` — the metadata will remain in the response
|
||||
> headers.
|
||||
|
||||
Streaming (`stream: true`) emits strict `chat.completion.chunk` events: the stream always
|
||||
opens with a `role` delta, carries exactly one terminal `finish_reason`, and ends with
|
||||
`data: [DONE]` — even when the upstream stops early or dies mid-flight, in which case the
|
||||
failure arrives as an SSE `error` event. A client can therefore never be left waiting on a
|
||||
half-formed stream.
|
||||
|
||||
### Metadata in response headers (always present)
|
||||
|
||||
```
|
||||
@@ -164,20 +188,27 @@ single user message and follows the same pipeline. Response `object` is `"text_c
|
||||
## 4. OpenAI Models
|
||||
|
||||
`GET /v1/models` — auto-discovery for clients. Returns the gateway's virtual models
|
||||
(`cortiq-auto`, `cortiq-embed`) and all real models in the pool.
|
||||
(`cortiq-auto`, `cortiq-embed`), every model in the `[[models]]` pool, and every managed
|
||||
local `[[cmf.servers]]` model, so a client can pin a local `.cmf` directly.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
|
||||
{ "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
|
||||
{ "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
|
||||
{ "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
|
||||
"cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Managed CMF models are listed as soon as they are configured, with their live state in the
|
||||
non-standard `cortiq` field (clients ignore it). They are not filtered by health: a large
|
||||
`.cmf` takes minutes to load, and a model that disappears from the list while it warms up
|
||||
is worse than one marked "not ready yet".
|
||||
|
||||
---
|
||||
|
||||
## 5. Anthropic Messages
|
||||
@@ -198,7 +229,7 @@ single user message and follows the same pipeline. Response `object` is `"text_c
|
||||
"id": "msg_...",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "local-qwen",
|
||||
"model": "cortiq-auto",
|
||||
"content": [ { "type": "text", "text": "x = 2 or x = 3" } ],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": { "input_tokens": 24, "output_tokens": 11 }
|
||||
|
||||
+38
-8
@@ -53,7 +53,17 @@
|
||||
```
|
||||
|
||||
Поддерживаются стандартные поля OpenAI; неизвестные поля проксируются к провайдеру
|
||||
без изменений (forward-compatible).
|
||||
без изменений (forward-compatible) — кроме служебных ключей, которые чат-интерфейсы
|
||||
добавляют в payload для собственных нужд (`chat_id`, `session_id`, `metadata`,
|
||||
`features`, …): они отбрасываются, чтобы строгий upstream не отклонил вызов.
|
||||
|
||||
Парсер намеренно терпим к тем формам, которые реально шлют клиенты: `content` может быть
|
||||
строкой, `null` или массивом content-частей (текстовые части склеиваются, изображения и
|
||||
прочие нетекстовые пропускаются); `tools` может быть `null`; `reasoning_effort` — строкой
|
||||
(`low` | `medium` | `high`) или бюджетом токенов; `max_completion_tokens` принимается как
|
||||
синоним `max_tokens`. Если тело всё-таки не читается, возвращается OpenAI-конверт ошибки с
|
||||
HTTP 400 — но не «голый» 422, который OpenAI-клиенты не умеют разбирать (для пользователя
|
||||
это выглядит как отсутствие ответа).
|
||||
|
||||
### Ответ (без стриминга)
|
||||
|
||||
@@ -62,12 +72,12 @@
|
||||
"id": "chatcmpl-9f...",
|
||||
"object": "chat.completion",
|
||||
"created": 1718800000,
|
||||
"model": "local-qwen", // РЕАЛЬНО выбранная модель
|
||||
"model": "cortiq-auto", // модель, которую ЗАПРОСИЛ клиент — возвращается как есть
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": { "role": "assistant", "content": "x = 2 или x = 3" },
|
||||
"finish_reason": "stop"
|
||||
"finish_reason": "stop" // всегда одно из stop|length|tool_calls|content_filter
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
@@ -86,9 +96,22 @@
|
||||
}
|
||||
```
|
||||
|
||||
> `model` возвращает тот id, который запросил клиент, — ровно как это делает OpenAI:
|
||||
> клиенты сопоставляют ответ с отправленным id. Модель, которая реально ответила, — в
|
||||
> `cortiq.selected_model` и в заголовке `X-Cortiq-Selected-Model`.
|
||||
|
||||
> Необязательные поля не сериализуются вместо того, чтобы приходить как `null`: если
|
||||
> tool-вызовов нет, ключа `tool_calls` в ответе нет вообще. `"tool_calls": null` не входит
|
||||
> в схему OpenAI и ломает строгие клиенты.
|
||||
|
||||
> Поле `cortiq` — необязательное расширение. OpenAI-SDK его игнорируют. Если клиент
|
||||
> не хочет «лишних» полей — выключите `cortiq.echo`, метаданные останутся в заголовках.
|
||||
|
||||
Стриминг (`stream: true`) отдаёт строгие события `chat.completion.chunk`: поток всегда
|
||||
открывается дельтой с `role`, содержит ровно один финальный `finish_reason` и завершается
|
||||
`data: [DONE]` — даже если upstream оборвался или упал на середине (тогда ошибка приходит
|
||||
SSE-событием `error`). Клиент никогда не остаётся ждать недосказанный поток.
|
||||
|
||||
### Метаданные в заголовках (всегда)
|
||||
|
||||
```
|
||||
@@ -163,20 +186,27 @@ data: [DONE]
|
||||
## 4. OpenAI Models
|
||||
|
||||
`GET /v1/models` — автодискавери для клиентов. Возвращает виртуальные модели шлюза
|
||||
(`cortiq-auto`, `cortiq-embed`) и все реальные модели пула.
|
||||
(`cortiq-auto`, `cortiq-embed`), все модели пула `[[models]]` и все управляемые локальные
|
||||
модели `[[cmf.servers]]` — чтобы клиент мог напрямую закрепить локальный `.cmf`.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
|
||||
{ "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
|
||||
{ "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
|
||||
{ "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
|
||||
"cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Управляемые CMF-модели попадают в список сразу, как только настроены; их живое состояние —
|
||||
в нестандартном поле `cortiq` (клиенты его игнорируют). По здоровью они не фильтруются:
|
||||
крупный `.cmf` грузится минутами, и модель, исчезающая из списка на время прогрева, хуже
|
||||
модели, помеченной «ещё не готова».
|
||||
|
||||
---
|
||||
|
||||
## 5. Anthropic Messages
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ timeout_ms = 15000 # allow for oracle escalation on hard
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <path> only if you keep the config elsewhere
|
||||
```
|
||||
|
||||
## API contract
|
||||
|
||||
+1
-1
@@ -49,7 +49,7 @@ timeout_ms = 15000 # запас под oracle-эскал
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <путь> нужен, только если конфиг лежит в другом месте
|
||||
```
|
||||
|
||||
## Контракт API
|
||||
|
||||
@@ -162,3 +162,44 @@ fine-tune the task mask. Over time this makes routing fit your specific pool mor
|
||||
enabled = false
|
||||
# the ground-truth source is set by the integrator (e.g. the X-Cortiq-Correct-Label header)
|
||||
```
|
||||
|
||||
## Task-based routing
|
||||
|
||||
The router classifies every request twice: a **task label** (what kind of work it
|
||||
is) and a **complexity tier** (how hard it is). Tiers pick *how much model* you
|
||||
need; task rules pick *which* model is best at the job:
|
||||
|
||||
```toml
|
||||
[routing.tasks]
|
||||
translation = ["local-qwen"] # translations are cheap — keep them local
|
||||
code = ["claude-sonnet", "gpt"] # code goes to the strongest coder first
|
||||
```
|
||||
|
||||
A matching rule's models go **first** in the candidate chain; the complexity
|
||||
chain follows as fallback, so a task pin never weakens failover. Tasks without
|
||||
a rule route by complexity alone.
|
||||
|
||||
**Per-complexity models for a task.** A task rule also accepts a per-tier
|
||||
map — "which model serves this task at each complexity":
|
||||
|
||||
```toml
|
||||
[routing.tasks.code]
|
||||
low = ["cheap-coder"] # routine code stays cheap
|
||||
high = ["best-coder"] # hard code goes to the strongest coder
|
||||
any = ["generic-coder"] # leads at every tier after the tier-specific pick
|
||||
```
|
||||
|
||||
The tier-specific list leads its own tier; `any` (and the legacy plain-array
|
||||
form) leads at every tier; the global tier chain and `default` always remain
|
||||
as fallback, so a task rule never weakens failover. The admin console's
|
||||
Routing page edits this as rows of “model + complexity” pairs per task.
|
||||
|
||||
**Exception — high complexity escalates.** When the router rates a request
|
||||
`high` (and a `high` chain is configured), the tier chain leads and the task
|
||||
rule demotes to fallback. Otherwise a rule like `code → cheap-model` would pin
|
||||
even the hardest prompts to the cheap model and the tier system would never
|
||||
get a say. Low/medium keep task-rule-first semantics. The available labels come from the router
|
||||
(`GET /v1/taxonomies`) — the admin console's Routing page builds the editor
|
||||
from your account's live taxonomy and shows each task's share of recent
|
||||
traffic. When a rule supplied the head of the chain the response reports
|
||||
`route_source: "router+task-rule"`.
|
||||
|
||||
@@ -160,3 +160,43 @@ cooldown = "30s"
|
||||
enabled = false
|
||||
# источник истины задаёт интегратор (например, заголовок X-Cortiq-Correct-Label)
|
||||
```
|
||||
|
||||
## Распределение по задачам
|
||||
|
||||
Роутер классифицирует каждый запрос дважды: **тип задачи** (что это за работа)
|
||||
и **уровень сложности** (насколько она трудна). Уровни выбирают, *сколько
|
||||
модели* нужно; правила по задачам — *какая* модель лучше в этом деле:
|
||||
|
||||
```toml
|
||||
[routing.tasks]
|
||||
translation = ["local-qwen"] # переводы дёшевы — оставляем локально
|
||||
code = ["claude-sonnet", "gpt"] # код — сильнейшему кодеру
|
||||
```
|
||||
|
||||
Модели подходящего правила идут **первыми** в цепочке кандидатов; цепочка по
|
||||
сложности следует за ними как резерв — правило по задаче не ослабляет failover.
|
||||
Задачи без правила маршрутизируются только по сложности.
|
||||
|
||||
**Модели по сложности внутри задачи.** Правило задачи принимает и карту по
|
||||
уровням — «какая модель обслуживает задачу на какой сложности»:
|
||||
|
||||
```toml
|
||||
[routing.tasks.code]
|
||||
low = ["cheap-coder"] # рутинный код — дёшево
|
||||
high = ["best-coder"] # сложный код — сильнейшему кодеру
|
||||
any = ["generic-coder"] # ведёт на всех уровнях после уровневого выбора
|
||||
```
|
||||
|
||||
Список конкретного уровня ведёт на своём уровне; `any` (и старая форма-массив)
|
||||
ведёт на всех; глобальная цепочка уровня и `default` всегда остаются резервом —
|
||||
правило задачи не ослабляет failover. На вкладке Роутинг это редактируется
|
||||
строками пар «модель + сложность» у каждой задачи.
|
||||
|
||||
**Исключение — высокая сложность эскалирует.** Когда роутер оценил запрос как
|
||||
`high` (и цепочка `high` настроена), цепочка уровня идёт первой, а правило
|
||||
задачи становится резервом. Иначе правило вида `code → дешёвая-модель`
|
||||
приковало бы к дешёвой модели даже самые сложные запросы, и система уровней
|
||||
не имела бы права голоса. Для low/medium правило задачи ведёт, как и раньше. Список меток приходит
|
||||
с роутера (`GET /v1/taxonomies`) — вкладка Роутинг строит редактор по живой
|
||||
таксономии аккаунта и показывает долю каждой задачи в трафике. Когда цепочку
|
||||
возглавило правило задачи, ответ сообщает `route_source: "router+task-rule"`.
|
||||
|
||||
+12
-2
@@ -16,11 +16,18 @@ use rust_embed::RustEmbed;
|
||||
struct WebAssets;
|
||||
|
||||
fn serve_file(path: &str) -> Response {
|
||||
// The panel is rebuilt often (assets are embedded at compile time and served
|
||||
// from memory), so tell the browser to always revalidate — otherwise a stale
|
||||
// cached SPA hides freshly shipped fixes until a hard reload.
|
||||
const NO_CACHE: &str = "no-store, must-revalidate";
|
||||
match WebAssets::get(path) {
|
||||
Some(content) => {
|
||||
let mime = mime_guess::from_path(path).first_or_octet_stream();
|
||||
(
|
||||
[(header::CONTENT_TYPE, mime.as_ref())],
|
||||
[
|
||||
(header::CONTENT_TYPE, mime.as_ref()),
|
||||
(header::CACHE_CONTROL, NO_CACHE),
|
||||
],
|
||||
content.data.into_owned(),
|
||||
)
|
||||
.into_response()
|
||||
@@ -29,7 +36,10 @@ fn serve_file(path: &str) -> Response {
|
||||
// SPA: unknown path under /admin → serve index.html
|
||||
match WebAssets::get("index.html") {
|
||||
Some(content) => (
|
||||
[(header::CONTENT_TYPE, "text/html; charset=utf-8")],
|
||||
[
|
||||
(header::CONTENT_TYPE, "text/html; charset=utf-8"),
|
||||
(header::CACHE_CONTROL, NO_CACHE),
|
||||
],
|
||||
content.data.into_owned(),
|
||||
)
|
||||
.into_response(),
|
||||
|
||||
+1784
-29
File diff suppressed because it is too large
Load Diff
+225
@@ -0,0 +1,225 @@
|
||||
//! One-click local model benchmark: run `cortiq bench <model> --json` and keep
|
||||
//! the parsed result per model, so the Models page can show real tok/s numbers
|
||||
//! measured on this machine.
|
||||
|
||||
use serde::Serialize;
|
||||
use std::collections::HashMap;
|
||||
use std::process::Stdio;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[derive(Clone, Serialize)]
|
||||
pub struct BenchRun {
|
||||
pub model: String, // file stem under models_dir
|
||||
pub state: String, // running | done | error
|
||||
pub log: Vec<String>,
|
||||
pub tok_s: Option<f64>,
|
||||
/// The full JSON object the benchmark printed, passed through untouched.
|
||||
pub raw: Option<serde_json::Value>,
|
||||
pub started: u64,
|
||||
pub finished: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct BenchStore {
|
||||
runs: Mutex<HashMap<String, BenchRun>>,
|
||||
busy: Mutex<bool>,
|
||||
}
|
||||
|
||||
fn now() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// tok/s out of the benchmark's JSON: known field names first (steady decode
|
||||
/// speed is the honest number), then any `*tok_s*` field as a fallback. The
|
||||
/// old loose "contains tok + per" heuristic could grab `allocs_per_token`.
|
||||
fn extract_tok_s(v: &serde_json::Value) -> Option<f64> {
|
||||
let obj = v.as_object()?;
|
||||
for want in [
|
||||
"decode_tok_s_steady",
|
||||
"decode_tok_s_incl_prefill",
|
||||
"decode_tok_s",
|
||||
"tok_s",
|
||||
"tok_per_s",
|
||||
"tokens_per_sec",
|
||||
] {
|
||||
if let Some(n) = obj.get(want).and_then(|x| x.as_f64()) {
|
||||
return Some(n);
|
||||
}
|
||||
}
|
||||
obj.iter()
|
||||
.find(|(k, val)| {
|
||||
let k = k.to_ascii_lowercase();
|
||||
(k.contains("tok_s") || k.contains("tokens_per")) && val.as_f64().is_some()
|
||||
})
|
||||
.and_then(|(_, val)| val.as_f64())
|
||||
}
|
||||
|
||||
/// Last parseable JSON object in mixed CLI output. Newer cortiq releases
|
||||
/// pretty-print the `--json` result over many lines, older ones emit a single
|
||||
/// line — this handles both, skipping any log noise around it.
|
||||
fn last_json_object(text: &str) -> Option<serde_json::Value> {
|
||||
let mut end = text.len();
|
||||
while let Some(close) = text[..end].rfind('}') {
|
||||
let mut depth = 0i32;
|
||||
let mut start = None;
|
||||
for (i, c) in text[..=close].char_indices().rev() {
|
||||
match c {
|
||||
'}' => depth += 1,
|
||||
'{' => {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
start = Some(i);
|
||||
break;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if let Some(st) = start {
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(&text[st..=close]) {
|
||||
if v.is_object() {
|
||||
return Some(v);
|
||||
}
|
||||
}
|
||||
}
|
||||
end = close;
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
impl BenchStore {
|
||||
pub fn new() -> Arc<Self> {
|
||||
Arc::new(Self::default())
|
||||
}
|
||||
pub fn get(&self, model: &str) -> Option<BenchRun> {
|
||||
self.runs.lock().unwrap().get(model).cloned()
|
||||
}
|
||||
pub fn list(&self) -> Vec<BenchRun> {
|
||||
self.runs.lock().unwrap().values().cloned().collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// Kick off a benchmark for `<models_dir>/<model>.cmf`. One at a time — a
|
||||
/// benchmark loads the whole model and owns the machine while it runs.
|
||||
pub fn start(
|
||||
store: Arc<BenchStore>,
|
||||
cmf: &crate::config::CmfCfg,
|
||||
model: String,
|
||||
) -> Result<(), String> {
|
||||
let path = std::path::Path::new(&cmf.models_dir).join(format!("{model}.cmf"));
|
||||
if !path.exists() {
|
||||
return Err(format!("model file not found: {}", path.display()));
|
||||
}
|
||||
{
|
||||
let mut busy = store.busy.lock().unwrap();
|
||||
if *busy {
|
||||
return Err("a benchmark is already running — wait for it to finish".into());
|
||||
}
|
||||
*busy = true;
|
||||
}
|
||||
store.runs.lock().unwrap().insert(
|
||||
model.clone(),
|
||||
BenchRun {
|
||||
model: model.clone(),
|
||||
state: "running".into(),
|
||||
log: Vec::new(),
|
||||
tok_s: None,
|
||||
raw: None,
|
||||
started: now(),
|
||||
finished: None,
|
||||
},
|
||||
);
|
||||
let bin = cmf.cortiq_bin.clone();
|
||||
let gpu = cmf.gpu;
|
||||
tokio::spawn(async move {
|
||||
let out = tokio::process::Command::new(&bin)
|
||||
.arg("bench")
|
||||
.arg(&path)
|
||||
.arg("--json")
|
||||
.env("CMF_GPU", if gpu { "1" } else { "0" })
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.output()
|
||||
.await;
|
||||
let mut g = store.runs.lock().unwrap();
|
||||
let run = g.get_mut(&model).unwrap();
|
||||
run.finished = Some(now());
|
||||
match out {
|
||||
Ok(o) => {
|
||||
let text = format!(
|
||||
"{}\n{}",
|
||||
String::from_utf8_lossy(&o.stdout),
|
||||
String::from_utf8_lossy(&o.stderr)
|
||||
);
|
||||
// keep a readable tail; the JSON line is parsed separately
|
||||
run.log = text
|
||||
.lines()
|
||||
.map(|l| l.trim())
|
||||
.filter(|l| !l.is_empty() && !l.contains("Metal GPU path"))
|
||||
.map(String::from)
|
||||
.collect::<Vec<_>>();
|
||||
let n = run.log.len();
|
||||
if n > 30 {
|
||||
run.log.drain(0..n - 30);
|
||||
}
|
||||
let parsed = last_json_object(&text);
|
||||
match parsed {
|
||||
Some(v) if o.status.success() => {
|
||||
run.tok_s = extract_tok_s(&v);
|
||||
run.raw = Some(v);
|
||||
run.state = "done".into();
|
||||
}
|
||||
_ => run.state = "error".into(),
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
run.log.push(format!("✗ failed to start cortiq bench: {e}"));
|
||||
run.state = "error".into();
|
||||
}
|
||||
}
|
||||
drop(g);
|
||||
*store.busy.lock().unwrap() = false;
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn tok_s_from_common_keys() {
|
||||
let v = serde_json::json!({"tok_s": 42.5, "allocs_per_token": 0});
|
||||
assert_eq!(extract_tok_s(&v), Some(42.5));
|
||||
let v = serde_json::json!({"decode_tokens_per_sec": 31.0});
|
||||
assert_eq!(extract_tok_s(&v), Some(31.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn steady_decode_wins_and_allocs_never_matches() {
|
||||
let v = serde_json::json!({
|
||||
"allocs_per_token": 769.4,
|
||||
"decode_tok_s_incl_prefill": 9.98,
|
||||
"decode_tok_s_steady": 11.86,
|
||||
"prefill_tok_s": 25.5,
|
||||
});
|
||||
assert_eq!(extract_tok_s(&v), Some(11.86));
|
||||
let v = serde_json::json!({"allocs_per_token": 769.4, "seq_len": 140});
|
||||
assert_eq!(extract_tok_s(&v), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_found_single_line_and_pretty() {
|
||||
let single = "noise\n{\"tok_s\": 5.0}\nmore noise";
|
||||
assert_eq!(last_json_object(single).unwrap()["tok_s"], 5.0);
|
||||
let pretty = "INFO loading\n{\n \"decode_tok_s_steady\": 11.86,\n \"seq_len\": 140\n}\nINFO done {not json";
|
||||
assert_eq!(
|
||||
last_json_object(pretty).unwrap()["decode_tok_s_steady"],
|
||||
11.86
|
||||
);
|
||||
assert!(last_json_object("no braces here").is_none());
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -55,7 +55,7 @@ impl SemanticCache {
|
||||
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
enabled: cfg.enabled,
|
||||
threshold: cfg.threshold,
|
||||
threshold: cfg.threshold as f32,
|
||||
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
|
||||
max_entries: cfg.max_entries.max(1),
|
||||
embed_model: cfg.embed_model.clone(),
|
||||
|
||||
@@ -0,0 +1,467 @@
|
||||
//! Local CMF model server lifecycle.
|
||||
//!
|
||||
//! Closes the loop with <https://github.com/infosave2007/cmf>: install the
|
||||
//! `cortiq` CLI from crates.io, keep it up to date, and run one `cortiq serve`
|
||||
//! per configured local model as a managed child process. Each served model
|
||||
//! exposes an OpenAI-compatible API, so the registry registers it as an ordinary
|
||||
//! provider and the gateway routes to it like any other backend.
|
||||
|
||||
use crate::config::{CmfCfg, CmfServer};
|
||||
use std::process::Stdio;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
use tokio::process::{Child, Command};
|
||||
|
||||
/// Observable state of one managed local model server.
|
||||
#[derive(Clone, Debug, Default, serde::Serialize)]
|
||||
pub struct CmfServerStatus {
|
||||
/// Pool id it is registered under.
|
||||
pub id: String,
|
||||
/// Path of the `.cmf` being served.
|
||||
pub model: String,
|
||||
/// Port the server binds to.
|
||||
pub port: u16,
|
||||
/// OpenAI base URL the local server is registered under.
|
||||
pub base_url: String,
|
||||
/// The child `cortiq serve` process has been spawned.
|
||||
pub running: bool,
|
||||
/// `/healthz` on the local server answered 200.
|
||||
pub healthy: bool,
|
||||
/// Last fatal error for this server, if any.
|
||||
pub last_error: Option<String>,
|
||||
}
|
||||
|
||||
/// Observable state of the managed local CMF layer (surfaced to the admin API).
|
||||
#[derive(Clone, Debug, Default, serde::Serialize)]
|
||||
pub struct CmfStatus {
|
||||
/// `manage_server` is on and at least one local model is configured.
|
||||
pub enabled: bool,
|
||||
/// Version reported by `cortiq --version` (None = not installed).
|
||||
pub installed_version: Option<String>,
|
||||
/// Newest `cortiq-cli` on crates.io (populated when `auto_update` runs).
|
||||
pub latest_version: Option<String>,
|
||||
/// Per-model server status.
|
||||
pub servers: Vec<CmfServerStatus>,
|
||||
/// Recent lifecycle log lines (bounded).
|
||||
pub log: Vec<String>,
|
||||
}
|
||||
|
||||
/// A spawned server: its pool id and the child process handle.
|
||||
struct Managed {
|
||||
id: String,
|
||||
child: Child,
|
||||
}
|
||||
|
||||
/// Holds the managed child processes and their live status.
|
||||
pub struct CmfRuntime {
|
||||
status: Mutex<CmfStatus>,
|
||||
children: Mutex<Vec<Managed>>,
|
||||
/// Server ids whose healthz handshake advertised tool support — lets the
|
||||
/// pipeline route tools traffic to local models with no manual caps.
|
||||
pub detected_tools: Mutex<std::collections::HashSet<String>>,
|
||||
}
|
||||
|
||||
impl CmfRuntime {
|
||||
pub fn new() -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
})
|
||||
}
|
||||
|
||||
pub fn status(&self) -> CmfStatus {
|
||||
self.status.lock().unwrap().clone()
|
||||
}
|
||||
|
||||
fn log(&self, msg: impl Into<String>) {
|
||||
let msg = msg.into();
|
||||
tracing::info!(target: "cmf", "{msg}");
|
||||
let mut s = self.status.lock().unwrap();
|
||||
s.log.push(msg);
|
||||
let n = s.log.len();
|
||||
if n > 80 {
|
||||
s.log.drain(0..n - 80);
|
||||
}
|
||||
}
|
||||
|
||||
/// A global (install/update) failure — not tied to a single server.
|
||||
fn fail(&self, msg: impl Into<String>) {
|
||||
let msg = msg.into();
|
||||
tracing::warn!(target: "cmf", "{msg}");
|
||||
self.status.lock().unwrap().log.push(format!("✗ {msg}"));
|
||||
}
|
||||
|
||||
/// Record a fatal error against a specific server (and the log).
|
||||
fn fail_server(&self, id: &str, msg: impl Into<String>) {
|
||||
let msg = msg.into();
|
||||
tracing::warn!(target: "cmf", "[{id}] {msg}");
|
||||
let mut s = self.status.lock().unwrap();
|
||||
if let Some(sv) = s.servers.iter_mut().find(|x| x.id == id) {
|
||||
sv.last_error = Some(msg.clone());
|
||||
}
|
||||
s.log.push(format!("✗ [{id}] {msg}"));
|
||||
}
|
||||
|
||||
/// Seed the per-server status slots from the effective server list.
|
||||
fn init_servers(&self, servers: &[CmfServer], host: &str) {
|
||||
let mut s = self.status.lock().unwrap();
|
||||
s.enabled = true;
|
||||
s.servers = servers
|
||||
.iter()
|
||||
.map(|sv| CmfServerStatus {
|
||||
id: sv.id.clone(),
|
||||
model: sv.model.clone(),
|
||||
port: sv.port,
|
||||
base_url: format!("http://{host}:{}/v1", sv.port),
|
||||
running: false,
|
||||
healthy: false,
|
||||
last_error: None,
|
||||
})
|
||||
.collect();
|
||||
}
|
||||
|
||||
fn set_running(&self, id: &str, running: bool) {
|
||||
if let Some(sv) = self
|
||||
.status
|
||||
.lock()
|
||||
.unwrap()
|
||||
.servers
|
||||
.iter_mut()
|
||||
.find(|x| x.id == id)
|
||||
{
|
||||
sv.running = running;
|
||||
}
|
||||
}
|
||||
|
||||
fn set_healthy(&self, id: &str, healthy: bool) {
|
||||
if let Some(sv) = self
|
||||
.status
|
||||
.lock()
|
||||
.unwrap()
|
||||
.servers
|
||||
.iter_mut()
|
||||
.find(|x| x.id == id)
|
||||
{
|
||||
sv.healthy = healthy;
|
||||
}
|
||||
}
|
||||
|
||||
fn add_child(&self, id: String, child: Child) {
|
||||
self.children.lock().unwrap().push(Managed {
|
||||
id: id.clone(),
|
||||
child,
|
||||
});
|
||||
self.set_running(&id, true);
|
||||
}
|
||||
|
||||
fn is_running(&self, id: &str) -> bool {
|
||||
self.status
|
||||
.lock()
|
||||
.unwrap()
|
||||
.servers
|
||||
.iter()
|
||||
.any(|s| s.id == id && s.running)
|
||||
}
|
||||
|
||||
/// Stop all managed servers (used on shutdown / reconfigure).
|
||||
pub async fn stop(&self) {
|
||||
let mut kids = std::mem::take(&mut *self.children.lock().unwrap());
|
||||
for m in kids.iter_mut() {
|
||||
let _ = m.child.start_kill();
|
||||
}
|
||||
let mut s = self.status.lock().unwrap();
|
||||
for sv in s.servers.iter_mut() {
|
||||
sv.running = false;
|
||||
sv.healthy = false;
|
||||
}
|
||||
}
|
||||
|
||||
/// Stop and forget a single managed server (used when a model is deleted).
|
||||
pub async fn stop_one(&self, id: &str) {
|
||||
let removed = {
|
||||
let mut kids = self.children.lock().unwrap();
|
||||
kids.iter()
|
||||
.position(|m| m.id == id)
|
||||
.map(|pos| kids.remove(pos))
|
||||
};
|
||||
if let Some(mut m) = removed {
|
||||
let _ = m.child.start_kill();
|
||||
}
|
||||
self.status.lock().unwrap().servers.retain(|sv| sv.id != id);
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for CmfRuntime {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// crates.io `newest_version` for any crate.
|
||||
async fn latest_version_of(krate: &str) -> Option<String> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(Duration::from_secs(10))
|
||||
.build()
|
||||
.ok()?;
|
||||
let body = client
|
||||
.get(format!("https://crates.io/api/v1/crates/{krate}"))
|
||||
.header("User-Agent", "cortiq-gateway")
|
||||
.send()
|
||||
.await
|
||||
.ok()?
|
||||
.text()
|
||||
.await
|
||||
.ok()?;
|
||||
let v: serde_json::Value = serde_json::from_str(&body).ok()?;
|
||||
v["crate"]["newest_version"].as_str().map(|s| s.to_string())
|
||||
}
|
||||
|
||||
/// crates.io `newest_version` for `cortiq-cli`.
|
||||
async fn latest_crates_version() -> Option<String> {
|
||||
latest_version_of("cortiq-cli").await
|
||||
}
|
||||
|
||||
/// Newest published gateway version — the "update available" footer check.
|
||||
pub(crate) async fn latest_gateway_version() -> Option<String> {
|
||||
latest_version_of("cortiq-gateway").await
|
||||
}
|
||||
|
||||
/// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2").
|
||||
pub(crate) fn installed_version(bin: &str) -> Option<String> {
|
||||
let out = std::process::Command::new(bin)
|
||||
.arg("--version")
|
||||
.output()
|
||||
.ok()?;
|
||||
if !out.status.success() {
|
||||
return None;
|
||||
}
|
||||
let s = String::from_utf8_lossy(&out.stdout);
|
||||
s.split_whitespace().last().map(|s| s.trim().to_string())
|
||||
}
|
||||
|
||||
fn parse_ver(s: &str) -> (u32, u32, u32) {
|
||||
let mut it = s.split('.').map(|x| x.trim().parse::<u32>().unwrap_or(0));
|
||||
(
|
||||
it.next().unwrap_or(0),
|
||||
it.next().unwrap_or(0),
|
||||
it.next().unwrap_or(0),
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn version_lt(a: &str, b: &str) -> bool {
|
||||
parse_ver(a) < parse_ver(b)
|
||||
}
|
||||
|
||||
/// `cargo install cortiq-cli [--force]` — blocking compile, run off-thread.
|
||||
async fn cargo_install(force: bool) -> Result<(), String> {
|
||||
let mut cmd = Command::new("cargo");
|
||||
cmd.args(["install", "cortiq-cli", "--locked"]);
|
||||
if force {
|
||||
cmd.arg("--force");
|
||||
}
|
||||
let out = cmd.output().await.map_err(|e| e.to_string())?;
|
||||
if out.status.success() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(String::from_utf8_lossy(&out.stderr).trim().to_string())
|
||||
}
|
||||
}
|
||||
|
||||
/// Install/upgrade `cortiq-cli` from crates.io on demand (the admin "Install
|
||||
/// cortiq" button). Runs in the background; progress + result land in the CMF
|
||||
/// status log, and `installed_version` is refreshed on success.
|
||||
pub async fn install_now(rt: Arc<CmfRuntime>, bin: String) {
|
||||
rt.log("installing cortiq-cli from crates.io (this compiles — a few minutes)…");
|
||||
match cargo_install(true).await {
|
||||
Ok(()) => {
|
||||
let v = installed_version(&bin);
|
||||
rt.status.lock().unwrap().installed_version = v.clone();
|
||||
rt.log(format!(
|
||||
"installed cortiq-cli {}",
|
||||
v.as_deref().unwrap_or("?")
|
||||
));
|
||||
}
|
||||
Err(e) => rt.fail(format!("cargo install cortiq-cli failed: {e}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Spawn `cortiq serve <model> --host <host> --port <port>` for one server; the
|
||||
/// child is killed when its handle is dropped (i.e. when the gateway shuts down).
|
||||
fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
let mut cmd = Command::new(&cfg.cortiq_bin);
|
||||
cmd.arg("serve")
|
||||
.arg(&server.model)
|
||||
.arg("--host")
|
||||
.arg(&cfg.local_host)
|
||||
.arg("--port")
|
||||
.arg(server.port.to_string());
|
||||
// Parallelise decode matvecs across cores (CMF_THREADS). 0/1 = leave the
|
||||
// runtime's own default; N>1 gave ~2.8x on a 15B CPU model.
|
||||
if server.threads > 1 {
|
||||
cmd.env("CMF_THREADS", server.threads.to_string());
|
||||
}
|
||||
if server.gpu {
|
||||
cmd.env("CMF_GPU", "1");
|
||||
}
|
||||
// O(1) Nyström attention: layers spec + sub-parameters.
|
||||
if let Some(o1) = &server.o1 {
|
||||
let s = o1.trim();
|
||||
if !s.is_empty() {
|
||||
// Pass even "off" so it explicitly overrides a file converter hint.
|
||||
cmd.arg("--o1").arg(s);
|
||||
}
|
||||
}
|
||||
if let Some(m) = server.o1_m {
|
||||
cmd.arg("--o1-m").arg(m.to_string());
|
||||
}
|
||||
if let Some(w) = server.o1_window {
|
||||
cmd.arg("--o1-window").arg(w.to_string());
|
||||
}
|
||||
if let Some(sink) = server.o1_sink {
|
||||
cmd.arg("--o1-sink").arg(sink.to_string());
|
||||
}
|
||||
// MTP (Multi-Token Prediction) speculative decoding: the engine reads
|
||||
// CMF_MTP env; "0" disables. There is no CLI flag for this.
|
||||
if server.skip_mtp {
|
||||
cmd.env("CMF_MTP", "0");
|
||||
}
|
||||
cmd.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.kill_on_drop(true)
|
||||
.spawn()
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline
|
||||
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
|
||||
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
|
||||
/// caller can register them; older servers return `None` there.
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
|
||||
let url = format!("http://{host}:{port}/healthz");
|
||||
let client = reqwest::Client::new();
|
||||
for _ in 0..tries {
|
||||
if let Ok(r) = client
|
||||
.get(&url)
|
||||
.timeout(Duration::from_secs(2))
|
||||
.send()
|
||||
.await
|
||||
{
|
||||
if r.status().is_success() {
|
||||
let tools = r
|
||||
.json::<serde_json::Value>()
|
||||
.await
|
||||
.ok()
|
||||
.and_then(|v| v["capabilities"]["tools"].as_bool());
|
||||
return (true, tools);
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
(false, None)
|
||||
}
|
||||
|
||||
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
||||
/// configured local model → wait healthy. Safe to call in a background task; it
|
||||
/// never panics.
|
||||
pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
|
||||
let servers = cfg.effective_servers();
|
||||
if servers.is_empty() {
|
||||
return;
|
||||
}
|
||||
rt.init_servers(&servers, &cfg.local_host);
|
||||
|
||||
// 1. Ensure the `cortiq` binary is available (once for all servers).
|
||||
let mut ver = installed_version(&cfg.cortiq_bin);
|
||||
if ver.is_none() {
|
||||
if cfg.auto_install {
|
||||
rt.log("cortiq not found — installing cortiq-cli from crates.io (this compiles, may take a few minutes)…");
|
||||
if let Err(e) = cargo_install(false).await {
|
||||
rt.fail(format!("cargo install cortiq-cli failed: {e}"));
|
||||
return;
|
||||
}
|
||||
ver = installed_version(&cfg.cortiq_bin);
|
||||
if ver.is_some() {
|
||||
rt.log("installed cortiq-cli");
|
||||
}
|
||||
} else {
|
||||
rt.fail(
|
||||
"cortiq binary not found (set [cmf].auto_install = true, or install cortiq-cli)",
|
||||
);
|
||||
return;
|
||||
}
|
||||
}
|
||||
rt.status.lock().unwrap().installed_version = ver.clone();
|
||||
|
||||
// 2. Optionally update to the newest crates.io release.
|
||||
if cfg.auto_update {
|
||||
if let Some(latest) = latest_crates_version().await {
|
||||
rt.status.lock().unwrap().latest_version = Some(latest.clone());
|
||||
if ver
|
||||
.as_deref()
|
||||
.map(|c| version_lt(c, &latest))
|
||||
.unwrap_or(false)
|
||||
{
|
||||
rt.log(format!(
|
||||
"updating cortiq-cli {} → {latest}…",
|
||||
ver.as_deref().unwrap_or("?")
|
||||
));
|
||||
if let Err(e) = cargo_install(true).await {
|
||||
rt.log(format!("update failed (keeping current): {e}"));
|
||||
} else {
|
||||
ver = installed_version(&cfg.cortiq_bin);
|
||||
rt.status.lock().unwrap().installed_version = ver.clone();
|
||||
rt.log(format!("updated to {}", ver.as_deref().unwrap_or("?")));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Spawn each server (fast).
|
||||
for server in &servers {
|
||||
if !std::path::Path::new(&server.model).exists() {
|
||||
rt.fail_server(
|
||||
&server.id,
|
||||
format!("model file not found: {}", server.model),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
rt.log(format!(
|
||||
"starting: cortiq serve {} --host {} --port {}",
|
||||
server.model, cfg.local_host, server.port
|
||||
));
|
||||
match spawn_serve(server, &cfg) {
|
||||
Ok(child) => rt.add_child(server.id.clone(), child),
|
||||
Err(e) => rt.fail_server(&server.id, format!("failed to spawn: {e}")),
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Wait until each spawned server is serving (model load can be slow).
|
||||
for server in &servers {
|
||||
if !rt.is_running(&server.id) {
|
||||
continue; // failed to spawn / file missing
|
||||
}
|
||||
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
|
||||
if ok {
|
||||
rt.set_healthy(&server.id, true);
|
||||
// capability handshake: the server said whether its model's chat
|
||||
// template can drive tool calling — no manual caps needed
|
||||
if tools == Some(true) {
|
||||
rt.detected_tools.lock().unwrap().insert(server.id.clone());
|
||||
tracing::info!(id = %server.id, "local server advertises tool support");
|
||||
} else {
|
||||
rt.detected_tools.lock().unwrap().remove(&server.id);
|
||||
}
|
||||
rt.log(format!(
|
||||
"local CMF server ready — '{}' at http://{}:{}/v1",
|
||||
server.id, cfg.local_host, server.port
|
||||
));
|
||||
} else {
|
||||
rt.fail_server(&server.id, "did not become healthy in time");
|
||||
}
|
||||
}
|
||||
}
|
||||
+663
-11
@@ -3,6 +3,51 @@
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// Base directory for gateway-managed data (models, logs). Honors
|
||||
/// `$CORTIQ_GATEWAY_HOME`, then `$XDG_DATA_HOME/cortiq-gateway`, then
|
||||
/// `~/.cortiq-gateway` — so defaults work regardless of the process CWD
|
||||
/// (important once the gateway is installed rather than run from its repo).
|
||||
/// On Windows also checks `%APPDATA%`, `%LOCALAPPDATA%` and `%USERPROFILE%`
|
||||
/// so double-clicking the exe from `Downloads` does not try to write into
|
||||
/// `Program Files`.
|
||||
pub fn data_dir() -> PathBuf {
|
||||
for (var, sub) in [
|
||||
("CORTIQ_GATEWAY_HOME", ""),
|
||||
("XDG_DATA_HOME", "cortiq-gateway"),
|
||||
("HOME", ".cortiq-gateway"),
|
||||
] {
|
||||
if let Ok(v) = std::env::var(var) {
|
||||
if !v.is_empty() {
|
||||
return if sub.is_empty() {
|
||||
PathBuf::from(v)
|
||||
} else {
|
||||
PathBuf::from(v).join(sub)
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(windows)]
|
||||
{
|
||||
for (var, sub) in [
|
||||
("APPDATA", "cortiq-gateway"),
|
||||
("LOCALAPPDATA", "cortiq-gateway"),
|
||||
("USERPROFILE", ".cortiq-gateway"),
|
||||
] {
|
||||
if let Ok(v) = std::env::var(var) {
|
||||
if !v.is_empty() {
|
||||
return PathBuf::from(v).join(sub);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
PathBuf::from("data")
|
||||
}
|
||||
|
||||
fn data_path(name: &str) -> String {
|
||||
data_dir().join(name).to_string_lossy().into_owned()
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct Config {
|
||||
@@ -33,10 +78,224 @@ pub struct Config {
|
||||
pub stats: StatsCfg,
|
||||
#[serde(default)]
|
||||
pub cache: CacheCfg,
|
||||
#[serde(default)]
|
||||
pub shadow: ShadowCfg,
|
||||
#[serde(default)]
|
||||
pub cmf: CmfCfg,
|
||||
}
|
||||
|
||||
/// One managed local model: the gateway runs a dedicated `cortiq serve` for it
|
||||
/// on its own port and registers it in the pool under `id`.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct CmfServer {
|
||||
/// Pool id it is registered under (e.g. "cmf-qwen3-5-2b").
|
||||
pub id: String,
|
||||
/// Path to the `.cmf` to serve.
|
||||
pub model: String,
|
||||
/// TCP port the dedicated `cortiq serve` binds to (must be unique).
|
||||
pub port: u16,
|
||||
/// Worker threads (CMF_THREADS). 0 = runtime default.
|
||||
#[serde(default = "default_cmf_threads")]
|
||||
pub threads: usize,
|
||||
/// Offload matvecs to the GPU (CMF_GPU=1).
|
||||
#[serde(default)]
|
||||
pub gpu: bool,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) Nyström attention layer spec (all | deepN | i,j,k | off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// O(1) landmark budget (default 32; ≥4).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_m: Option<u32>,
|
||||
/// O(1) exact-window width (default 128).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_window: Option<u32>,
|
||||
/// O(1) permanent exact sink keys (default 4, StreamingLLM discipline).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_sink: Option<u32>,
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
|
||||
/// gateway auto-detects tool support from the server's healthz handshake.
|
||||
#[serde(default)]
|
||||
pub caps: Vec<String>,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
/// local `.cmf` (quantized), and serve it via cortiq-server. Points the
|
||||
/// gateway at the Python converter + runtime on the host.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct CmfCfg {
|
||||
/// Python interpreter that has torch + the converter deps.
|
||||
#[serde(default = "default_cmf_python")]
|
||||
pub python_bin: String,
|
||||
/// Path to `convert_dtgma_to_cmf.py`.
|
||||
#[serde(default = "default_cmf_converter")]
|
||||
pub converter: String,
|
||||
/// Directory where produced `.cmf` files are written.
|
||||
#[serde(default = "default_cmf_models_dir")]
|
||||
pub models_dir: String,
|
||||
/// Base URL of the cortiq-server that serves the `.cmf` (OpenAI /v1),
|
||||
/// used when registering a converted model as a provider.
|
||||
#[serde(default = "default_cmf_server_url")]
|
||||
pub cortiq_server_url: String,
|
||||
/// Secret name of a HuggingFace token (higher search rate limits +
|
||||
/// gated/private repos). Empty = anonymous public search.
|
||||
#[serde(default)]
|
||||
pub hf_token_env: String,
|
||||
/// `cortiq` CLI binary for the local server (from `cargo install cortiq-cli`).
|
||||
#[serde(default = "default_cortiq_bin")]
|
||||
pub cortiq_bin: String,
|
||||
/// Install cortiq-cli from crates.io automatically if the binary is missing.
|
||||
#[serde(default)]
|
||||
pub auto_install: bool,
|
||||
/// On startup, check crates.io and reinstall if a newer cortiq-cli exists.
|
||||
#[serde(default)]
|
||||
pub auto_update: bool,
|
||||
/// Spawn and manage a local `cortiq serve` for `local_model`, and register
|
||||
/// it in the model pool — a local `.cmf` backend the gateway can use even
|
||||
/// when no external provider (or the router) is available.
|
||||
#[serde(default)]
|
||||
pub manage_server: bool,
|
||||
/// Path to the local `.cmf` to serve (empty = disabled).
|
||||
#[serde(default)]
|
||||
pub local_model: String,
|
||||
/// Host the managed local server binds to.
|
||||
#[serde(default = "default_cmf_local_host")]
|
||||
pub local_host: String,
|
||||
/// Port the managed local server binds to.
|
||||
#[serde(default = "default_cmf_local_port")]
|
||||
pub local_port: u16,
|
||||
/// Model id the local server is registered under in the pool.
|
||||
#[serde(default = "default_cmf_model_id")]
|
||||
pub model_id: String,
|
||||
/// Use ONLY local models: route every request to the local model, ignoring
|
||||
/// the router and any external providers ("use only local models").
|
||||
#[serde(default)]
|
||||
pub local_only: bool,
|
||||
/// Worker threads for the managed `cortiq serve` (exported as `CMF_THREADS`).
|
||||
/// 0 = let the runtime decide; N>1 parallelises decode matvecs across N cores
|
||||
/// (measured ~2.8x faster than serial on a 15B model). Default 8.
|
||||
#[serde(default = "default_cmf_threads")]
|
||||
pub threads: usize,
|
||||
/// Offload large matvecs to the GPU (exports CMF_GPU=1). On Apple Silicon the
|
||||
/// decode loop is memory-bandwidth bound, so this is ~neutral for generation
|
||||
/// (helps prefill of long prompts). Off by default.
|
||||
#[serde(default)]
|
||||
pub gpu: bool,
|
||||
/// Managed local models. When non-empty (and `manage_server` is on), the
|
||||
/// gateway spawns and supervises one `cortiq serve` per entry — each on its
|
||||
/// own port — and registers each in the pool. Empty = fall back to the single
|
||||
/// legacy `local_model` / `local_port` / `model_id` below (back-compat).
|
||||
#[serde(default)]
|
||||
pub servers: Vec<CmfServer>,
|
||||
}
|
||||
impl CmfCfg {
|
||||
/// The managed local models actually in effect: the `servers` list when set,
|
||||
/// otherwise the single legacy `local_model` (as a one-element list). Empty
|
||||
/// when `manage_server` is off or nothing is configured.
|
||||
pub fn effective_servers(&self) -> Vec<CmfServer> {
|
||||
if !self.manage_server {
|
||||
return Vec::new();
|
||||
}
|
||||
if !self.servers.is_empty() {
|
||||
return self.servers.clone();
|
||||
}
|
||||
if !self.local_model.trim().is_empty() {
|
||||
return vec![CmfServer {
|
||||
id: self.model_id.clone(),
|
||||
model: self.local_model.clone(),
|
||||
port: self.local_port,
|
||||
threads: self.threads,
|
||||
gpu: self.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
}
|
||||
}
|
||||
impl Default for CmfCfg {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
python_bin: default_cmf_python(),
|
||||
converter: default_cmf_converter(),
|
||||
models_dir: default_cmf_models_dir(),
|
||||
cortiq_server_url: default_cmf_server_url(),
|
||||
hf_token_env: String::new(),
|
||||
cortiq_bin: default_cortiq_bin(),
|
||||
auto_install: false,
|
||||
auto_update: false,
|
||||
manage_server: false,
|
||||
local_model: String::new(),
|
||||
local_host: default_cmf_local_host(),
|
||||
local_port: default_cmf_local_port(),
|
||||
model_id: default_cmf_model_id(),
|
||||
local_only: false,
|
||||
threads: default_cmf_threads(),
|
||||
gpu: false,
|
||||
servers: Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
fn default_cmf_threads() -> usize {
|
||||
8
|
||||
}
|
||||
fn default_cortiq_bin() -> String {
|
||||
"cortiq".into()
|
||||
}
|
||||
fn default_cmf_local_host() -> String {
|
||||
"127.0.0.1".into()
|
||||
}
|
||||
fn default_cmf_local_port() -> u16 {
|
||||
8081
|
||||
}
|
||||
fn default_cmf_model_id() -> String {
|
||||
"cmf-local".into()
|
||||
}
|
||||
fn default_cmf_python() -> String {
|
||||
"python3".into()
|
||||
}
|
||||
fn default_cmf_converter() -> String {
|
||||
"../cmf/converter/convert_dtgma_to_cmf.py".into()
|
||||
}
|
||||
fn default_cmf_models_dir() -> String {
|
||||
data_path("models")
|
||||
}
|
||||
fn default_cmf_server_url() -> String {
|
||||
"http://127.0.0.1:8081/v1".into()
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct RouterCfg {
|
||||
/// Master switch — when false the router is never called; the gateway
|
||||
/// serves from the local model pool (local `.cmf` / configured default).
|
||||
#[serde(default = "default_router_enabled")]
|
||||
pub enabled: bool,
|
||||
pub url: String,
|
||||
#[serde(default = "default_router_timeout")]
|
||||
pub timeout_ms: u64,
|
||||
@@ -47,8 +306,13 @@ pub struct RouterCfg {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub taxonomy_id: Option<String>,
|
||||
}
|
||||
fn default_router_enabled() -> bool {
|
||||
true
|
||||
}
|
||||
fn default_router_timeout() -> u64 {
|
||||
800
|
||||
// the hosted router escalates hard prompts to an oracle (~10 s);
|
||||
// an 800 ms budget silently degraded every one of those to the fallback
|
||||
15_000
|
||||
}
|
||||
fn default_verify_tls() -> bool {
|
||||
true
|
||||
@@ -106,6 +370,27 @@ pub struct ModelCfg {
|
||||
pub api_key_env: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub caps: Vec<String>,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) constant-memory streaming attention hint (all, deep, custom, off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// Skip Multi-Token Prediction (MTP) heads.
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
fn default_kind() -> String {
|
||||
"chat".into()
|
||||
@@ -118,17 +403,149 @@ pub struct RoutingCfg {
|
||||
pub default: String,
|
||||
#[serde(default)]
|
||||
pub policy: RoutingPolicy,
|
||||
/// Per-task rules: `task_label → ordered model list`. A matching rule's
|
||||
/// models go FIRST; the complexity-tier chain follows as fallback, so a
|
||||
/// task pin never weakens failover. Named field, so a tier cannot be
|
||||
/// called "tasks" — serde extracts it before the flattened tier map.
|
||||
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
|
||||
pub tasks: HashMap<String, TierTargets>,
|
||||
/// Local complexity bands: re-map the router's continuous score into
|
||||
/// low/medium/high with YOUR OWN boundaries. The hosted router's bands
|
||||
/// are global for all customers; these are per-gateway. `None` = keep
|
||||
/// the router's tier verdict as delivered.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub bands: Option<TierBands>,
|
||||
}
|
||||
|
||||
/// Score boundaries: `score ≤ low` → low, `≤ medium` → medium, else high.
|
||||
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
|
||||
pub struct TierBands {
|
||||
pub low: f32,
|
||||
pub medium: f32,
|
||||
}
|
||||
|
||||
impl TierBands {
|
||||
pub fn tier_of(&self, score: f32) -> &'static str {
|
||||
if score <= self.low {
|
||||
"low"
|
||||
} else if score <= self.medium {
|
||||
"medium"
|
||||
} else {
|
||||
"high"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A routing tier is either a list of model_ids or the string `default`.
|
||||
/// (`default` is handled by a separate field; only tier lists appear here.)
|
||||
///
|
||||
/// TASK rules additionally accept a per-complexity map — "which model serves
|
||||
/// this task at each tier":
|
||||
///
|
||||
/// ```toml
|
||||
/// [routing.tasks.code]
|
||||
/// low = ["cheap-coder"]
|
||||
/// high = ["big-model"]
|
||||
/// any = ["fallback"] # leads at every tier after the tier-specific list
|
||||
/// ```
|
||||
///
|
||||
/// The plain-array form stays valid and means "leads at any tier". The
|
||||
/// transitional `{ models = [...], tier = "high" }` form is normalized into
|
||||
/// the map on load. Untagged: map is tried first, arrays keep parsing as is.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum TierTargets {
|
||||
Map(HashMap<String, TaskTierVal>),
|
||||
List(Vec<String>),
|
||||
One(String),
|
||||
}
|
||||
|
||||
/// A value inside a task's per-tier map: a model list, a single model, or the
|
||||
/// transitional string value of the legacy `tier = "high"` key.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum TaskTierVal {
|
||||
List(Vec<String>),
|
||||
Str(String),
|
||||
}
|
||||
|
||||
impl TaskTierVal {
|
||||
fn list(&self) -> Vec<String> {
|
||||
match self {
|
||||
TaskTierVal::List(v) => v.clone(),
|
||||
TaskTierVal::Str(s) => vec![s.clone()],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub const TASK_TIER_KEYS: [&str; 4] = ["any", "low", "medium", "high"];
|
||||
|
||||
impl TierTargets {
|
||||
/// Every referenced model id, whatever the syntactic form (validation).
|
||||
pub fn models(&self) -> Vec<String> {
|
||||
match self {
|
||||
TierTargets::Map(m) => {
|
||||
let mut out = Vec::new();
|
||||
for (k, v) in m {
|
||||
if k == "tier" {
|
||||
continue; // legacy marker, not a model list
|
||||
}
|
||||
out.extend(v.list());
|
||||
}
|
||||
out
|
||||
}
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
}
|
||||
}
|
||||
|
||||
/// Normalized task rule: `(any_tier_models, per_tier_models)`.
|
||||
/// Legacy `{models, tier}` becomes `per[tier] = models`; a plain array
|
||||
/// becomes `any`.
|
||||
pub fn task_rule(&self) -> (Vec<String>, HashMap<String, Vec<String>>) {
|
||||
match self {
|
||||
TierTargets::Map(m) => {
|
||||
let mut any: Vec<String> = Vec::new();
|
||||
let mut per: HashMap<String, Vec<String>> = HashMap::new();
|
||||
let legacy_tier = m.get("tier").and_then(|v| match v {
|
||||
TaskTierVal::Str(s) => Some(s.clone()),
|
||||
_ => None,
|
||||
});
|
||||
for (k, v) in m {
|
||||
match k.as_str() {
|
||||
"tier" => {}
|
||||
"any" | "models" => any.extend(v.list()),
|
||||
"low" | "medium" | "high" => {
|
||||
per.entry(k.clone()).or_default().extend(v.list())
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if let Some(tier) = legacy_tier {
|
||||
// legacy pin: the models were meant for that tier
|
||||
let moved = std::mem::take(&mut any);
|
||||
per.entry(tier).or_default().extend(moved);
|
||||
}
|
||||
(any, per)
|
||||
}
|
||||
TierTargets::List(v) => (v.clone(), HashMap::new()),
|
||||
TierTargets::One(s) => (vec![s.clone()], HashMap::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Unknown keys inside a task map (validation).
|
||||
pub fn invalid_task_keys(&self) -> Vec<String> {
|
||||
match self {
|
||||
TierTargets::Map(m) => m
|
||||
.keys()
|
||||
.filter(|k| !TASK_TIER_KEYS.contains(&k.as_str()) && *k != "models" && *k != "tier")
|
||||
.cloned()
|
||||
.collect(),
|
||||
_ => Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct RoutingPolicy {
|
||||
#[serde(default = "default_mode")]
|
||||
@@ -208,6 +625,9 @@ pub struct ApiKeyCfg {
|
||||
pub account: String,
|
||||
#[serde(default)]
|
||||
pub rate_per_min: u32,
|
||||
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub monthly_budget_usd: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub allow_models: Vec<String>,
|
||||
}
|
||||
@@ -261,10 +681,12 @@ pub struct AdminCfg {
|
||||
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
|
||||
#[serde(default = "default_admin_enabled")]
|
||||
pub enabled: bool,
|
||||
/// Name of the env variable holding the admin token. If unset (or empty),
|
||||
/// a token is generated at startup and printed to the log.
|
||||
/// Name of the env variable holding the admin token.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token_env: Option<String>,
|
||||
/// Persistent admin token configured directly in toml.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token: Option<String>,
|
||||
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
|
||||
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
@@ -275,6 +697,7 @@ impl Default for AdminCfg {
|
||||
Self {
|
||||
enabled: default_admin_enabled(),
|
||||
token_env: None,
|
||||
token: None,
|
||||
listen: None,
|
||||
}
|
||||
}
|
||||
@@ -298,6 +721,12 @@ pub struct StatsCfg {
|
||||
/// Retention window for aggregates/time-series replayed on startup, e.g. `7d`.
|
||||
#[serde(default = "default_stats_retention")]
|
||||
pub retention: String,
|
||||
/// Compact the JSONL file once it exceeds this size: records older than
|
||||
/// `retention` fold into cumulative per-account totals (a small sidecar
|
||||
/// file) and are removed, so per-key accounting survives while the disk
|
||||
/// stays bounded. 0 disables compaction.
|
||||
#[serde(default = "default_stats_max_file_mb")]
|
||||
pub max_file_mb: u64,
|
||||
}
|
||||
impl Default for StatsCfg {
|
||||
fn default() -> Self {
|
||||
@@ -306,6 +735,7 @@ impl Default for StatsCfg {
|
||||
file: default_stats_file(),
|
||||
ring_size: default_stats_ring(),
|
||||
retention: default_stats_retention(),
|
||||
max_file_mb: default_stats_max_file_mb(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -313,7 +743,7 @@ fn default_stats_enabled() -> bool {
|
||||
true
|
||||
}
|
||||
fn default_stats_file() -> String {
|
||||
"config/stats.jsonl".into()
|
||||
data_path("stats.jsonl")
|
||||
}
|
||||
fn default_stats_ring() -> usize {
|
||||
500
|
||||
@@ -321,6 +751,85 @@ fn default_stats_ring() -> usize {
|
||||
fn default_stats_retention() -> String {
|
||||
"7d".into()
|
||||
}
|
||||
fn default_stats_max_file_mb() -> u64 {
|
||||
16
|
||||
}
|
||||
|
||||
/// Self-warming shadow loop (docs/SELF_WARMING_GATEWAY.md). While a task
|
||||
/// label is not yet promoted, the client is served the CLOUD answer; in
|
||||
/// parallel (sampled, non-blocking) the local `cmf-local` model answers and
|
||||
/// a cheap `judge` model scores it against the cloud answer. The per-label
|
||||
/// Wilson-LB of that pass stream gates promotion to local serving.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct ShadowCfg {
|
||||
/// Master switch. Off = zero overhead, classic gateway behavior.
|
||||
#[serde(default)]
|
||||
pub enabled: bool,
|
||||
/// model_id (from `[[models]]`) of the local CMF model (OpenAI-compatible
|
||||
/// cortiq-server). Empty disables the loop.
|
||||
#[serde(default)]
|
||||
pub local_model_id: String,
|
||||
/// model_id of the cheap judge (different family than the answerer).
|
||||
#[serde(default)]
|
||||
pub judge_model_id: String,
|
||||
/// Fraction of eligible requests to shadow+judge (0..1). Bounds cost.
|
||||
#[serde(default = "default_shadow_sample")]
|
||||
pub sample_rate: f64,
|
||||
/// judge.jsonl append+replay path. Empty = in-memory only.
|
||||
#[serde(default = "default_shadow_file")]
|
||||
pub file: String,
|
||||
/// Wilson-95 lower bound of pass-rate required to promote.
|
||||
#[serde(default = "default_shadow_lb")]
|
||||
pub promote_lb: f64,
|
||||
/// Min judgments before promotion is considered.
|
||||
#[serde(default = "default_shadow_nmin")]
|
||||
pub n_min: usize,
|
||||
/// Rolling window of judgments per label.
|
||||
#[serde(default = "default_shadow_window")]
|
||||
pub window: usize,
|
||||
/// Extra canary judgments before full local serving.
|
||||
#[serde(default = "default_shadow_soak")]
|
||||
pub soak: usize,
|
||||
/// GATE past shadow-only: actually SERVE promoted labels from the local
|
||||
/// model (with failover to cloud on error + a complexity veto). Default
|
||||
/// OFF — the memo's 3 risks (§8) should be retired by measurement first.
|
||||
#[serde(default)]
|
||||
pub serve_when_promoted: bool,
|
||||
}
|
||||
impl Default for ShadowCfg {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
enabled: false,
|
||||
local_model_id: String::new(),
|
||||
judge_model_id: String::new(),
|
||||
sample_rate: default_shadow_sample(),
|
||||
file: default_shadow_file(),
|
||||
promote_lb: default_shadow_lb(),
|
||||
n_min: default_shadow_nmin(),
|
||||
window: default_shadow_window(),
|
||||
soak: default_shadow_soak(),
|
||||
serve_when_promoted: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
fn default_shadow_sample() -> f64 {
|
||||
0.15
|
||||
}
|
||||
fn default_shadow_file() -> String {
|
||||
data_path("judge.jsonl")
|
||||
}
|
||||
fn default_shadow_lb() -> f64 {
|
||||
0.95
|
||||
}
|
||||
fn default_shadow_nmin() -> usize {
|
||||
200
|
||||
}
|
||||
fn default_shadow_window() -> usize {
|
||||
500
|
||||
}
|
||||
fn default_shadow_soak() -> usize {
|
||||
100
|
||||
}
|
||||
|
||||
/// Semantic (embedding-based) response cache — returns a cached answer for prompts
|
||||
/// that are semantically near a previous one, skipping the (expensive) model call.
|
||||
@@ -328,9 +837,10 @@ fn default_stats_retention() -> String {
|
||||
pub struct CacheCfg {
|
||||
#[serde(default)]
|
||||
pub enabled: bool,
|
||||
/// Cosine-similarity threshold for a hit (0..1).
|
||||
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
|
||||
/// the number you typed (an f32 round-trips as 0.9200000166893005).
|
||||
#[serde(default = "default_semcache_threshold")]
|
||||
pub threshold: f32,
|
||||
pub threshold: f64,
|
||||
/// Entry time-to-live, e.g. `1h`.
|
||||
#[serde(default = "default_semcache_ttl")]
|
||||
pub ttl: String,
|
||||
@@ -352,7 +862,7 @@ impl Default for CacheCfg {
|
||||
}
|
||||
}
|
||||
}
|
||||
fn default_semcache_threshold() -> f32 {
|
||||
fn default_semcache_threshold() -> f64 {
|
||||
0.92
|
||||
}
|
||||
fn default_semcache_ttl() -> String {
|
||||
@@ -362,6 +872,83 @@ fn default_semcache_max() -> usize {
|
||||
1000
|
||||
}
|
||||
|
||||
/// Sensible defaults for a fresh install (empty pool — user fills via wizard/UI).
|
||||
impl Default for Config {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
listen: "0.0.0.0:9000".to_string(),
|
||||
// The hosted allaigate router — the thing this gateway is built
|
||||
// around. Without a key the pipeline degrades gracefully to local
|
||||
// models and the console asks for one; nothing breaks.
|
||||
router: RouterCfg {
|
||||
enabled: true,
|
||||
url: "https://138.226.222.209".to_string(),
|
||||
timeout_ms: 15_000, // hard prompts escalate to an oracle (~10 s)
|
||||
verify_tls: false, // the hosted endpoint is IP-bound
|
||||
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
|
||||
taxonomy_id: Some("data-assistant".to_string()),
|
||||
},
|
||||
route: RouteCfg::default(),
|
||||
models: Vec::new(),
|
||||
routing: RoutingCfg {
|
||||
tiers: HashMap::new(),
|
||||
default: String::new(),
|
||||
policy: RoutingPolicy::default(),
|
||||
tasks: HashMap::new(),
|
||||
bands: None,
|
||||
},
|
||||
breaker: BreakerCfg::default(),
|
||||
protocols: ProtocolsCfg {
|
||||
openai_chat: true,
|
||||
openai_completions: true,
|
||||
openai_embeddings: true,
|
||||
openai_models: true,
|
||||
// on by default: the API-keys view ships a Claude Code
|
||||
// auto-setup snippet that targets /v1/messages
|
||||
anthropic_messages: true,
|
||||
mcp: false,
|
||||
native_passthrough: true,
|
||||
},
|
||||
cortiq: CortiqCfg::default(),
|
||||
api_keys: Vec::new(),
|
||||
idempotency: IdempotencyCfg::default(),
|
||||
log: LogCfg::default(),
|
||||
telemetry: TelemetryCfg::default(),
|
||||
admin: AdminCfg::default(),
|
||||
stats: StatsCfg::default(),
|
||||
cache: CacheCfg::default(),
|
||||
shadow: ShadowCfg::default(),
|
||||
cmf: CmfCfg::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Where the config lives when `--config` is not given.
|
||||
/// A readable `./config/gateway.toml` (a real gateway checkout / install dir)
|
||||
/// wins; otherwise the per-user home (`$CORTIQ_GATEWAY_HOME` or
|
||||
/// `~/.cortiq-gateway`) — so `cortiq-gateway` typed in any directory picks up
|
||||
/// the same configuration it was set up with. A foreign project that merely
|
||||
/// has `Cargo.toml` and a `config/` folder must NOT capture the config
|
||||
/// (that used to shadow the user's real config with a broken path).
|
||||
pub fn default_config_path() -> String {
|
||||
let repo_path = "config/gateway.toml";
|
||||
// the file itself, readable — not just an entry that exists()
|
||||
if std::fs::read_to_string(repo_path).is_ok() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
// our own repo checkout before first run: config/ exists, file not yet
|
||||
let ours = std::fs::read_to_string("Cargo.toml")
|
||||
.map(|t| t.contains("name = \"cortiq-gateway\""))
|
||||
.unwrap_or(false);
|
||||
if ours && std::path::Path::new("config").is_dir() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
data_dir()
|
||||
.join("gateway.toml")
|
||||
.to_string_lossy()
|
||||
.into_owned()
|
||||
}
|
||||
|
||||
impl Config {
|
||||
/// Load and validate the config from a file.
|
||||
pub fn load(path: &str) -> anyhow::Result<Self> {
|
||||
@@ -372,10 +959,36 @@ impl Config {
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Load if exists, otherwise create a fresh default config at `path` and return it.
|
||||
/// Never requires the user to create or copy a file by hand.
|
||||
pub fn load_or_create(path: &str) -> anyhow::Result<Self> {
|
||||
if std::path::Path::new(path).exists() {
|
||||
return Self::load(path);
|
||||
}
|
||||
let cfg = Self::default();
|
||||
// ensure parent dir exists
|
||||
if let Some(dir) = std::path::Path::new(path).parent() {
|
||||
let _ = std::fs::create_dir_all(dir);
|
||||
}
|
||||
cfg.save(path)?;
|
||||
eprintln!("created default config at {path} — open /admin to finish setup");
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Integrity validation: non-empty model pool, valid routing targets.
|
||||
pub fn validate(&self) -> anyhow::Result<()> {
|
||||
if self.models.is_empty() {
|
||||
anyhow::bail!("config: at least one [[models]] entry is required");
|
||||
// A managed local CMF model counts as an available model even though it
|
||||
// is registered dynamically rather than via a [[models]] entry — this
|
||||
// is what lets the gateway run on local models alone (no cloud, no router).
|
||||
let servers = self.cmf.effective_servers();
|
||||
let has_local = !servers.is_empty();
|
||||
if self.models.is_empty() && !has_local {
|
||||
// Fresh install: start with an EMPTY model pool so the admin console
|
||||
// comes up — the user adds models in /admin → Models (or points [cmf]
|
||||
// at a local .cmf). Routing simply has nothing to select until then.
|
||||
tracing::warn!(
|
||||
"no models configured — starting with an empty pool; add one in /admin → Models"
|
||||
);
|
||||
}
|
||||
// uniqueness of model ids
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
@@ -385,9 +998,19 @@ impl Config {
|
||||
}
|
||||
}
|
||||
// every target in [routing] must exist in the model pool
|
||||
let ids: std::collections::HashSet<&str> =
|
||||
let mut ids: std::collections::HashSet<&str> =
|
||||
self.models.iter().map(|m| m.id.as_str()).collect();
|
||||
if !ids.contains(self.routing.default.as_str()) {
|
||||
// managed local servers are known ids too; also enforce unique ports/ids
|
||||
let mut ports = std::collections::HashSet::new();
|
||||
for s in &servers {
|
||||
if !ids.insert(s.id.as_str()) {
|
||||
anyhow::bail!("config: duplicate managed model id '{}'", s.id);
|
||||
}
|
||||
if !ports.insert(s.port) {
|
||||
anyhow::bail!("config: two managed models share port {}", s.port);
|
||||
}
|
||||
}
|
||||
if !self.routing.default.is_empty() && !ids.contains(self.routing.default.as_str()) {
|
||||
anyhow::bail!(
|
||||
"config: routing.default '{}' is not a known model id",
|
||||
self.routing.default
|
||||
@@ -395,6 +1018,7 @@ impl Config {
|
||||
}
|
||||
for (tier, targets) in &self.routing.tiers {
|
||||
let list = match targets {
|
||||
TierTargets::Map(_) => targets.models(),
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
@@ -408,6 +1032,33 @@ impl Config {
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(b) = &self.routing.bands {
|
||||
if !(b.low > 0.0 && b.low < b.medium && b.medium < 1.0) {
|
||||
anyhow::bail!(
|
||||
"config: routing.bands must satisfy 0 < low < medium < 1 (got {} / {})",
|
||||
b.low,
|
||||
b.medium
|
||||
);
|
||||
}
|
||||
}
|
||||
for (task, targets) in &self.routing.tasks {
|
||||
for id in &targets.models() {
|
||||
if !ids.contains(id.as_str()) {
|
||||
anyhow::bail!(
|
||||
"config: routing.tasks.{} references unknown model id '{}'",
|
||||
task,
|
||||
id
|
||||
);
|
||||
}
|
||||
}
|
||||
if let Some(bad) = targets.invalid_task_keys().first() {
|
||||
anyhow::bail!(
|
||||
"config: routing.tasks.{}.{} is not a tier (use any/low/medium/high)",
|
||||
task,
|
||||
bad
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -498,6 +1149,7 @@ min_class = { low = "cheap", medium = "mid" }
|
||||
|
||||
let low = back.routing.tiers.get("low").expect("low tier present");
|
||||
match low {
|
||||
TierTargets::Map(_) => assert_eq!(low.models(), vec!["local-qwen".to_string()]),
|
||||
TierTargets::List(v) => assert_eq!(v, &vec!["local-qwen".to_string()]),
|
||||
TierTargets::One(s) => assert_eq!(s, "local-qwen"),
|
||||
}
|
||||
|
||||
@@ -13,6 +13,8 @@ pub enum GatewayError {
|
||||
Unauthorized(String),
|
||||
#[error("rate limit exceeded")]
|
||||
RateLimited { retry_after_secs: u64 },
|
||||
#[error("monthly budget exhausted (limit ${budget_usd})")]
|
||||
BudgetExhausted { budget_usd: f64 },
|
||||
#[error("upstream unavailable: {0}")]
|
||||
UpstreamUnavailable(String),
|
||||
#[error("internal error: {0}")]
|
||||
@@ -37,6 +39,11 @@ impl GatewayError {
|
||||
"rate_limit_error",
|
||||
"rate_limited",
|
||||
),
|
||||
Self::BudgetExhausted { .. } => (
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"insufficient_quota",
|
||||
"insufficient_quota",
|
||||
),
|
||||
Self::UpstreamUnavailable(_) => (
|
||||
StatusCode::BAD_GATEWAY,
|
||||
"upstream_unavailable",
|
||||
|
||||
+1317
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,78 @@
|
||||
//! In-memory ring of recent log lines, so the admin console can show logs to
|
||||
//! users who never see the terminal (Docker, autostart, double-clicked exe).
|
||||
//! Fed by a second `tracing` layer writing plain text (no ANSI) into the ring
|
||||
//! while the normal stdout layer stays colored.
|
||||
|
||||
use std::collections::VecDeque;
|
||||
use std::io;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
|
||||
const CAP: usize = 500;
|
||||
|
||||
fn ring() -> &'static Arc<Mutex<VecDeque<String>>> {
|
||||
static RING: OnceLock<Arc<Mutex<VecDeque<String>>>> = OnceLock::new();
|
||||
RING.get_or_init(|| Arc::new(Mutex::new(VecDeque::with_capacity(CAP))))
|
||||
}
|
||||
|
||||
/// Last `limit` lines, oldest first.
|
||||
pub fn snapshot(limit: usize) -> Vec<String> {
|
||||
let g = ring().lock().unwrap();
|
||||
g.iter()
|
||||
.skip(g.len().saturating_sub(limit))
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Drop everything in the ring (the admin console's "clear" button).
|
||||
pub fn clear() {
|
||||
ring().lock().unwrap().clear();
|
||||
}
|
||||
|
||||
pub fn push_line(line: &str) {
|
||||
let line = line.trim_end();
|
||||
if line.is_empty() {
|
||||
return;
|
||||
}
|
||||
let mut g = ring().lock().unwrap();
|
||||
if g.len() >= CAP {
|
||||
g.pop_front();
|
||||
}
|
||||
g.push_back(line.to_string());
|
||||
}
|
||||
|
||||
/// `MakeWriter` for the ring layer: buffers partial writes until a newline.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct RingMakeWriter;
|
||||
|
||||
pub struct RingWriter {
|
||||
buf: String,
|
||||
}
|
||||
|
||||
impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for RingMakeWriter {
|
||||
type Writer = RingWriter;
|
||||
fn make_writer(&'a self) -> Self::Writer {
|
||||
RingWriter { buf: String::new() }
|
||||
}
|
||||
}
|
||||
|
||||
impl io::Write for RingWriter {
|
||||
fn write(&mut self, data: &[u8]) -> io::Result<usize> {
|
||||
self.buf.push_str(&String::from_utf8_lossy(data));
|
||||
while let Some(idx) = self.buf.find('\n') {
|
||||
let line: String = self.buf.drain(..=idx).collect();
|
||||
push_line(&line);
|
||||
}
|
||||
Ok(data.len())
|
||||
}
|
||||
fn flush(&mut self) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for RingWriter {
|
||||
fn drop(&mut self) {
|
||||
if !self.buf.is_empty() {
|
||||
push_line(&self.buf.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
+276
-27
@@ -9,11 +9,17 @@
|
||||
#![allow(dead_code)]
|
||||
|
||||
mod admin;
|
||||
mod bench;
|
||||
mod cache;
|
||||
mod cmf_runtime;
|
||||
mod config;
|
||||
mod error;
|
||||
mod import;
|
||||
mod logbuf;
|
||||
mod media;
|
||||
mod model;
|
||||
mod pipeline;
|
||||
mod promotion;
|
||||
mod protocols;
|
||||
mod providers;
|
||||
mod registry;
|
||||
@@ -22,6 +28,7 @@ mod routing;
|
||||
mod secrets;
|
||||
mod state;
|
||||
mod stats;
|
||||
mod sysmon;
|
||||
|
||||
use axum::{routing::get, Router};
|
||||
use clap::Parser;
|
||||
@@ -35,55 +42,223 @@ use state::SharedState;
|
||||
about = "Universal LLM gateway with intelligent routing"
|
||||
)]
|
||||
struct Args {
|
||||
/// Path to the TOML config file.
|
||||
#[arg(long, default_value = "config/gateway.toml")]
|
||||
config: String,
|
||||
/// Path to the TOML config file. If omitted, uses `config/gateway.toml` when running
|
||||
/// inside the repo, otherwise `~/.cortiq-gateway/gateway.toml` (or `$CORTIQ_GATEWAY_HOME`).
|
||||
/// Missing file is created automatically — no manual copying needed.
|
||||
#[arg(long)]
|
||||
config: Option<String>,
|
||||
/// Admin token for the web panel (overrides `[admin].token_env`).
|
||||
#[arg(long)]
|
||||
admin_token: Option<String>,
|
||||
/// Do not automatically open the browser on first run.
|
||||
#[arg(long)]
|
||||
no_browser: bool,
|
||||
/// Do not auto-select a free port if the configured one is busy.
|
||||
#[arg(long)]
|
||||
no_port_search: bool,
|
||||
}
|
||||
|
||||
/// Resolve the admin token: env from `[admin].token_env` → `--admin-token` →
|
||||
/// generate a random one. Returns `(token, generated)`.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
|
||||
///
|
||||
/// By default no token is required (open mode). User sets it explicitly via
|
||||
/// `--admin-token`, `[admin].token` or `[admin].token_env` if they want auth.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> String {
|
||||
if let Some(env) = &cfg.admin.token_env {
|
||||
if let Ok(v) = std::env::var(env) {
|
||||
if !v.is_empty() {
|
||||
return (v, false);
|
||||
return v;
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(t) = &args.admin_token {
|
||||
if !t.is_empty() {
|
||||
return (t.clone(), false);
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
(admin::random_token(24), true)
|
||||
if let Some(t) = &cfg.admin.token {
|
||||
if !t.is_empty() {
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
String::new()
|
||||
}
|
||||
|
||||
fn is_docker() -> bool {
|
||||
std::path::Path::new("/.dockerenv").exists()
|
||||
|| std::fs::read_to_string("/proc/1/cgroup")
|
||||
.map(|s| s.contains("docker") || s.contains("kubepods"))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn should_open_browser(args: &Args) -> bool {
|
||||
if args.no_browser {
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CORTIQ_NO_BROWSER")
|
||||
.map(|v| !v.is_empty() && v != "0")
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CI").is_ok() || std::env::var("NO_BROWSER").is_ok() {
|
||||
return false;
|
||||
}
|
||||
if is_docker() {
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn parse_listen(listen: &str) -> (String, u16) {
|
||||
if let Some((host, port_s)) = listen.rsplit_once(':') {
|
||||
if let Ok(p) = port_s.parse::<u16>() {
|
||||
return (host.to_string(), p);
|
||||
}
|
||||
}
|
||||
(listen.to_string(), 9000)
|
||||
}
|
||||
|
||||
fn is_port_free(host: &str, port: u16) -> bool {
|
||||
let addr = format!("{host}:{port}");
|
||||
// bind test + connect test (docker port forwarding can make bind succeed while port is still forwarded)
|
||||
if std::net::TcpListener::bind(&addr).is_err() {
|
||||
return false;
|
||||
}
|
||||
use std::net::ToSocketAddrs;
|
||||
let probe_host = if host == "0.0.0.0" { "127.0.0.1" } else { host };
|
||||
if let Ok(mut addrs) = (probe_host, port).to_socket_addrs() {
|
||||
if let Some(a) = addrs.next() {
|
||||
if std::net::TcpStream::connect_timeout(&a, std::time::Duration::from_millis(200))
|
||||
.is_ok()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn resolve_listen(cfg: &mut Config, no_search: bool) -> String {
|
||||
if no_search {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
let (host, port) = parse_listen(&cfg.listen);
|
||||
let clean_host = if host.is_empty() {
|
||||
"0.0.0.0".to_string()
|
||||
} else {
|
||||
host
|
||||
};
|
||||
if is_port_free(&clean_host, port) {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
for p in (port + 1)..=port.saturating_add(50) {
|
||||
if is_port_free(&clean_host, p) {
|
||||
let new_listen = format!("{clean_host}:{p}");
|
||||
eprintln!("port {port} busy — switching to free port {p} ({new_listen})");
|
||||
cfg.listen = new_listen.clone();
|
||||
return new_listen;
|
||||
}
|
||||
}
|
||||
if !is_port_free(&clean_host, port) {
|
||||
eprintln!(
|
||||
"warning: port {port} busy and no free port found in next 50, will try to bind anyway"
|
||||
);
|
||||
}
|
||||
cfg.listen.clone()
|
||||
}
|
||||
|
||||
fn setup_marker_path(config_path: &str) -> std::path::PathBuf {
|
||||
// marker lives next to config file: gateway.toml -> .setup_done
|
||||
std::path::Path::new(config_path)
|
||||
.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.join(".setup_done")
|
||||
}
|
||||
|
||||
fn needs_first_run_wizard(cfg: &Config, config_path: &str) -> bool {
|
||||
let marker = setup_marker_path(config_path);
|
||||
if marker.exists() {
|
||||
return false;
|
||||
}
|
||||
// also consider "needs setup" if model pool is empty and no managed cmf
|
||||
if !cfg.models.is_empty() || !cfg.cmf.effective_servers().is_empty() {
|
||||
// still show wizard once if marker missing, but not if user already has models
|
||||
// we treat missing marker + existing models as "already configured" to avoid nagging
|
||||
// persist marker lazily
|
||||
let _ = std::fs::write(&marker, "done");
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let args = Args::parse();
|
||||
let config_path = args
|
||||
.config
|
||||
.clone()
|
||||
.unwrap_or_else(config::default_config_path);
|
||||
|
||||
let cfg = Config::load(&args.config)?;
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| cfg.log.level.clone().into()),
|
||||
)
|
||||
.init();
|
||||
let mut cfg = Config::load_or_create(&config_path)?;
|
||||
// auto-pick a free port if the configured one is busy (unless disabled)
|
||||
let original_listen = cfg.listen.clone();
|
||||
let listen = resolve_listen(&mut cfg, args.no_port_search);
|
||||
// persist the resolved port if it changed
|
||||
if listen != original_listen {
|
||||
let _ = cfg.save(&config_path);
|
||||
}
|
||||
{
|
||||
use tracing_subscriber::layer::SubscriberExt;
|
||||
use tracing_subscriber::util::SubscriberInitExt;
|
||||
let filter = tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| cfg.log.level.clone().into());
|
||||
tracing_subscriber::registry()
|
||||
.with(filter)
|
||||
.with(tracing_subscriber::fmt::layer())
|
||||
// second copy of every line, without ANSI, into the in-memory ring
|
||||
// that backs the admin console's Логи view
|
||||
.with(
|
||||
tracing_subscriber::fmt::layer()
|
||||
.with_ansi(false)
|
||||
.with_writer(logbuf::RingMakeWriter),
|
||||
)
|
||||
.init();
|
||||
}
|
||||
|
||||
let listen = cfg.listen.clone();
|
||||
let admin_enabled = cfg.admin.enabled;
|
||||
let (admin_token, generated) = resolve_admin_token(&cfg, &args);
|
||||
let admin_token = resolve_admin_token(&cfg, &args);
|
||||
let first_run = needs_first_run_wizard(&cfg, &config_path);
|
||||
|
||||
let state = SharedState::build(cfg, args.config.clone())?;
|
||||
let state = SharedState::build(cfg, config_path.clone())?;
|
||||
|
||||
// Open admin console reachable from other machines: fine on a laptop,
|
||||
// a footgun on a VPS. Flag it for the UI banner and say it in the log.
|
||||
let host_public = listen.starts_with("0.0.0.0") || listen.starts_with("[::]");
|
||||
if admin_enabled && admin_token.is_empty() && host_public && !is_docker() {
|
||||
state
|
||||
.admin_open_public
|
||||
.store(true, std::sync::atomic::Ordering::Relaxed);
|
||||
tracing::warn!(
|
||||
"admin console is OPEN (no token) and listening on {listen} — anyone on your network can manage this gateway; set a token via --admin-token or [admin].token if this machine is reachable from outside"
|
||||
);
|
||||
}
|
||||
|
||||
// Close the loop with the CMF format (github.com/infosave2007/cmf): install
|
||||
// / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
|
||||
// the background so startup is never blocked by a cargo install or a model
|
||||
// load; the served model is registered in the pool and usable offline.
|
||||
{
|
||||
let cmf_rt = state.cmf.clone();
|
||||
let cmf_cfg = state.live().cfg.cmf.clone();
|
||||
tokio::spawn(async move { cmf_runtime::manage(cmf_rt, cmf_cfg).await });
|
||||
}
|
||||
|
||||
let mut app = Router::new()
|
||||
.route("/healthz", get(|| async { "ok" }))
|
||||
.route("/health", get(|| async { "ok" }))
|
||||
.route("/readyz", get(|| async { "ready" }))
|
||||
.route("/metrics", get(admin::metrics))
|
||||
.merge(protocols::build_router());
|
||||
.merge(protocols::build_router(state.clone()));
|
||||
|
||||
if admin_enabled {
|
||||
app = app
|
||||
@@ -91,18 +266,92 @@ async fn main() -> anyhow::Result<()> {
|
||||
.fallback(admin::assets::fallback);
|
||||
}
|
||||
|
||||
let app = app.with_state(state);
|
||||
let cors = tower_http::cors::CorsLayer::new()
|
||||
.allow_origin(tower_http::cors::Any)
|
||||
.allow_methods(tower_http::cors::Any)
|
||||
.allow_headers(tower_http::cors::Any);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(&listen).await?;
|
||||
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
|
||||
// small enough that a hostile client can't balloon memory
|
||||
let app =
|
||||
app.with_state(state)
|
||||
.layer(cors)
|
||||
.layer(tower_http::limit::RequestBodyLimitLayer::new(
|
||||
32 * 1024 * 1024,
|
||||
));
|
||||
|
||||
let listener = {
|
||||
let mut attempt = 0u32;
|
||||
loop {
|
||||
match tokio::net::TcpListener::bind(&listen).await {
|
||||
Ok(l) => break l,
|
||||
Err(e) => {
|
||||
// during self-restart the old process is still letting go of
|
||||
// the port — retry for a few seconds before giving up
|
||||
let retrying = std::env::var("CORTIQ_BIND_RETRY").is_ok();
|
||||
if retrying && attempt < 50 {
|
||||
attempt += 1;
|
||||
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
|
||||
continue;
|
||||
}
|
||||
return Err(e.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
tracing::info!("cortiq-gateway listening on {listen}");
|
||||
if admin_enabled {
|
||||
tracing::info!("admin console: http://{listen}/admin");
|
||||
if generated {
|
||||
tracing::warn!(
|
||||
"admin token (auto-generated, set [admin].token_env to persist): {admin_token}"
|
||||
);
|
||||
let host_for_url = if listen.starts_with("0.0.0.0:") {
|
||||
listen.replacen("0.0.0.0", "127.0.0.1", 1)
|
||||
} else {
|
||||
listen.clone()
|
||||
};
|
||||
// On first run open the onboarding wizard, otherwise the normal console.
|
||||
let admin_path = if first_run {
|
||||
"/admin#/onboarding"
|
||||
} else {
|
||||
"/admin"
|
||||
};
|
||||
let url = if admin_token.is_empty() {
|
||||
format!("http://{host_for_url}{admin_path}")
|
||||
} else {
|
||||
format!("http://{host_for_url}{admin_path}?token={admin_token}")
|
||||
};
|
||||
if admin_token.is_empty() {
|
||||
tracing::info!("admin console (open, no token): http://{host_for_url}/admin");
|
||||
} else {
|
||||
tracing::info!("admin console: http://{host_for_url}/admin");
|
||||
}
|
||||
if first_run {
|
||||
tracing::info!("first run — opening onboarding wizard at {url}");
|
||||
}
|
||||
if admin_enabled && should_open_browser(&args) {
|
||||
let browser_url = url.clone();
|
||||
tokio::spawn(async move {
|
||||
// give server a moment to be ready
|
||||
tokio::time::sleep(std::time::Duration::from_millis(600)).await;
|
||||
#[cfg(feature = "open-browser")]
|
||||
{
|
||||
if let Err(e) = open::that(&browser_url) {
|
||||
tracing::warn!("failed to open browser: {e}");
|
||||
} else {
|
||||
tracing::info!("opened browser at {browser_url}");
|
||||
}
|
||||
}
|
||||
#[cfg(not(feature = "open-browser"))]
|
||||
{
|
||||
tracing::info!("open browser: {browser_url} (feature disabled)");
|
||||
}
|
||||
});
|
||||
} else if first_run {
|
||||
tracing::info!("open this URL to finish setup: {url}");
|
||||
}
|
||||
}
|
||||
axum::serve(listener, app).await?;
|
||||
axum::serve(listener, app)
|
||||
.with_graceful_shutdown(async {
|
||||
let _ = tokio::signal::ctrl_c().await;
|
||||
tracing::info!("shutdown signal received — draining connections");
|
||||
})
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
+861
@@ -0,0 +1,861 @@
|
||||
//! Media generation: run `cortiq imagine` (text→image, Lumina) and
|
||||
//! `cortiq animate` (text→video+audio, MiniMax-H3) over local `.cmf` files as
|
||||
//! tracked background jobs, the same shape as import jobs so the admin UI can
|
||||
//! poll progress and fetch the result.
|
||||
//!
|
||||
//! These models are NOT served by `cortiq serve` — each generation is one CLI
|
||||
//! run. The gateway owns the process, parses its progress, and keeps the output
|
||||
//! under `<data>/media/<job>/`.
|
||||
|
||||
use serde::Serialize;
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Stdio;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
use tokio::io::{AsyncBufReadExt, BufReader};
|
||||
use tokio::sync::Notify;
|
||||
|
||||
fn now() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Random job id — media jobs are never deduplicated (every run is a new work).
|
||||
fn gen_media_id() -> String {
|
||||
let mut b = [0u8; 8];
|
||||
let _ = getrandom::getrandom(&mut b);
|
||||
b.iter().map(|x| format!("{x:02x}")).collect()
|
||||
}
|
||||
|
||||
/// What a `.cmf` under models_dir can do, decided by `cortiq info`.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, Debug, Serialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum Modality {
|
||||
Chat,
|
||||
Image,
|
||||
Video,
|
||||
}
|
||||
|
||||
/// `cortiq info` output → modality. The arch line names the family; the media
|
||||
/// packs also carry their pipeline components (DiT/VAE/vocoder) in the text.
|
||||
fn classify_info(info: &str) -> Modality {
|
||||
let lower = info.to_ascii_lowercase();
|
||||
let arch = lower
|
||||
.lines()
|
||||
.find_map(|l| l.trim().strip_prefix("arch:"))
|
||||
.unwrap_or("")
|
||||
.trim()
|
||||
.to_string();
|
||||
// generic arch suffixes first (e.g. "lumina2-image", "mmh3-video"), then
|
||||
// known family names as a fallback for packs without the suffix
|
||||
if arch.contains("video") || arch.contains("mmh3") || arch.contains("minimax") {
|
||||
return Modality::Video;
|
||||
}
|
||||
if arch.contains("image") || arch.contains("lumina") || arch.contains("diffusion") {
|
||||
return Modality::Image;
|
||||
}
|
||||
if lower.contains("text-to-video") {
|
||||
return Modality::Video;
|
||||
}
|
||||
if lower.contains("text-to-image") {
|
||||
return Modality::Image;
|
||||
}
|
||||
Modality::Chat
|
||||
}
|
||||
|
||||
/// One local `.cmf` with its detected modality.
|
||||
#[derive(Clone, Serialize)]
|
||||
pub struct MediaModel {
|
||||
pub name: String, // file stem
|
||||
pub path: String,
|
||||
pub size_bytes: u64,
|
||||
pub modality: Modality,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct InfoCache(Mutex<HashMap<String, (u64, u64, Modality)>>); // path → (mtime, size, modality)
|
||||
|
||||
/// Modality of one `.cmf`, via `cortiq info` — cached by (path, mtime, size),
|
||||
/// so the process runs once per file version.
|
||||
pub async fn classify_file(path: &Path, cortiq_bin: &str) -> Modality {
|
||||
static CACHE: std::sync::OnceLock<InfoCache> = std::sync::OnceLock::new();
|
||||
let cache = CACHE.get_or_init(InfoCache::default);
|
||||
|
||||
let size = crate::import::path_size(path);
|
||||
let mtime = std::fs::metadata(path)
|
||||
.ok()
|
||||
.and_then(|m| m.modified().ok())
|
||||
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0);
|
||||
let key = path.to_string_lossy().to_string();
|
||||
if let Some(k) = cache
|
||||
.0
|
||||
.lock()
|
||||
.unwrap()
|
||||
.get(&key)
|
||||
.filter(|(m, s, _)| *m == mtime && *s == size)
|
||||
.map(|(_, _, k)| *k)
|
||||
{
|
||||
return k;
|
||||
}
|
||||
let output = tokio::process::Command::new(cortiq_bin)
|
||||
.arg("info")
|
||||
.arg(path)
|
||||
.output()
|
||||
.await;
|
||||
let text = output
|
||||
.map(|o| {
|
||||
format!(
|
||||
"{}{}",
|
||||
String::from_utf8_lossy(&o.stdout),
|
||||
String::from_utf8_lossy(&o.stderr)
|
||||
)
|
||||
})
|
||||
.unwrap_or_default();
|
||||
let k = classify_info(&text);
|
||||
cache.0.lock().unwrap().insert(key, (mtime, size, k));
|
||||
k
|
||||
}
|
||||
|
||||
/// Scan models_dir and classify every `.cmf` (file or sharded dir).
|
||||
pub async fn list_media_models(models_dir: &str, cortiq_bin: &str) -> Vec<MediaModel> {
|
||||
let mut out = Vec::new();
|
||||
let Ok(rd) = std::fs::read_dir(models_dir) else {
|
||||
return out;
|
||||
};
|
||||
for entry in rd.filter_map(|e| e.ok()) {
|
||||
let path = entry.path();
|
||||
let name = path
|
||||
.file_name()
|
||||
.unwrap_or_default()
|
||||
.to_string_lossy()
|
||||
.to_string();
|
||||
if !name.to_ascii_lowercase().ends_with(".cmf") {
|
||||
continue;
|
||||
}
|
||||
let size = crate::import::path_size(&path);
|
||||
if size == 0 {
|
||||
continue; // empty or still being written
|
||||
}
|
||||
let modality = classify_file(&path, cortiq_bin).await;
|
||||
out.push(MediaModel {
|
||||
name: name
|
||||
.trim_end_matches(".cmf")
|
||||
.trim_end_matches(".CMF")
|
||||
.to_string(),
|
||||
path: path.to_string_lossy().to_string(),
|
||||
size_bytes: size,
|
||||
modality,
|
||||
});
|
||||
}
|
||||
out.sort_by(|a, b| a.name.cmp(&b.name));
|
||||
out
|
||||
}
|
||||
|
||||
/// Parameters for one generation, straight from the playground form.
|
||||
#[derive(Clone, Debug, serde::Deserialize)]
|
||||
pub struct MediaParams {
|
||||
pub model: String, // file stem under models_dir
|
||||
pub kind: String, // image | video
|
||||
pub prompt: String,
|
||||
#[serde(default)]
|
||||
pub width: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub height: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub steps: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub cfg: Option<f32>, // image only
|
||||
#[serde(default)]
|
||||
pub seed: Option<u64>,
|
||||
#[serde(default)]
|
||||
pub frames: Option<u32>, // video only
|
||||
#[serde(default)]
|
||||
pub quality: Option<u32>, // video only: AVI JPEG quality
|
||||
/// Optional first/last keyframe for video, base64-encoded binary P6 PPM.
|
||||
#[serde(default)]
|
||||
pub first_frame_b64: Option<String>,
|
||||
#[serde(default)]
|
||||
pub last_frame_b64: Option<String>,
|
||||
}
|
||||
|
||||
/// Live state of one generation.
|
||||
#[derive(Clone, Serialize, serde::Deserialize)]
|
||||
pub struct MediaJob {
|
||||
pub id: String,
|
||||
pub kind: String, // image | video
|
||||
pub model: String,
|
||||
pub prompt: String,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub steps: u32,
|
||||
pub seed: u64,
|
||||
pub frames: Option<u32>,
|
||||
pub state: String, // queued | running | done | error | cancelled
|
||||
pub progress: Option<f32>,
|
||||
pub phase: Option<String>,
|
||||
pub log: Vec<String>,
|
||||
pub started: u64,
|
||||
pub finished: Option<u64>,
|
||||
/// Files this job produced: name → relative kind ("image", "video", "audio", "mp4").
|
||||
pub outputs: Vec<String>,
|
||||
pub out_bytes: Option<u64>,
|
||||
}
|
||||
|
||||
/// Everything needed to actually launch a queued job when its turn comes.
|
||||
struct Prepared {
|
||||
bin: String,
|
||||
args: Vec<String>,
|
||||
gpu: bool,
|
||||
dir: PathBuf,
|
||||
out_path: PathBuf,
|
||||
kind: String,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct MediaStore {
|
||||
jobs: Mutex<HashMap<String, MediaJob>>,
|
||||
cancels: Mutex<HashMap<String, Arc<Notify>>>,
|
||||
/// One generation runs at a time — these models take the whole GPU/RAM
|
||||
/// budget — but new requests queue up instead of being rejected.
|
||||
queue: Mutex<std::collections::VecDeque<String>>,
|
||||
current: Mutex<Option<String>>,
|
||||
prepared: Mutex<HashMap<String, Prepared>>,
|
||||
persist: Mutex<Option<PathBuf>>,
|
||||
}
|
||||
|
||||
impl MediaStore {
|
||||
pub fn new() -> Arc<Self> {
|
||||
Arc::new(Self::default())
|
||||
}
|
||||
/// Load saved generations (their files are still under media_dir) and start
|
||||
/// persisting. Jobs that were queued/running when the gateway died settle
|
||||
/// as errors — the process is gone.
|
||||
pub fn attach_persistence(&self, path: PathBuf) {
|
||||
if let Ok(text) = std::fs::read_to_string(&path) {
|
||||
if let Ok(mut jobs) = serde_json::from_str::<Vec<MediaJob>>(&text) {
|
||||
let mut g = self.jobs.lock().unwrap();
|
||||
for mut j in jobs.drain(..) {
|
||||
if j.state == "running" || j.state == "queued" {
|
||||
j.state = "error".into();
|
||||
j.log.push("✗ interrupted by a gateway restart".into());
|
||||
j.finished = Some(now());
|
||||
}
|
||||
g.insert(j.id.clone(), j);
|
||||
}
|
||||
}
|
||||
}
|
||||
*self.persist.lock().unwrap() = Some(path);
|
||||
}
|
||||
fn save(&self) {
|
||||
let Some(path) = self.persist.lock().unwrap().clone() else {
|
||||
return;
|
||||
};
|
||||
let jobs: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
|
||||
if let Ok(json) = serde_json::to_vec(&jobs) {
|
||||
let tmp = path.with_extension("json.tmp");
|
||||
if std::fs::write(&tmp, json).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &path);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// How many jobs sit in the queue ahead of `id` (0 = next up).
|
||||
pub fn queue_position(&self, id: &str) -> Option<usize> {
|
||||
self.queue.lock().unwrap().iter().position(|x| x == id)
|
||||
}
|
||||
pub fn list(&self) -> Vec<MediaJob> {
|
||||
let mut v: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
|
||||
v.sort_by_key(|j| std::cmp::Reverse(j.started));
|
||||
v.truncate(20);
|
||||
v
|
||||
}
|
||||
pub fn get(&self, id: &str) -> Option<MediaJob> {
|
||||
self.jobs.lock().unwrap().get(id).cloned()
|
||||
}
|
||||
pub fn cancel(self: &Arc<Self>, id: &str, media_dir: &Path) -> bool {
|
||||
let state = self
|
||||
.jobs
|
||||
.lock()
|
||||
.unwrap()
|
||||
.get(id)
|
||||
.map(|j| j.state.clone())
|
||||
.unwrap_or_default();
|
||||
match state.as_str() {
|
||||
"queued" => {
|
||||
self.queue.lock().unwrap().retain(|x| x != id);
|
||||
self.prepared.lock().unwrap().remove(id);
|
||||
let _ = std::fs::remove_dir_all(media_dir.join(id));
|
||||
self.set_cancelled(id);
|
||||
true
|
||||
}
|
||||
"running" => {
|
||||
if let Some(n) = self.cancels.lock().unwrap().get(id) {
|
||||
n.notify_one();
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
/// Remove a settled job and its output directory. Err = still running.
|
||||
pub fn delete(&self, id: &str, media_dir: &Path) -> Result<bool, String> {
|
||||
let job = self.jobs.lock().unwrap().get(id).cloned();
|
||||
let Some(job) = job else { return Ok(false) };
|
||||
if job.state == "running" || job.state == "queued" {
|
||||
return Err("cancel the running generation first".into());
|
||||
}
|
||||
let _ = std::fs::remove_dir_all(media_dir.join(id));
|
||||
self.jobs.lock().unwrap().remove(id);
|
||||
self.save();
|
||||
Ok(true)
|
||||
}
|
||||
fn push_line(&self, id: &str, line: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.log.push(line);
|
||||
let n = j.log.len();
|
||||
if n > 120 {
|
||||
j.log.drain(0..n - 120);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn set_progress(&self, id: &str, frac: f32, phase: Option<String>) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
let f = frac.clamp(0.0, 1.0);
|
||||
if j.progress.is_none() || f > j.progress.unwrap() {
|
||||
j.progress = Some(f);
|
||||
}
|
||||
if let Some(p) = phase {
|
||||
j.phase = Some(p);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn finish(self: &Arc<Self>, id: &str, ok: bool, outputs: Vec<String>, out_bytes: u64) {
|
||||
let mut g = self.jobs.lock().unwrap();
|
||||
if let Some(j) = g.get_mut(id) {
|
||||
if j.state == "cancelled" {
|
||||
drop(g);
|
||||
self.after_slot_freed(id);
|
||||
return;
|
||||
}
|
||||
j.finished = Some(now());
|
||||
j.state = if ok { "done" } else { "error" }.into();
|
||||
if ok {
|
||||
j.progress = Some(1.0);
|
||||
j.outputs = outputs;
|
||||
j.out_bytes = Some(out_bytes);
|
||||
}
|
||||
}
|
||||
drop(g);
|
||||
self.after_slot_freed(id);
|
||||
}
|
||||
fn set_cancelled(self: &Arc<Self>, id: &str) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.state = "cancelled".into();
|
||||
j.finished = Some(now());
|
||||
}
|
||||
self.after_slot_freed(id);
|
||||
}
|
||||
fn after_slot_freed(self: &Arc<Self>, id: &str) {
|
||||
{
|
||||
let mut cur = self.current.lock().unwrap();
|
||||
if cur.as_deref() == Some(id) {
|
||||
*cur = None;
|
||||
}
|
||||
}
|
||||
self.cancels.lock().unwrap().remove(id);
|
||||
self.save();
|
||||
promote_next(self.clone());
|
||||
}
|
||||
}
|
||||
|
||||
/// Start the next queued generation, if the slot is free.
|
||||
fn promote_next(store: Arc<MediaStore>) {
|
||||
let next = {
|
||||
let mut cur = store.current.lock().unwrap();
|
||||
if cur.is_some() {
|
||||
return;
|
||||
}
|
||||
let Some(id) = store.queue.lock().unwrap().pop_front() else {
|
||||
return;
|
||||
};
|
||||
*cur = Some(id.clone());
|
||||
id
|
||||
};
|
||||
let Some(p) = store.prepared.lock().unwrap().remove(&next) else {
|
||||
// job was cancelled while queued and slipped through — free the slot
|
||||
*store.current.lock().unwrap() = None;
|
||||
return;
|
||||
};
|
||||
if let Some(j) = store.jobs.lock().unwrap().get_mut(&next) {
|
||||
j.state = "running".into();
|
||||
j.phase = Some("starting".into());
|
||||
j.started = now();
|
||||
}
|
||||
store.save();
|
||||
let cancel = Arc::new(Notify::new());
|
||||
store
|
||||
.cancels
|
||||
.lock()
|
||||
.unwrap()
|
||||
.insert(next.clone(), cancel.clone());
|
||||
let s = store.clone();
|
||||
tokio::spawn(async move {
|
||||
run_generation(
|
||||
s, next, p.bin, p.args, p.gpu, p.dir, p.out_path, p.kind, cancel,
|
||||
)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
|
||||
/// Progress markers in `cortiq imagine`/`animate` stdout. The exact wording has
|
||||
/// varied between releases, so accept anything that carries an X/Y pair near a
|
||||
/// step/frame word, a bare percentage, or the converter-style `@PROGRESS`.
|
||||
fn parse_media_progress(line: &str) -> Option<(f32, Option<String>)> {
|
||||
let l = line.trim();
|
||||
if let Some(rest) = l.strip_prefix("@PROGRESS ") {
|
||||
let mut it = rest.splitn(2, ' ');
|
||||
let frac: f32 = it.next()?.trim().parse().ok()?;
|
||||
let phase = it
|
||||
.next()
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty());
|
||||
return Some((frac.clamp(0.0, 1.0), phase));
|
||||
}
|
||||
let lower = l.to_ascii_lowercase();
|
||||
// "step 12/30", "denoise 3/4", "frame 7/39" — any word followed by x/y
|
||||
for word in ["step", "frame", "sigma", "denois"] {
|
||||
if let Some(pos) = lower.find(word) {
|
||||
let tail = &lower[pos..];
|
||||
if let Some((x, y)) = extract_ratio(tail) {
|
||||
return Some((x / y, Some(l.to_string())));
|
||||
}
|
||||
}
|
||||
}
|
||||
// bare "42%" (avoid matching sizes like "42%faster")
|
||||
if let Some(idx) = lower.find('%') {
|
||||
let head: String = lower[..idx]
|
||||
.chars()
|
||||
.rev()
|
||||
.take_while(|c| c.is_ascii_digit() || *c == '.')
|
||||
.collect();
|
||||
let num: String = head.chars().rev().collect();
|
||||
if let Ok(p) = num.parse::<f32>() {
|
||||
if (0.0..=100.0).contains(&p) {
|
||||
return Some((p / 100.0, None));
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// First "N/M" pair in the string with M > 0.
|
||||
fn extract_ratio(s: &str) -> Option<(f32, f32)> {
|
||||
let bytes = s.as_bytes();
|
||||
let mut i = 0;
|
||||
while i < bytes.len() {
|
||||
if bytes[i].is_ascii_digit() {
|
||||
let start = i;
|
||||
while i < bytes.len() && bytes[i].is_ascii_digit() {
|
||||
i += 1;
|
||||
}
|
||||
if i < bytes.len() && bytes[i] == b'/' {
|
||||
let x: f32 = s[start..i].parse().ok()?;
|
||||
let ystart = i + 1;
|
||||
let mut j = ystart;
|
||||
while j < bytes.len() && bytes[j].is_ascii_digit() {
|
||||
j += 1;
|
||||
}
|
||||
if j > ystart {
|
||||
if let Ok(y) = s[ystart..j].parse::<f32>() {
|
||||
if y > 0.0 && x <= y {
|
||||
return Some((x, y));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Root for generated files: `<data>/media`.
|
||||
pub fn media_dir() -> PathBuf {
|
||||
crate::config::data_dir().join("media")
|
||||
}
|
||||
|
||||
const B64: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
/// Minimal base64 decode (standard alphabet, padding optional) — the keyframes
|
||||
/// are the only base64 input the gateway takes; not worth a dependency.
|
||||
fn b64_decode(s: &str) -> Option<Vec<u8>> {
|
||||
let mut lut = [255u8; 256];
|
||||
for (i, c) in B64.iter().enumerate() {
|
||||
lut[*c as usize] = i as u8;
|
||||
}
|
||||
let mut out = Vec::with_capacity(s.len() * 3 / 4);
|
||||
let mut buf = 0u32;
|
||||
let mut bits = 0u32;
|
||||
for c in s.bytes() {
|
||||
if c == b'=' || c == b'\n' || c == b'\r' {
|
||||
continue;
|
||||
}
|
||||
let v = lut[c as usize];
|
||||
if v == 255 {
|
||||
return None;
|
||||
}
|
||||
buf = (buf << 6) | v as u32;
|
||||
bits += 6;
|
||||
if bits >= 8 {
|
||||
bits -= 8;
|
||||
out.push((buf >> bits) as u8);
|
||||
}
|
||||
}
|
||||
Some(out)
|
||||
}
|
||||
|
||||
/// Kick off a generation. Returns the job id, or an error when another one is
|
||||
/// already running / the model file is missing.
|
||||
pub fn start_generation(
|
||||
store: Arc<MediaStore>,
|
||||
cmf: &crate::config::CmfCfg,
|
||||
p: MediaParams,
|
||||
) -> Result<String, String> {
|
||||
if p.prompt.trim().is_empty() {
|
||||
return Err("empty prompt".into());
|
||||
}
|
||||
let kind = p.kind.as_str();
|
||||
if kind != "image" && kind != "video" {
|
||||
return Err(format!("unknown kind '{kind}'"));
|
||||
}
|
||||
let model_path = Path::new(&cmf.models_dir).join(format!("{}.cmf", p.model));
|
||||
if !model_path.exists() {
|
||||
return Err(format!("model file not found: {}", model_path.display()));
|
||||
}
|
||||
|
||||
let id = gen_media_id();
|
||||
let dir = media_dir().join(&id);
|
||||
if let Err(e) = std::fs::create_dir_all(&dir) {
|
||||
return Err(format!("cannot create media dir {}: {e}", dir.display()));
|
||||
}
|
||||
|
||||
// Optional keyframes (video): decode to PPM files next to the output.
|
||||
let mut first_frame: Option<PathBuf> = None;
|
||||
let mut last_frame: Option<PathBuf> = None;
|
||||
for (b64, slot, name) in [
|
||||
(&p.first_frame_b64, &mut first_frame, "first.ppm"),
|
||||
(&p.last_frame_b64, &mut last_frame, "last.ppm"),
|
||||
] {
|
||||
if let Some(data) = b64.as_deref().filter(|s| !s.is_empty()) {
|
||||
let bytes = b64_decode(data).ok_or("invalid keyframe base64")?;
|
||||
if !bytes.starts_with(b"P6") {
|
||||
return Err("keyframe must be a binary P6 PPM".into());
|
||||
}
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, bytes).map_err(|e| format!("write keyframe: {e}"))?;
|
||||
*slot = Some(path);
|
||||
}
|
||||
}
|
||||
|
||||
let width = p.width.unwrap_or(512).clamp(64, 2048);
|
||||
let height = p
|
||||
.height
|
||||
.unwrap_or(if kind == "video" { 288 } else { 512 })
|
||||
.clamp(64, 2048);
|
||||
let steps = p
|
||||
.steps
|
||||
.unwrap_or(if kind == "video" { 4 } else { 30 })
|
||||
.clamp(1, 200);
|
||||
let seed = p.seed.unwrap_or(42);
|
||||
let frames = p.frames.unwrap_or(39).clamp(1, 1000);
|
||||
|
||||
let out_name = if kind == "image" {
|
||||
"out.ppm"
|
||||
} else {
|
||||
"out.avi"
|
||||
};
|
||||
let out_path = dir.join(out_name);
|
||||
|
||||
let mut args: Vec<String> = if kind == "image" {
|
||||
vec![
|
||||
"imagine".into(),
|
||||
model_path.to_string_lossy().to_string(),
|
||||
"--prompt".into(),
|
||||
p.prompt.clone(),
|
||||
"--width".into(),
|
||||
width.to_string(),
|
||||
"--height".into(),
|
||||
height.to_string(),
|
||||
"--steps".into(),
|
||||
steps.to_string(),
|
||||
"--cfg".into(),
|
||||
format!("{}", p.cfg.unwrap_or(4.0)),
|
||||
"--seed".into(),
|
||||
seed.to_string(),
|
||||
"--out".into(),
|
||||
out_path.to_string_lossy().to_string(),
|
||||
]
|
||||
} else {
|
||||
let mut a = vec![
|
||||
"animate".into(),
|
||||
model_path.to_string_lossy().to_string(),
|
||||
"--prompt".into(),
|
||||
p.prompt.clone(),
|
||||
"--width".into(),
|
||||
width.to_string(),
|
||||
"--height".into(),
|
||||
height.to_string(),
|
||||
"--frames".into(),
|
||||
frames.to_string(),
|
||||
"--steps".into(),
|
||||
steps.to_string(),
|
||||
"--seed".into(),
|
||||
seed.to_string(),
|
||||
"--quality".into(),
|
||||
p.quality.unwrap_or(92).clamp(10, 100).to_string(),
|
||||
"--out".into(),
|
||||
out_path.to_string_lossy().to_string(),
|
||||
];
|
||||
if let Some(f) = &first_frame {
|
||||
a.push("--first-frame".into());
|
||||
a.push(f.to_string_lossy().to_string());
|
||||
}
|
||||
if let Some(f) = &last_frame {
|
||||
a.push("--last-frame".into());
|
||||
a.push(f.to_string_lossy().to_string());
|
||||
}
|
||||
a
|
||||
};
|
||||
// keep the model path first for readability in logs
|
||||
let _ = &mut args;
|
||||
|
||||
store.jobs.lock().unwrap().insert(
|
||||
id.clone(),
|
||||
MediaJob {
|
||||
id: id.clone(),
|
||||
kind: kind.to_string(),
|
||||
model: p.model.clone(),
|
||||
prompt: p.prompt.clone(),
|
||||
width,
|
||||
height,
|
||||
steps,
|
||||
seed,
|
||||
frames: (kind == "video").then_some(frames),
|
||||
state: "queued".into(),
|
||||
progress: None,
|
||||
phase: None,
|
||||
log: vec![format!("→ cortiq {}", args.join(" "))],
|
||||
started: now(),
|
||||
finished: None,
|
||||
outputs: Vec::new(),
|
||||
out_bytes: None,
|
||||
},
|
||||
);
|
||||
store.prepared.lock().unwrap().insert(
|
||||
id.clone(),
|
||||
Prepared {
|
||||
bin: cmf.cortiq_bin.clone(),
|
||||
args,
|
||||
gpu: cmf.gpu,
|
||||
dir,
|
||||
out_path,
|
||||
kind: kind.to_string(),
|
||||
},
|
||||
);
|
||||
store.queue.lock().unwrap().push_back(id.clone());
|
||||
store.save();
|
||||
promote_next(store.clone());
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn run_generation(
|
||||
store: Arc<MediaStore>,
|
||||
id: String,
|
||||
bin: String,
|
||||
args: Vec<String>,
|
||||
gpu: bool,
|
||||
dir: PathBuf,
|
||||
out_path: PathBuf,
|
||||
kind: String,
|
||||
cancel: Arc<Notify>,
|
||||
) {
|
||||
let mut cmd = tokio::process::Command::new(&bin);
|
||||
cmd.args(&args)
|
||||
.env("CMF_GPU", if gpu { "1" } else { "0" })
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped());
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed to start cortiq: {e}"));
|
||||
store.finish(&id, false, Vec::new(), 0);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let mut readers = Vec::new();
|
||||
if let Some(o) = child.stdout.take() {
|
||||
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), o)));
|
||||
}
|
||||
if let Some(e) = child.stderr.take() {
|
||||
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), e)));
|
||||
}
|
||||
|
||||
let status = tokio::select! {
|
||||
s = child.wait() => s,
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
for r in readers { r.abort(); }
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
store.push_line(&id, "✗ cancelled".into());
|
||||
store.set_cancelled(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
for r in readers {
|
||||
let _ = r.await;
|
||||
}
|
||||
|
||||
let ok = status.map(|s| s.success()).unwrap_or(false);
|
||||
if !ok || !out_path.exists() {
|
||||
store.push_line(&id, "✗ generation failed — see the log above".into());
|
||||
store.finish(&id, false, Vec::new(), 0);
|
||||
return;
|
||||
}
|
||||
|
||||
let mut outputs = vec![if kind == "image" { "image" } else { "video" }.to_string()];
|
||||
if kind == "video" {
|
||||
// `animate` writes a .wav next to the .avi
|
||||
if dir.join("out.wav").exists() {
|
||||
outputs.push("audio".into());
|
||||
}
|
||||
// opportunistic browser-friendly remux when ffmpeg is present
|
||||
if let Ok(p) = which_ffmpeg() {
|
||||
store.push_line(&id, "→ remuxing to mp4 (ffmpeg found)".into());
|
||||
let mp4 = dir.join("out.mp4");
|
||||
let mut f = tokio::process::Command::new(p);
|
||||
f.arg("-y").arg("-i").arg(dir.join("out.avi"));
|
||||
if dir.join("out.wav").exists() {
|
||||
f.arg("-i")
|
||||
.arg(dir.join("out.wav"))
|
||||
.args(["-map", "0:v", "-map", "1:a", "-c:a", "aac"]);
|
||||
}
|
||||
f.args(["-c:v", "libx264", "-pix_fmt", "yuv420p"]).arg(&mp4);
|
||||
f.stdout(Stdio::null()).stderr(Stdio::null());
|
||||
if matches!(f.status().await, Ok(s) if s.success()) && mp4.exists() {
|
||||
outputs.push("mp4".into());
|
||||
}
|
||||
}
|
||||
}
|
||||
let bytes = crate::import::path_size(&out_path);
|
||||
store.push_line(
|
||||
&id,
|
||||
format!("✓ done — {}", crate::import::format_bytes(bytes)),
|
||||
);
|
||||
store.finish(&id, true, outputs, bytes);
|
||||
}
|
||||
|
||||
/// Strip ANSI color escapes that `tracing` writes even into a pipe.
|
||||
fn strip_ansi(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
let mut chars = s.chars();
|
||||
while let Some(c) = chars.next() {
|
||||
if c == '\u{1b}' {
|
||||
for d in chars.by_ref() {
|
||||
if d.is_ascii_alphabetic() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
out.push(c);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
async fn read_lines(store: Arc<MediaStore>, id: String, stream: impl tokio::io::AsyncRead + Unpin) {
|
||||
let mut lines = BufReader::new(stream).lines();
|
||||
while let Ok(Some(l)) = lines.next_line().await {
|
||||
let l = strip_ansi(l.trim());
|
||||
if l.is_empty() {
|
||||
continue;
|
||||
}
|
||||
// the Metal backend logs one line per pipeline — dozens per run, all identical
|
||||
if l.contains("Metal GPU path") {
|
||||
continue;
|
||||
}
|
||||
if let Some((frac, phase)) = parse_media_progress(&l) {
|
||||
store.set_progress(&id, frac, phase);
|
||||
} else {
|
||||
store.push_line(&id, l);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn which_ffmpeg() -> Result<String, ()> {
|
||||
for p in [
|
||||
"ffmpeg",
|
||||
"/usr/local/bin/ffmpeg",
|
||||
"/opt/homebrew/bin/ffmpeg",
|
||||
] {
|
||||
if std::process::Command::new(p)
|
||||
.arg("-version")
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.status()
|
||||
.map(|s| s.success())
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return Ok(p.to_string());
|
||||
}
|
||||
}
|
||||
Err(())
|
||||
}
|
||||
|
||||
/// (relative file, content-type) for an output kind of a finished job.
|
||||
pub fn output_file(kind: &str) -> Option<(&'static str, &'static str)> {
|
||||
match kind {
|
||||
"image" => Some(("out.ppm", "image/x-portable-pixmap")),
|
||||
"video" => Some(("out.avi", "video/x-msvideo")),
|
||||
"audio" => Some(("out.wav", "audio/wav")),
|
||||
"mp4" => Some(("out.mp4", "video/mp4")),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn progress_parses_common_shapes() {
|
||||
assert_eq!(parse_media_progress("step 15/30").unwrap().0, 0.5);
|
||||
assert_eq!(
|
||||
parse_media_progress("Denoising step 3/4 ...").unwrap().0,
|
||||
0.75
|
||||
);
|
||||
assert_eq!(parse_media_progress("frame 39/39 decoded").unwrap().0, 1.0);
|
||||
assert_eq!(parse_media_progress("@PROGRESS 0.25 vae").unwrap().0, 0.25);
|
||||
assert!(parse_media_progress("loaded 2361 tensors").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_by_arch() {
|
||||
assert_eq!(classify_info(" Arch: lumina2\n"), Modality::Image);
|
||||
assert_eq!(classify_info(" Arch: mmh3\n"), Modality::Video);
|
||||
assert_eq!(classify_info(" Arch: qwen3\n"), Modality::Chat);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn b64_roundtrip() {
|
||||
assert_eq!(b64_decode("UDYgd2g=").unwrap(), b"P6 wh");
|
||||
assert!(b64_decode("!!").is_none());
|
||||
}
|
||||
}
|
||||
+43
-2
@@ -15,13 +15,53 @@ pub enum RoutingDirective {
|
||||
Pinned { model_id: String },
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub role: String, // system | user | assistant | tool
|
||||
#[serde(default)]
|
||||
/// Always a plain string internally. On the wire OpenAI clients legitimately
|
||||
/// send `null` (assistant turns that only carry tool calls) or an array of
|
||||
/// content parts (Open WebUI and every other multimodal UI), so
|
||||
/// [`de_content`] flattens both shapes instead of rejecting the request.
|
||||
#[serde(default, deserialize_with = "de_content")]
|
||||
pub content: String,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub tool_calls: Vec<serde_json::Value>,
|
||||
/// Speaker name (`messages[].name`) — proxied through when the client sends it.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
/// Set on `role = "tool"` replies; dropping it breaks multi-turn tool calling.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tool_call_id: Option<String>,
|
||||
}
|
||||
|
||||
/// Deserialize `messages[].content` from any shape an OpenAI-compatible client
|
||||
/// may send: a string, `null`, or an array of content parts. Text parts are
|
||||
/// concatenated; non-text parts (images, audio, files) are skipped — this
|
||||
/// gateway routes text, and a picture must not turn into a 422.
|
||||
fn de_content<'de, D>(d: D) -> std::result::Result<String, D::Error>
|
||||
where
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
use serde::Deserialize as _;
|
||||
Ok(match serde_json::Value::deserialize(d)? {
|
||||
serde_json::Value::Null => String::new(),
|
||||
serde_json::Value::String(s) => s,
|
||||
serde_json::Value::Array(parts) => {
|
||||
let mut out = String::new();
|
||||
for part in parts {
|
||||
// {"type":"text","text":"..."} / {"type":"input_text","text":"..."}
|
||||
let text = part
|
||||
.get("text")
|
||||
.and_then(|t| t.as_str())
|
||||
.or_else(|| part.as_str());
|
||||
if let Some(t) = text {
|
||||
out.push_str(t);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
other => other.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default)]
|
||||
@@ -29,6 +69,7 @@ pub struct GenParams {
|
||||
pub temperature: Option<f32>,
|
||||
pub max_tokens: Option<u32>,
|
||||
pub top_p: Option<f32>,
|
||||
pub think_budget: Option<u32>,
|
||||
pub stop: Vec<String>,
|
||||
/// Fields we do not interpret but must proxy through to the provider.
|
||||
pub passthrough: serde_json::Map<String, serde_json::Value>,
|
||||
|
||||
+531
-56
@@ -4,7 +4,8 @@
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{
|
||||
ChatRequest, ChatResponse, Choice, Message, RouteDecision, RouteInfo, RoutingDirective, Usage,
|
||||
ChatRequest, ChatResponse, Choice, GenParams, Message, RequestMeta, RouteDecision, RouteInfo,
|
||||
RoutingDirective, Usage,
|
||||
};
|
||||
use crate::state::{Live, SharedState};
|
||||
use crate::stats::RequestRecord;
|
||||
@@ -67,6 +68,21 @@ impl Pipeline {
|
||||
}
|
||||
}
|
||||
|
||||
// Self-warming shadow loop: capture the prompt BEFORE `req` is moved,
|
||||
// so we can (non-blocking, sampled) judge a local answer after the
|
||||
// cloud reply. Client latency is unaffected. Nothing is served locally
|
||||
// here — this only feeds the promotion table.
|
||||
let shadow_cfg = state.live().cfg.shadow.clone();
|
||||
let shadow_prompt = if shadow_cfg.enabled
|
||||
&& !shadow_cfg.local_model_id.is_empty()
|
||||
&& !shadow_cfg.judge_model_id.is_empty()
|
||||
{
|
||||
let live = state.live();
|
||||
Some(self.extract_text(&req.messages, "last_user", live.cfg.route.max_chars))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let result = self.run_inner(req, state).await;
|
||||
|
||||
// record statistics (for both success and error)
|
||||
@@ -135,6 +151,29 @@ impl Pipeline {
|
||||
}
|
||||
}
|
||||
|
||||
// Fire the shadow judgment for a sampled subset (non-blocking).
|
||||
if let (Some(prompt), Ok(resp)) = (shadow_prompt, &result) {
|
||||
let cloud_answer = resp
|
||||
.choices
|
||||
.first()
|
||||
.map(|c| c.message.content.clone())
|
||||
.unwrap_or_default();
|
||||
let label = resp.cortiq.task_label.clone();
|
||||
// Only judge CLOUD-served answers — the reference must be the cloud
|
||||
// answer, not a local one (a promoted label served locally would
|
||||
// otherwise judge local-vs-local). Canary auditing of served-local
|
||||
// answers (forcing a cloud reference for a fraction) is the next step.
|
||||
if !cloud_answer.is_empty()
|
||||
&& !label.is_empty()
|
||||
&& label != "cache"
|
||||
&& resp.cortiq.selected_model != shadow_cfg.local_model_id
|
||||
&& should_shadow(&prompt, shadow_cfg.sample_rate)
|
||||
{
|
||||
let st = state.0.clone();
|
||||
tokio::spawn(shadow_capture(st, shadow_cfg, prompt, cloud_answer, label));
|
||||
}
|
||||
}
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
@@ -144,6 +183,9 @@ impl Pipeline {
|
||||
&self,
|
||||
req: &ChatRequest,
|
||||
live: &crate::state::Live,
|
||||
// Tool support detected via the local servers' healthz handshake —
|
||||
// treated as equivalent to declared caps when filtering candidates.
|
||||
detected_tools: &std::collections::HashSet<String>,
|
||||
) -> Result<(RouteDecision, String, Vec<String>)> {
|
||||
// 1. Extract text for classification
|
||||
let text = self.extract_text(
|
||||
@@ -152,60 +194,96 @@ impl Pipeline {
|
||||
live.cfg.route.max_chars,
|
||||
);
|
||||
|
||||
// 2. Obtain the router decision (or use the pinned model)
|
||||
let (decision, route_source) = match &req.routing {
|
||||
RoutingDirective::Pinned { model_id: _ } => {
|
||||
let dec = RouteDecision {
|
||||
task_label: "pinned".to_string(),
|
||||
complexity_score: 0.0,
|
||||
complexity_tier: "pinned".to_string(),
|
||||
router_request_id: None,
|
||||
source: "pinned".to_string(),
|
||||
};
|
||||
(dec, "pinned".to_string())
|
||||
}
|
||||
RoutingDirective::Auto { profile } => {
|
||||
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
|
||||
match live.router.route(&text, prof).await {
|
||||
Ok(Some(dec)) => {
|
||||
let src = dec.source.clone();
|
||||
(dec, src)
|
||||
}
|
||||
Ok(None) | Err(_) => {
|
||||
tracing::warn!(
|
||||
"Router is unavailable. Gracefully degrading to default model."
|
||||
);
|
||||
let dec = RouteDecision {
|
||||
task_label: "degraded".to_string(),
|
||||
complexity_score: 0.5,
|
||||
complexity_tier: "degraded".to_string(),
|
||||
router_request_id: None,
|
||||
source: "fallback".to_string(),
|
||||
};
|
||||
(dec, "fallback".to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
// 2–3. Decide the route and pick candidate models.
|
||||
//
|
||||
// Local-first rules, so CMF models work with no router at all:
|
||||
// • `[cmf].local_only` → every request goes to the local pool,
|
||||
// ignoring the router, the pin, and external providers;
|
||||
// • `[router].enabled = false` → skip the router, use the local pool;
|
||||
// • router unavailable at runtime → graceful degrade to the local pool;
|
||||
// • an explicitly pinned model → used directly, no router call.
|
||||
let mk = |label: &str, source: &str| RouteDecision {
|
||||
task_label: label.to_string(),
|
||||
complexity_score: 0.0,
|
||||
complexity_tier: label.to_string(),
|
||||
router_request_id: None,
|
||||
source: source.to_string(),
|
||||
};
|
||||
|
||||
// 3. Select candidates by complexity tier / pinned model
|
||||
let candidates = match &req.routing {
|
||||
RoutingDirective::Pinned { model_id } => vec![model_id.clone()],
|
||||
RoutingDirective::Auto { .. } => {
|
||||
if route_source == "fallback" {
|
||||
vec![live.routing.default_model().to_string()]
|
||||
} else {
|
||||
live.routing.candidates(&decision.complexity_tier)
|
||||
let (decision, route_source, candidates) = if live.cfg.cmf.local_only {
|
||||
(
|
||||
mk("local-only", "local"),
|
||||
"local".to_string(),
|
||||
live.local_candidates(),
|
||||
)
|
||||
} else {
|
||||
match &req.routing {
|
||||
RoutingDirective::Pinned { model_id } => (
|
||||
mk("pinned", "pinned"),
|
||||
"pinned".to_string(),
|
||||
vec![model_id.clone()],
|
||||
),
|
||||
RoutingDirective::Auto { profile } => {
|
||||
if !live.cfg.router.enabled {
|
||||
(
|
||||
mk("router-disabled", "local"),
|
||||
"local".to_string(),
|
||||
live.local_candidates(),
|
||||
)
|
||||
} else {
|
||||
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
|
||||
match live.router.route(&text, prof).await {
|
||||
Ok(Some(dec)) => {
|
||||
let mut dec = dec;
|
||||
// per-gateway complexity bands: re-map the
|
||||
// router's continuous score with the user's
|
||||
// own boundaries (sliders on the Routing page)
|
||||
if let Some(b) = &live.cfg.routing.bands {
|
||||
if matches!(
|
||||
dec.complexity_tier.as_str(),
|
||||
"low" | "medium" | "high"
|
||||
) {
|
||||
dec.complexity_tier =
|
||||
b.tier_of(dec.complexity_score).to_string();
|
||||
}
|
||||
}
|
||||
let src = dec.source.clone();
|
||||
let (cands, task_rule) = live
|
||||
.routing
|
||||
.candidates_for(&dec.task_label, &dec.complexity_tier);
|
||||
// surface which rule chose the head of the chain
|
||||
let src = if task_rule {
|
||||
format!("{src}+task-rule")
|
||||
} else {
|
||||
src
|
||||
};
|
||||
(dec, src, cands)
|
||||
}
|
||||
Ok(None) | Err(_) => {
|
||||
tracing::warn!(
|
||||
"Router is unavailable. Gracefully degrading to the local/default model."
|
||||
);
|
||||
(
|
||||
mk("degraded", "fallback"),
|
||||
"fallback".to_string(),
|
||||
live.local_candidates(),
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Filter candidates by capabilities (e.g. tools support)
|
||||
let needs_tools = !req.tools.is_empty();
|
||||
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
|
||||
prov.caps().tools || detected_tools.contains(id)
|
||||
};
|
||||
let mut final_candidates = Vec::new();
|
||||
for model_id in candidates {
|
||||
if let Some(prov) = live.registry.get(&model_id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&model_id, &prov) {
|
||||
tracing::debug!(
|
||||
"Candidate '{}' skipped: lacks support for requested tools",
|
||||
model_id
|
||||
@@ -216,6 +294,67 @@ impl Pipeline {
|
||||
}
|
||||
}
|
||||
|
||||
// An explicit pin must never silently answer from a different model: if
|
||||
// the pinned id is unknown or was filtered out, fail loudly so the caller
|
||||
// knows their choice was not honoured (instead of getting a stray reply
|
||||
// from the managed local model via the safety net below).
|
||||
if final_candidates.is_empty() {
|
||||
if let RoutingDirective::Pinned { model_id } = &req.routing {
|
||||
return Err(GatewayError::UpstreamUnavailable(format!(
|
||||
"pinned model '{model_id}' is not available — it is not in the pool (deleted?) or lacks a required capability such as tools"
|
||||
)));
|
||||
}
|
||||
}
|
||||
|
||||
// Local-first safety net: for AUTO/degraded routes only, if routing
|
||||
// produced no usable candidate, fall back to the managed local model so a
|
||||
// configured local backend still answers.
|
||||
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
|
||||
for id in live.local_candidates() {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Capability widening: a tier chain built of tools-less local models
|
||||
// must not kill an agent request. When the filtered chain is empty on
|
||||
// an AUTO route, widen to any pool model that satisfies the required
|
||||
// capabilities — preferring the routing table's own medium→high order
|
||||
// (an agentic request is worth a capable model), then the rest of the
|
||||
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
|
||||
// this every low-tier verdict was a hard failure.
|
||||
if final_candidates.is_empty() {
|
||||
let mut widened: Vec<String> = Vec::new();
|
||||
for tier in ["medium", "high"] {
|
||||
for id in live.routing.candidates(tier) {
|
||||
if !widened.contains(&id) {
|
||||
widened.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in live.registry.ids() {
|
||||
if !widened.contains(id) {
|
||||
widened.push(id.clone());
|
||||
}
|
||||
}
|
||||
for id in widened {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
if !final_candidates.is_empty() {
|
||||
tracing::info!(
|
||||
chosen = %final_candidates[0],
|
||||
"capability widening: tier chain had no model with the required capabilities"
|
||||
);
|
||||
}
|
||||
}
|
||||
if final_candidates.is_empty() {
|
||||
return Err(GatewayError::UpstreamUnavailable(
|
||||
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
||||
@@ -227,7 +366,28 @@ impl Pipeline {
|
||||
|
||||
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
||||
let live = state.live();
|
||||
let (decision, route_source, final_candidates) = self.resolve(&req, &live).await?;
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, mut final_candidates) =
|
||||
self.resolve(&req, &live, &detected_tools).await?;
|
||||
|
||||
// Serving-when-promoted (gated): a promoted task-label is served by the
|
||||
// local model FIRST (failover to cloud on error). Per-request VETO uses
|
||||
// the router's own complexity signal — a "high"-tier request escalates
|
||||
// to cloud even in a promoted label (the local skill is for the common
|
||||
// case, not the hard tail). Off by default.
|
||||
let sh = &live.cfg.shadow;
|
||||
if sh.serve_when_promoted
|
||||
&& !sh.local_model_id.is_empty()
|
||||
&& matches!(req.routing, RoutingDirective::Auto { .. })
|
||||
&& state.promotion.serves_local(&decision.task_label)
|
||||
&& !matches!(
|
||||
decision.complexity_tier.as_str(),
|
||||
"high" | "hard" | "complex"
|
||||
)
|
||||
&& !final_candidates.iter().any(|m| m == &sh.local_model_id)
|
||||
{
|
||||
final_candidates.insert(0, sh.local_model_id.clone());
|
||||
}
|
||||
|
||||
// 4. Attempt provider call with failover (fall back to alternatives on error)
|
||||
let mut last_err = None;
|
||||
@@ -246,7 +406,66 @@ impl Pipeline {
|
||||
route_source
|
||||
);
|
||||
|
||||
match provider.chat(req.clone()).await {
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat(req_call).await {
|
||||
Ok(mut resp) => {
|
||||
let cost =
|
||||
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
|
||||
@@ -298,13 +517,15 @@ impl Pipeline {
|
||||
.to_string();
|
||||
|
||||
let live = state.live();
|
||||
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, final_candidates) =
|
||||
match self.resolve(&req, &live, &detected_tools).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
|
||||
let mut last_err = None;
|
||||
let mut attempt = 0usize;
|
||||
@@ -320,7 +541,67 @@ impl Pipeline {
|
||||
model_id,
|
||||
route_source
|
||||
);
|
||||
match provider.chat_stream(req.clone()).await {
|
||||
// Merge per-model/server config (same as run_inner).
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat_stream(req_call).await {
|
||||
Ok(stream) => {
|
||||
let info = RouteInfo {
|
||||
task_label: decision.task_label.clone(),
|
||||
@@ -332,6 +613,11 @@ impl Pipeline {
|
||||
cost_usd: 0.0,
|
||||
failover: attempt > 0,
|
||||
};
|
||||
// for the client-facing usage estimate when the provider
|
||||
// sends none: the prompt size is known here, the reply is
|
||||
// measured inside the tap
|
||||
let prompt_chars: usize =
|
||||
req.messages.iter().map(|m| m.content.chars().count()).sum();
|
||||
let tapped = tap_stream(
|
||||
stream,
|
||||
state.stats.clone(),
|
||||
@@ -341,6 +627,7 @@ impl Pipeline {
|
||||
protocol,
|
||||
directive,
|
||||
started,
|
||||
prompt_chars,
|
||||
);
|
||||
return Ok((info, Box::pin(tapped)));
|
||||
}
|
||||
@@ -451,6 +738,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
|
||||
role: "assistant".into(),
|
||||
content: hit.content.clone(),
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
},
|
||||
finish_reason: "stop".into(),
|
||||
}],
|
||||
@@ -512,6 +800,7 @@ fn tap_stream(
|
||||
protocol: String,
|
||||
directive: String,
|
||||
started: Instant,
|
||||
prompt_chars: usize,
|
||||
) -> impl futures::Stream<Item = Result<bytes::Bytes>> {
|
||||
use futures::StreamExt;
|
||||
async_stream::stream! {
|
||||
@@ -521,12 +810,15 @@ fn tap_stream(
|
||||
let mut completion_tokens = 0u32;
|
||||
let mut got_usage = false;
|
||||
let mut delta_chars = 0usize;
|
||||
let mut delta_chunks = 0u32;
|
||||
let mut errored: Option<String> = None;
|
||||
|
||||
while let Some(item) = inner.next().await {
|
||||
let mut done_marker = false;
|
||||
match &item {
|
||||
Ok(bytes) => {
|
||||
if let Ok(s) = std::str::from_utf8(bytes) {
|
||||
done_marker = s.contains("data: [DONE]");
|
||||
buf.push_str(s);
|
||||
while let Some(idx) = buf.find("\n\n") {
|
||||
let evt: String = buf.drain(..idx + 2).collect();
|
||||
@@ -555,6 +847,7 @@ fn tap_stream(
|
||||
v["choices"][0]["delta"]["content"].as_str()
|
||||
{
|
||||
delta_chars += c.chars().count();
|
||||
delta_chunks += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -564,12 +857,52 @@ fn tap_stream(
|
||||
}
|
||||
Err(e) => errored = Some(e.to_string()),
|
||||
}
|
||||
// The local cortiq server (and some providers) never send a usage
|
||||
// chunk. Clients still deserve token counts, so ahead of [DONE] we
|
||||
// inject one final OpenAI-shaped usage event with the gateway's
|
||||
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
|
||||
// mistakes it for the tokenizer's truth.
|
||||
if done_marker {
|
||||
// The initial X-Cortiq-Cost-USD header is always 0 — cost is
|
||||
// only known after generation. Ship the final number (and the
|
||||
// token estimate when the provider sent none) ahead of [DONE].
|
||||
if !got_usage {
|
||||
prompt_tokens = (prompt_chars / 4) as u32;
|
||||
completion_tokens = if delta_chunks > 0 {
|
||||
delta_chunks
|
||||
} else {
|
||||
(delta_chars / 4) as u32
|
||||
};
|
||||
}
|
||||
let cost = provider.price(prompt_tokens, completion_tokens);
|
||||
let mut evt = serde_json::json!({
|
||||
"object": "chat.completion.chunk",
|
||||
"model": info.selected_model,
|
||||
"choices": [],
|
||||
"cortiq_cost_usd": cost,
|
||||
});
|
||||
if !got_usage {
|
||||
evt["usage"] = serde_json::json!({
|
||||
"prompt_tokens": prompt_tokens,
|
||||
"completion_tokens": completion_tokens,
|
||||
"total_tokens": prompt_tokens + completion_tokens,
|
||||
});
|
||||
evt["cortiq_estimated"] = serde_json::json!(true);
|
||||
got_usage = true;
|
||||
}
|
||||
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
|
||||
}
|
||||
yield item;
|
||||
}
|
||||
|
||||
if !got_usage && completion_tokens == 0 {
|
||||
// estimate when the server didn't return usage (~4 chars per token)
|
||||
completion_tokens = (delta_chars / 4) as u32;
|
||||
// stream ended without a [DONE] marker — estimate for the stats
|
||||
completion_tokens = if delta_chunks > 0 {
|
||||
delta_chunks
|
||||
} else {
|
||||
(delta_chars / 4) as u32
|
||||
};
|
||||
prompt_tokens = (prompt_chars / 4) as u32;
|
||||
}
|
||||
let cost = provider.price(prompt_tokens, completion_tokens);
|
||||
stats.record(RequestRecord {
|
||||
@@ -592,3 +925,145 @@ fn tap_stream(
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// ─────────────────────────── self-warming shadow loop ───────────────────────────
|
||||
|
||||
/// Deterministic sampling by prompt hash — no RNG dependency, and the same
|
||||
/// prompt is always (not) sampled, so repeated office templates don't skew
|
||||
/// the judged set request-by-request.
|
||||
fn should_shadow(prompt: &str, rate: f64) -> bool {
|
||||
if rate <= 0.0 {
|
||||
return false;
|
||||
}
|
||||
if rate >= 1.0 {
|
||||
return true;
|
||||
}
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
prompt.hash(&mut h);
|
||||
(h.finish() % 10_000) < (rate * 10_000.0) as u64
|
||||
}
|
||||
|
||||
fn clip(s: &str, max: usize) -> String {
|
||||
if s.len() <= max {
|
||||
s.to_string()
|
||||
} else {
|
||||
s.chars().take(max).collect()
|
||||
}
|
||||
}
|
||||
|
||||
fn shadow_req(model_id: &str, content: String, max_tokens: u32) -> ChatRequest {
|
||||
ChatRequest {
|
||||
routing: RoutingDirective::Pinned {
|
||||
model_id: model_id.to_string(),
|
||||
},
|
||||
messages: vec![Message {
|
||||
role: "user".to_string(),
|
||||
content,
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
}],
|
||||
tools: Vec::new(),
|
||||
params: GenParams {
|
||||
temperature: Some(0.0),
|
||||
max_tokens: Some(max_tokens),
|
||||
..Default::default()
|
||||
},
|
||||
stream: false,
|
||||
meta: RequestMeta::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract `{"score":N,"hard_fail":bool}` from a judge reply, tolerant of
|
||||
/// surrounding prose. Missing/garbled → (0, true) (fail-closed: an
|
||||
/// unparseable judgment must never count as a pass).
|
||||
fn parse_judge(text: &str) -> (u8, bool) {
|
||||
let score = text
|
||||
.split("score")
|
||||
.nth(1)
|
||||
.and_then(|t| t.chars().find(|c| c.is_ascii_digit()))
|
||||
.and_then(|c| c.to_digit(10))
|
||||
.map(|d| d.min(4) as u8);
|
||||
let hard_fail = text
|
||||
.split("hard_fail")
|
||||
.nth(1)
|
||||
.map(|t| t.contains("true"))
|
||||
.unwrap_or(true);
|
||||
match score {
|
||||
Some(s) => (s, hard_fail),
|
||||
None => (0, true),
|
||||
}
|
||||
}
|
||||
|
||||
/// Non-blocking: answer locally, have the cheap judge score it against the
|
||||
/// already-paid cloud answer, and feed the per-label promotion table. Any
|
||||
/// failure (missing provider, upstream error, unparseable judgment) is
|
||||
/// silently dropped — the client already got the cloud answer.
|
||||
async fn shadow_capture(
|
||||
st: std::sync::Arc<crate::state::AppState>,
|
||||
cfg: crate::config::ShadowCfg,
|
||||
prompt: String,
|
||||
cloud_answer: String,
|
||||
label: String,
|
||||
) {
|
||||
let live = st.live.load_full();
|
||||
let (Some(local), Some(judge)) = (
|
||||
live.registry.get(&cfg.local_model_id),
|
||||
live.registry.get(&cfg.judge_model_id),
|
||||
) else {
|
||||
return;
|
||||
};
|
||||
|
||||
// 1. Local answer (free, CPU).
|
||||
let a_local = match local
|
||||
.chat(shadow_req(&cfg.local_model_id, clip(&prompt, 4000), 512))
|
||||
.await
|
||||
{
|
||||
Ok(r) => r
|
||||
.choices
|
||||
.into_iter()
|
||||
.next()
|
||||
.map(|c| c.message.content)
|
||||
.unwrap_or_default(),
|
||||
Err(_) => return,
|
||||
};
|
||||
if a_local.trim().is_empty() {
|
||||
return;
|
||||
}
|
||||
|
||||
// 2. Reference-based judgment (cheap model, different family).
|
||||
let jp = format!(
|
||||
"You grade a CANDIDATE answer against a REFERENCE answer for the same task.\n\
|
||||
Task type: {label}\n\n[TASK]\n{task}\n\n[REFERENCE]\n{reference}\n\n[CANDIDATE]\n{candidate}\n\n\
|
||||
Reply with ONLY compact JSON: {{\"score\":0-4,\"hard_fail\":true|false}}. \
|
||||
score>=3 means the candidate is acceptable (correct, complete, right format). \
|
||||
hard_fail=true for fabricated facts, broken format, or unsafe content.",
|
||||
label = label,
|
||||
task = clip(&prompt, 2000),
|
||||
reference = clip(&cloud_answer, 3000),
|
||||
candidate = clip(&a_local, 3000),
|
||||
);
|
||||
let Ok(jr) = judge.chat(shadow_req(&cfg.judge_model_id, jp, 32)).await else {
|
||||
return;
|
||||
};
|
||||
let judge_tokens = jr.usage.total_tokens;
|
||||
let jtext = jr
|
||||
.choices
|
||||
.into_iter()
|
||||
.next()
|
||||
.map(|c| c.message.content)
|
||||
.unwrap_or_default();
|
||||
let (score, hard_fail) = parse_judge(&jtext);
|
||||
|
||||
st.promotion.record(crate::promotion::JudgeRecord {
|
||||
ts: now_secs(),
|
||||
task_label: label,
|
||||
pass: score >= 3 && !hard_fail,
|
||||
hard_fail,
|
||||
score,
|
||||
source: "judge".to_string(),
|
||||
recon_e: None,
|
||||
born_conf: None,
|
||||
judge_tokens,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,447 @@
|
||||
//! Cloud-as-judge promotion table (self-warming local model, MVP).
|
||||
//!
|
||||
//! Per task-label we accumulate a judge/checker PASS stream and gate the
|
||||
//! per-label lifecycle on the **Wilson-95 LOWER bound** of the pass rate —
|
||||
//! never on the point estimate, so a lucky streak of easy requests can NOT
|
||||
//! promote a label (small n → wide interval → low bound → stays SHADOW).
|
||||
//! The full design + honest holes: docs/SELF_WARMING_GATEWAY.md.
|
||||
//!
|
||||
//! This module is pure bookkeeping (like `stats::Stats`): it decides the
|
||||
//! STATE; the pipeline consults the state to decide who serves. Nothing here
|
||||
//! serves a local answer — that is the caller's decision, guarded by these
|
||||
//! states plus a per-request veto.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::{HashMap, VecDeque};
|
||||
use std::io::Write;
|
||||
use std::sync::Mutex;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
fn now_secs() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// One judgment of a local answer (JSONL line, replayed on startup).
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct JudgeRecord {
|
||||
pub ts: u64,
|
||||
pub task_label: String,
|
||||
/// 1 = local answer accepted (checker pass OR judge score≥bar & !hard_fail).
|
||||
pub pass: bool,
|
||||
/// Fact hallucination / broken format / unsafe — blocks promotion hard.
|
||||
#[serde(default)]
|
||||
pub hard_fail: bool,
|
||||
/// Raw judge score 0..4 (diagnostic only); 4 for a $0 programmatic checker.
|
||||
#[serde(default)]
|
||||
pub score: u8,
|
||||
/// "checker" ($0 ground-truth) or "judge" (cloud LLM-as-judge).
|
||||
#[serde(default)]
|
||||
pub source: String,
|
||||
/// Local recon-E (novelty) at judge time — diagnostic; NOT a correctness gate.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub recon_e: Option<f32>,
|
||||
/// Local calibrated Born-mass confidence — for the negative-control study.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub born_conf: Option<f32>,
|
||||
/// Cloud tokens spent judging (0 for a programmatic checker).
|
||||
#[serde(default)]
|
||||
pub judge_tokens: u32,
|
||||
}
|
||||
|
||||
/// Per-label lifecycle. Serving-locally is only ever Canary/LocalServed.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PromotionState {
|
||||
CloudOnly,
|
||||
ShadowJudged,
|
||||
Canary,
|
||||
LocalServed,
|
||||
Demoted,
|
||||
}
|
||||
|
||||
/// Tuning (defaults from the memo; refit to measured ECE/traffic).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct PromotionCfg {
|
||||
pub enabled: bool,
|
||||
pub window: usize, // rolling judgments per label (W)
|
||||
pub n_min: usize, // min judgments before promotion is even considered
|
||||
pub promote_lb: f64, // Wilson-95 LB of pass must clear this
|
||||
pub demote_pass: f64, // point pass-rate below this → demote a served label
|
||||
pub hardfail_ub_max: f64, // Wilson-95 UPPER bound of hard_fail must stay under this
|
||||
pub soak: usize, // extra Canary judgments before LocalServed
|
||||
pub z: f64, // 1.96 = 95%
|
||||
pub file: Option<String>, // judge.jsonl path (append + replay)
|
||||
}
|
||||
|
||||
impl Default for PromotionCfg {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
enabled: false,
|
||||
window: 500,
|
||||
n_min: 200,
|
||||
promote_lb: 0.95,
|
||||
demote_pass: 0.90,
|
||||
hardfail_ub_max: 0.015,
|
||||
soak: 100,
|
||||
z: 1.96,
|
||||
file: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct LabelState {
|
||||
passes: VecDeque<bool>, // rolling window of pass/fail
|
||||
hard_fails: VecDeque<bool>, // rolling window of hard_fail flags
|
||||
state: PromotionState,
|
||||
soaked: usize, // judgments accumulated while in Canary
|
||||
}
|
||||
|
||||
impl LabelState {
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
passes: VecDeque::new(),
|
||||
hard_fails: VecDeque::new(),
|
||||
state: PromotionState::ShadowJudged,
|
||||
soaked: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Promotion {
|
||||
cfg: PromotionCfg,
|
||||
labels: Mutex<HashMap<String, LabelState>>,
|
||||
}
|
||||
|
||||
/// Wilson score interval bound for a Bernoulli rate. `upper=false` → lower.
|
||||
/// s successes of n trials, z the normal quantile. Distribution-aware: small
|
||||
/// n gives a wide interval (low lower bound / high upper bound) by construction.
|
||||
pub fn wilson_bound(s: usize, n: usize, z: f64, upper: bool) -> f64 {
|
||||
if n == 0 {
|
||||
return if upper { 1.0 } else { 0.0 };
|
||||
}
|
||||
let n = n as f64;
|
||||
let p = s as f64 / n;
|
||||
let z2 = z * z;
|
||||
let denom = 1.0 + z2 / n;
|
||||
let centre = p + z2 / (2.0 * n);
|
||||
let margin = z * ((p * (1.0 - p) / n) + z2 / (4.0 * n * n)).sqrt();
|
||||
let v = if upper {
|
||||
centre + margin
|
||||
} else {
|
||||
centre - margin
|
||||
};
|
||||
(v / denom).clamp(0.0, 1.0)
|
||||
}
|
||||
|
||||
impl Promotion {
|
||||
pub fn new(cfg: PromotionCfg) -> std::sync::Arc<Self> {
|
||||
let p = std::sync::Arc::new(Self {
|
||||
cfg,
|
||||
labels: Mutex::new(HashMap::new()),
|
||||
});
|
||||
p.replay();
|
||||
p
|
||||
}
|
||||
|
||||
fn replay(&self) {
|
||||
let Some(path) = &self.cfg.file else { return };
|
||||
let Ok(content) = std::fs::read_to_string(path) else {
|
||||
return;
|
||||
};
|
||||
for line in content.lines() {
|
||||
if let Ok(rec) = serde_json::from_str::<JudgeRecord>(line) {
|
||||
self.apply(&rec);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Record a judgment: update the rolling windows, advance the state
|
||||
/// machine, append to judge.jsonl. Returns the (possibly new) state.
|
||||
pub fn record(&self, rec: JudgeRecord) -> PromotionState {
|
||||
let st = self.apply(&rec);
|
||||
if let Some(path) = &self.cfg.file {
|
||||
if let Ok(line) = serde_json::to_string(&rec) {
|
||||
if let Ok(mut f) = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(path)
|
||||
{
|
||||
let _ = writeln!(f, "{line}");
|
||||
}
|
||||
}
|
||||
}
|
||||
st
|
||||
}
|
||||
|
||||
fn apply(&self, rec: &JudgeRecord) -> PromotionState {
|
||||
let mut map = self.labels.lock().unwrap();
|
||||
let ls = map
|
||||
.entry(rec.task_label.clone())
|
||||
.or_insert_with(LabelState::new);
|
||||
push_cap(&mut ls.passes, rec.pass, self.cfg.window);
|
||||
push_cap(&mut ls.hard_fails, rec.hard_fail, self.cfg.window);
|
||||
self.advance(ls);
|
||||
ls.state
|
||||
}
|
||||
|
||||
fn advance(&self, ls: &mut LabelState) {
|
||||
let n = ls.passes.len();
|
||||
let s = ls.passes.iter().filter(|&&p| p).count();
|
||||
let hf = ls.hard_fails.iter().filter(|&&h| h).count();
|
||||
let lb = wilson_bound(s, n, self.cfg.z, false);
|
||||
let point = if n > 0 { s as f64 / n as f64 } else { 0.0 };
|
||||
let recent_hf = ls.hard_fails.iter().rev().take(50).filter(|&&h| h).count();
|
||||
|
||||
// Zero-tolerance on hard-fails in the window: a Wilson UPPER bound on
|
||||
// hard-fail rate ≤ 0.015 is UNREACHABLE at n=200 even with 0 fails
|
||||
// (rule-of-three: 0/200 → ~0.019) — the panel flagged this. Requiring
|
||||
// ZERO recent hard-fails is achievable AND safer (a recent
|
||||
// hallucination/unsafe answer → stay on cloud until it ages out).
|
||||
let bar = n >= self.cfg.n_min && lb >= self.cfg.promote_lb && hf == 0;
|
||||
let regressed = point < self.cfg.demote_pass || recent_hf >= 2;
|
||||
|
||||
ls.state = match ls.state {
|
||||
PromotionState::CloudOnly | PromotionState::ShadowJudged => {
|
||||
if bar {
|
||||
ls.soaked = 0;
|
||||
PromotionState::Canary
|
||||
} else {
|
||||
PromotionState::ShadowJudged
|
||||
}
|
||||
}
|
||||
PromotionState::Canary => {
|
||||
if regressed {
|
||||
PromotionState::Demoted
|
||||
} else {
|
||||
ls.soaked += 1;
|
||||
if ls.soaked >= self.cfg.soak && bar {
|
||||
PromotionState::LocalServed
|
||||
} else {
|
||||
PromotionState::Canary
|
||||
}
|
||||
}
|
||||
}
|
||||
PromotionState::LocalServed => {
|
||||
if regressed {
|
||||
PromotionState::Demoted
|
||||
} else {
|
||||
PromotionState::LocalServed
|
||||
}
|
||||
}
|
||||
// Demoted goes back to SHADOW to re-earn promotion (skill stays in .cmf).
|
||||
PromotionState::Demoted => PromotionState::ShadowJudged,
|
||||
};
|
||||
}
|
||||
|
||||
/// Current lifecycle state of a label.
|
||||
pub fn state(&self, label: &str) -> PromotionState {
|
||||
self.labels
|
||||
.lock()
|
||||
.unwrap()
|
||||
.get(label)
|
||||
.map(|l| l.state)
|
||||
.unwrap_or(PromotionState::CloudOnly)
|
||||
}
|
||||
|
||||
/// Should a request of this label be SERVED by the local model?
|
||||
/// (Canary/LocalServed; caller still applies the per-request veto.)
|
||||
pub fn serves_local(&self, label: &str) -> bool {
|
||||
matches!(
|
||||
self.state(label),
|
||||
PromotionState::Canary | PromotionState::LocalServed
|
||||
)
|
||||
}
|
||||
|
||||
/// Snapshot for /admin: (state, n, pass-rate, Wilson-LB) per label.
|
||||
pub fn snapshot(&self) -> Vec<(String, PromotionState, usize, f64, f64)> {
|
||||
let map = self.labels.lock().unwrap();
|
||||
let mut out: Vec<_> = map
|
||||
.iter()
|
||||
.map(|(k, ls)| {
|
||||
let n = ls.passes.len();
|
||||
let s = ls.passes.iter().filter(|&&p| p).count();
|
||||
let p = if n > 0 { s as f64 / n as f64 } else { 0.0 };
|
||||
(
|
||||
k.clone(),
|
||||
ls.state,
|
||||
n,
|
||||
p,
|
||||
wilson_bound(s, n, self.cfg.z, false),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
out.sort_by_key(|x| std::cmp::Reverse(x.2));
|
||||
out
|
||||
}
|
||||
|
||||
pub fn enabled(&self) -> bool {
|
||||
self.cfg.enabled
|
||||
}
|
||||
}
|
||||
|
||||
fn push_cap<T>(q: &mut VecDeque<T>, v: T, cap: usize) {
|
||||
q.push_back(v);
|
||||
while q.len() > cap {
|
||||
q.pop_front();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn cfg() -> PromotionCfg {
|
||||
PromotionCfg {
|
||||
enabled: true,
|
||||
n_min: 200,
|
||||
soak: 0,
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
fn rec(label: &str, pass: bool, hard_fail: bool) -> JudgeRecord {
|
||||
JudgeRecord {
|
||||
ts: 0,
|
||||
task_label: label.into(),
|
||||
pass,
|
||||
hard_fail,
|
||||
score: if pass { 4 } else { 0 },
|
||||
source: "checker".into(),
|
||||
recon_e: None,
|
||||
born_conf: None,
|
||||
judge_tokens: 0,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wilson_lb_penalizes_small_n() {
|
||||
// 5/5 perfect but tiny n → lower bound far below 0.95.
|
||||
assert!(wilson_bound(5, 5, 1.96, false) < 0.6);
|
||||
// 195/200 at 97.5% → lower bound clears 0.95.
|
||||
assert!(wilson_bound(195, 200, 1.96, false) >= 0.94);
|
||||
// upper bound of 0 hard-fails over 200 ≈ 0.019 (rule-of-three-ish).
|
||||
assert!(wilson_bound(0, 200, 1.96, true) < 0.025);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_promotion_on_a_lucky_streak() {
|
||||
let p = Promotion::new(cfg());
|
||||
// 50 straight passes: point-rate 100% but n < n_min AND LB < bar.
|
||||
for _ in 0..50 {
|
||||
p.record(rec("code", true, false));
|
||||
}
|
||||
assert_eq!(p.state("code"), PromotionState::ShadowJudged);
|
||||
assert!(!p.serves_local("code"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn promotes_only_with_enough_high_evidence() {
|
||||
let p = Promotion::new(cfg());
|
||||
// 198/200 (99%) + zero hard-fails → Wilson-LB ≈ 0.964 ≥ 0.95 → Canary.
|
||||
// (195/200 = 97.5% gives LB ≈ 0.943 < 0.95 and would NOT promote —
|
||||
// the bound is deliberately strict.)
|
||||
for i in 0..200 {
|
||||
p.record(rec("code", i % 100 != 0, false)); // 2 fails / 200
|
||||
}
|
||||
assert_eq!(p.state("code"), PromotionState::Canary); // soak=0 → straight to Canary
|
||||
assert!(p.serves_local("code"));
|
||||
}
|
||||
|
||||
/// Runnable narrative of the full loop:
|
||||
/// cargo test --release demo_full_lifecycle -- --nocapture
|
||||
/// Two labels side by side — a mature skill that promotes to local
|
||||
/// serving, and a task the local can't do (stays on cloud). Proves the
|
||||
/// state machine both PROMOTES on real evidence and REFUSES safely.
|
||||
#[test]
|
||||
fn demo_full_lifecycle() {
|
||||
let p = Promotion::new(PromotionCfg { soak: 50, ..cfg() });
|
||||
let show = |label: &str| {
|
||||
let snap = p.snapshot();
|
||||
let row = snap.iter().find(|r| r.0 == label);
|
||||
match row {
|
||||
Some((_, st, n, pr, lb)) => println!(
|
||||
" {label:<12} {:<13} n={n:<4} pass={:.0}% LB={:.3} serves_local={}",
|
||||
format!("{st:?}"),
|
||||
pr * 100.0,
|
||||
lb,
|
||||
p.serves_local(label)
|
||||
),
|
||||
None => println!(" {label:<12} (cloud-only, no judgments)"),
|
||||
}
|
||||
};
|
||||
|
||||
println!("\n── self-warming loop, live ──");
|
||||
// A mature skill (e.g. ru-tech): local answers pass the judge ~99%.
|
||||
println!(
|
||||
" 1) SHADOW→CANARY: судим локаль 'ru_tech' (сильный скилл ~99% pass), 210 судейств:"
|
||||
);
|
||||
for i in 0..210 {
|
||||
p.record(rec("ru_tech", i % 100 != 0, false));
|
||||
}
|
||||
show("ru_tech");
|
||||
assert_eq!(
|
||||
p.state("ru_tech"),
|
||||
PromotionState::Canary,
|
||||
"n≥200 & LB≥0.95 → CANARY"
|
||||
);
|
||||
|
||||
println!(" 2) CANARY soak (ещё 60 судейств держат планку) → LOCAL_SERVED:");
|
||||
for i in 0..60 {
|
||||
p.record(rec("ru_tech", i % 100 != 0, false));
|
||||
}
|
||||
show("ru_tech");
|
||||
assert_eq!(
|
||||
p.state("ru_tech"),
|
||||
PromotionState::LocalServed,
|
||||
"soak → LOCAL_SERVED"
|
||||
);
|
||||
assert!(p.serves_local("ru_tech"), "теперь обслуживается локально");
|
||||
|
||||
// The code dataset: local fails the checker → never promotes.
|
||||
println!(" 3) параллельно 'code' (0.8B не тянет генерацию кода, 0% pass):");
|
||||
for _ in 0..250 {
|
||||
p.record(rec("code", false, false));
|
||||
}
|
||||
show("code");
|
||||
assert_eq!(
|
||||
p.state("code"),
|
||||
PromotionState::ShadowJudged,
|
||||
"остаётся на облаке — безопасно"
|
||||
);
|
||||
assert!(!p.serves_local("code"));
|
||||
|
||||
// Drift on the promoted label → auto-demote back to cloud.
|
||||
println!(" 4) деградация 'ru_tech' (дрейф) → авто-DEMOTED, назад на облако:");
|
||||
for _ in 0..120 {
|
||||
p.record(rec("ru_tech", false, false));
|
||||
}
|
||||
show("ru_tech");
|
||||
assert!(!p.serves_local("ru_tech"), "демоушен снял с локали");
|
||||
println!(" ── петля: промоушен по доказательству, отказ и откат — безопасно ──\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hard_fail_and_regression_block_or_demote() {
|
||||
// A hard-fail spike keeps hf upper bound high → no promotion.
|
||||
let p = Promotion::new(cfg());
|
||||
for i in 0..200 {
|
||||
p.record(rec("risky", true, i % 20 == 0)); // 5% hard_fail
|
||||
}
|
||||
assert_ne!(p.state("risky"), PromotionState::LocalServed);
|
||||
|
||||
// Promote a clean label, then feed a regression → demote.
|
||||
let p2 = Promotion::new(PromotionCfg { soak: 0, ..cfg() });
|
||||
for i in 0..200 {
|
||||
p2.record(rec("t", i % 100 != 0, false)); // 198/200, 0 hard-fails
|
||||
}
|
||||
assert_eq!(p2.state("t"), PromotionState::Canary);
|
||||
for _ in 0..300 {
|
||||
p2.record(rec("t", false, false)); // quality collapses
|
||||
}
|
||||
assert!(matches!(
|
||||
p2.state("t"),
|
||||
PromotionState::Demoted | PromotionState::ShadowJudged
|
||||
));
|
||||
}
|
||||
}
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
|
||||
use crate::protocols::openai_chat::cortiq_headers;
|
||||
use crate::protocols::openai_chat::{cortiq_headers, sse_headers};
|
||||
use crate::state::SharedState;
|
||||
use axum::response::{IntoResponse, Response};
|
||||
use axum::{extract::State, routing::post, Json, Router};
|
||||
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<MessagesRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.anthropic_messages {
|
||||
@@ -83,6 +84,7 @@ async fn handler(
|
||||
role: "system".into(),
|
||||
content: sys,
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
});
|
||||
}
|
||||
for m in &req.messages {
|
||||
@@ -90,9 +92,12 @@ async fn handler(
|
||||
role: m["role"].as_str().unwrap_or("user").to_string(),
|
||||
content: content_to_string(&m["content"]),
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
});
|
||||
}
|
||||
|
||||
// Echoed back as-is: clients match the reply against the id they asked for.
|
||||
let requested_model = req.model.clone();
|
||||
let canonical = ChatRequest {
|
||||
routing: parse_routing(&req.model),
|
||||
messages,
|
||||
@@ -101,25 +106,26 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "anthropic_messages".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
if req.stream {
|
||||
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let mut headers = cortiq_headers(&info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
axum::http::HeaderValue::from_static("text/event-stream"),
|
||||
);
|
||||
let translated = openai_to_anthropic_sse(stream);
|
||||
return Ok((headers, axum::body::Body::from_stream(translated)).into_response());
|
||||
let translated = openai_to_anthropic_sse(stream, requested_model);
|
||||
return Ok((
|
||||
sse_headers(&info),
|
||||
axum::body::Body::from_stream(translated),
|
||||
)
|
||||
.into_response());
|
||||
}
|
||||
|
||||
let resp = state.pipeline.run(canonical, &state).await?;
|
||||
@@ -133,7 +139,7 @@ async fn handler(
|
||||
"id": resp.id,
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": resp.model_used,
|
||||
"model": requested_model,
|
||||
"content": [{ "type": "text", "text": answer }],
|
||||
"stop_reason": "end_turn",
|
||||
"stop_sequence": serde_json::Value::Null,
|
||||
@@ -149,6 +155,7 @@ async fn handler(
|
||||
/// events, so Anthropic-native clients can stream regardless of the upstream provider.
|
||||
fn openai_to_anthropic_sse(
|
||||
stream: crate::providers::ChatStream,
|
||||
model: String,
|
||||
) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
|
||||
use futures::StreamExt;
|
||||
async_stream::stream! {
|
||||
@@ -159,7 +166,7 @@ fn openai_to_anthropic_sse(
|
||||
yield Ok(evt("message_start", serde_json::json!({
|
||||
"type": "message_start",
|
||||
"message": {"id": "msg_gw", "type": "message", "role": "assistant",
|
||||
"content": [], "model": "cortiq-auto", "stop_reason": null,
|
||||
"content": [], "model": model, "stop_reason": null,
|
||||
"usage": {"input_tokens": 0, "output_tokens": 0}}
|
||||
})));
|
||||
yield Ok(evt("content_block_start", serde_json::json!({
|
||||
|
||||
@@ -94,6 +94,7 @@ async fn handle_tool_call(
|
||||
role: "user".into(),
|
||||
content: prompt.to_string(),
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
}],
|
||||
tools: vec![],
|
||||
params: GenParams::default(),
|
||||
|
||||
+51
-1
@@ -12,10 +12,59 @@ pub mod openai_models;
|
||||
use crate::state::SharedState;
|
||||
use axum::Router;
|
||||
|
||||
/// Account resolved by the API-key middleware, consumed by protocol handlers
|
||||
/// for attribution (stats, budgets). Present only when keys are configured.
|
||||
#[derive(Clone)]
|
||||
pub struct AccountTag(pub String);
|
||||
|
||||
/// API-key gate for the model-facing surface. Open mode (no keys configured)
|
||||
/// passes everything through — the admin console already warns about it.
|
||||
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
|
||||
/// enforced with a fixed one-minute window (429 + Retry-After), and a
|
||||
/// monthly budget, when set, turns into 429 insufficient_quota once the
|
||||
/// account's month-to-date spend crosses it.
|
||||
async fn api_auth(
|
||||
axum::extract::State(state): axum::extract::State<SharedState>,
|
||||
mut req: axum::extract::Request,
|
||||
next: axum::middleware::Next,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
let live = state.live();
|
||||
if live.cfg.api_keys.is_empty() {
|
||||
return next.run(req).await;
|
||||
}
|
||||
let provided = req
|
||||
.headers()
|
||||
.get(axum::http::header::AUTHORIZATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
|
||||
.map(str::trim)
|
||||
.unwrap_or("");
|
||||
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
|
||||
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
|
||||
.into_response();
|
||||
};
|
||||
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
|
||||
return crate::error::GatewayError::RateLimited {
|
||||
retry_after_secs: state.rate.retry_after_secs(),
|
||||
}
|
||||
.into_response();
|
||||
}
|
||||
if let Some(budget) = key.monthly_budget_usd {
|
||||
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
|
||||
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
|
||||
.into_response();
|
||||
}
|
||||
}
|
||||
req.extensions_mut().insert(AccountTag(key.account.clone()));
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
/// Build the protocols router. Implemented adapters are **always** mounted;
|
||||
/// each handler checks the live `protocols.*` flag from config itself, so
|
||||
/// toggles in the admin panel take effect without a restart (hot switching).
|
||||
pub fn build_router() -> Router<SharedState> {
|
||||
pub fn build_router(state: SharedState) -> Router<SharedState> {
|
||||
Router::new()
|
||||
.merge(openai_chat::routes())
|
||||
.merge(openai_completions::routes())
|
||||
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
|
||||
.merge(anthropic_messages::routes())
|
||||
.merge(mcp::routes())
|
||||
.merge(native_passthrough::routes())
|
||||
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
|
||||
}
|
||||
|
||||
+418
-40
@@ -1,6 +1,17 @@
|
||||
//! Incoming adapter for OpenAI Chat Completions: `POST /v1/chat/completions`.
|
||||
//! Translates the OpenAI request body into a canonical [`ChatRequest`], passes it
|
||||
//! to the pipeline, then converts the canonical [`ChatResponse`] back to OpenAI format.
|
||||
//!
|
||||
//! Wire-compatibility rules this adapter enforces (see docs/PROTOCOLS.md §2):
|
||||
//! * the request body is parsed leniently — every shape a real OpenAI client
|
||||
//! sends (`content: null`, content-part arrays, `tools: null`, a string
|
||||
//! `reasoning_effort`, UI-private extras) is accepted, and a body we truly
|
||||
//! cannot read comes back as an OpenAI error envelope, never a bare 422;
|
||||
//! * the response is strict OpenAI JSON — no optional `null` fields, a
|
||||
//! spec-legal `finish_reason`, and the *requested* model id echoed back;
|
||||
//! * the SSE stream is re-normalised chunk by chunk and always terminates
|
||||
//! with `data: [DONE]`, so a client can never sit waiting on a half-formed
|
||||
//! stream from an upstream that improvises.
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RouteInfo, RoutingDirective};
|
||||
@@ -34,28 +45,113 @@ pub(crate) fn cortiq_headers(c: &RouteInfo) -> axum::http::HeaderMap {
|
||||
h
|
||||
}
|
||||
|
||||
/// Headers every SSE response carries. `X-Accel-Buffering: no` matters when the
|
||||
/// gateway sits behind nginx/Traefik — without it the proxy buffers the whole
|
||||
/// stream and the client's chat window just spins.
|
||||
pub(crate) fn sse_headers(info: &RouteInfo) -> axum::http::HeaderMap {
|
||||
use axum::http::HeaderValue;
|
||||
let mut headers = cortiq_headers(info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
HeaderValue::from_static("text/event-stream; charset=utf-8"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::CACHE_CONTROL,
|
||||
HeaderValue::from_static("no-cache"),
|
||||
);
|
||||
headers.insert("X-Accel-Buffering", HeaderValue::from_static("no"));
|
||||
headers
|
||||
}
|
||||
|
||||
pub fn routes() -> Router<SharedState> {
|
||||
Router::new().route("/v1/chat/completions", post(handler))
|
||||
}
|
||||
|
||||
/// `reasoning_effort` is a string in the OpenAI spec (`low` | `medium` | `high`)
|
||||
/// but a token budget in several local runtimes. Accept both.
|
||||
#[derive(Deserialize)]
|
||||
#[serde(untagged)]
|
||||
enum Effort {
|
||||
Budget(u32),
|
||||
Level(String),
|
||||
}
|
||||
|
||||
impl Effort {
|
||||
fn to_budget(&self) -> Option<u32> {
|
||||
match self {
|
||||
Self::Budget(n) => Some(*n),
|
||||
Self::Level(s) => match s.trim().to_ascii_lowercase().as_str() {
|
||||
"none" | "off" | "minimal" => Some(0),
|
||||
"low" => Some(512),
|
||||
"medium" | "default" | "auto" => Some(2048),
|
||||
"high" | "max" => Some(8192),
|
||||
_ => None,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct OpenAiChatRequest {
|
||||
model: String,
|
||||
model: Option<String>,
|
||||
messages: Vec<Message>,
|
||||
#[serde(default)]
|
||||
temperature: Option<f32>,
|
||||
#[serde(default)]
|
||||
max_tokens: Option<u32>,
|
||||
/// The modern spelling of `max_tokens`; clients send one or the other.
|
||||
#[serde(default)]
|
||||
max_completion_tokens: Option<u32>,
|
||||
#[serde(default)]
|
||||
top_p: Option<f32>,
|
||||
#[serde(default)]
|
||||
stream: bool,
|
||||
think_budget: Option<u32>,
|
||||
#[serde(default)]
|
||||
tools: Vec<serde_json::Value>,
|
||||
reasoning_effort: Option<Effort>,
|
||||
/// `null` is a legal value here for several clients, hence `Option`.
|
||||
#[serde(default)]
|
||||
stream: Option<bool>,
|
||||
#[serde(default)]
|
||||
tools: Option<Vec<serde_json::Value>>,
|
||||
#[serde(flatten)]
|
||||
rest: serde_json::Map<String, serde_json::Value>,
|
||||
}
|
||||
|
||||
/// Keys chat UIs attach to the payload for their own bookkeeping. They are not
|
||||
/// part of the OpenAI request schema, and forwarding them to a strict upstream
|
||||
/// (llama.cpp, vLLM, `cortiq serve`) is a needless way to earn a 400.
|
||||
const CLIENT_PRIVATE_KEYS: &[&str] = &[
|
||||
"chat_id",
|
||||
"session_id",
|
||||
"id",
|
||||
"metadata",
|
||||
"background_tasks",
|
||||
"features",
|
||||
"variables",
|
||||
"model_item",
|
||||
"tool_ids",
|
||||
"filter_ids",
|
||||
"files",
|
||||
"citations",
|
||||
"params",
|
||||
"direct",
|
||||
];
|
||||
|
||||
/// Strip client-private bookkeeping from the passthrough map, and drop a
|
||||
/// non-string `user` (Open WebUI sends an object there for pipelines, which the
|
||||
/// OpenAI schema does not allow).
|
||||
fn sanitize_passthrough(
|
||||
mut rest: serde_json::Map<String, serde_json::Value>,
|
||||
) -> serde_json::Map<String, serde_json::Value> {
|
||||
for k in CLIENT_PRIVATE_KEYS {
|
||||
rest.remove(*k);
|
||||
}
|
||||
if rest.get("user").map(|u| !u.is_string()).unwrap_or(false) {
|
||||
rest.remove("user");
|
||||
}
|
||||
rest
|
||||
}
|
||||
|
||||
/// Parse the `model` field value into a routing directive.
|
||||
/// `cortiq-auto[:profile]` → Auto; otherwise → Pinned(real id).
|
||||
fn parse_routing(model: &str) -> RoutingDirective {
|
||||
@@ -69,9 +165,43 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
}
|
||||
}
|
||||
|
||||
fn now_secs() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Map whatever an upstream calls its stop reason onto the closed set the
|
||||
/// OpenAI schema defines. Clients that validate the field (LiteLLM, the
|
||||
/// official SDKs, several agent frameworks) reject anything else.
|
||||
pub(crate) fn normalize_finish_reason(raw: &str) -> &'static str {
|
||||
match raw.trim().to_ascii_lowercase().as_str() {
|
||||
"length" | "max_tokens" | "max_new_tokens" | "token_limit" => "length",
|
||||
"tool_calls" | "tool_use" | "function_call" => "tool_calls",
|
||||
"content_filter" | "safety" => "content_filter",
|
||||
_ => "stop",
|
||||
}
|
||||
}
|
||||
|
||||
/// A response id every client can key on. Upstreams that omit `id` (or send an
|
||||
/// empty one) would otherwise leave the field blank, which some clients treat
|
||||
/// as a malformed completion.
|
||||
fn response_id(upstream: &str) -> String {
|
||||
let trimmed = upstream.trim();
|
||||
if !trimmed.is_empty() {
|
||||
return trimmed.to_string();
|
||||
}
|
||||
format!("chatcmpl-{}", crate::admin::random_token(12))
|
||||
}
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
Json(req): Json<OpenAiChatRequest>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
// Taken as raw bytes rather than `Json<T>`: axum's extractor rejection is a
|
||||
// plain-text 422 that OpenAI clients cannot parse, and a client that cannot
|
||||
// parse the failure has no answer to show — it just hangs.
|
||||
body: axum::body::Bytes,
|
||||
) -> Result<Response> {
|
||||
// hot protocol toggle: if the adapter is disabled in config — return 404
|
||||
if !state.live().cfg.protocols.openai_chat {
|
||||
@@ -79,67 +209,95 @@ async fn handler(
|
||||
"openai_chat protocol is disabled".into(),
|
||||
));
|
||||
}
|
||||
let req: OpenAiChatRequest = serde_json::from_slice(&body)
|
||||
.map_err(|e| GatewayError::InvalidRequest(format!("invalid request body: {e}")))?;
|
||||
|
||||
if req.messages.is_empty() {
|
||||
return Err(GatewayError::InvalidRequest(
|
||||
"messages must not be empty".into(),
|
||||
));
|
||||
}
|
||||
|
||||
// Echoed back verbatim: OpenAI returns the model the caller asked for, and
|
||||
// clients (Open WebUI among them) match the reply against their own id. The
|
||||
// model that actually answered travels in `cortiq.selected_model` and the
|
||||
// `X-Cortiq-Selected-Model` header.
|
||||
let requested_model = req
|
||||
.model
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|m| !m.is_empty())
|
||||
.unwrap_or("cortiq-auto")
|
||||
.to_string();
|
||||
let stream = req.stream.unwrap_or(false);
|
||||
|
||||
let canonical = ChatRequest {
|
||||
routing: parse_routing(&req.model),
|
||||
routing: parse_routing(&requested_model),
|
||||
messages: req.messages,
|
||||
tools: req.tools,
|
||||
tools: req.tools.unwrap_or_default(),
|
||||
params: GenParams {
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
max_tokens: req.max_tokens.or(req.max_completion_tokens),
|
||||
top_p: req.top_p,
|
||||
think_budget: req
|
||||
.think_budget
|
||||
.or_else(|| req.reasoning_effort.as_ref().and_then(Effort::to_budget)),
|
||||
stop: Vec::new(),
|
||||
passthrough: req.rest,
|
||||
passthrough: sanitize_passthrough(req.rest),
|
||||
},
|
||||
stream: req.stream,
|
||||
stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_chat".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
// streaming: forward provider SSE chunks verbatim with X-Cortiq-* headers
|
||||
if req.stream {
|
||||
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let mut headers = cortiq_headers(&info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
axum::http::HeaderValue::from_static("text/event-stream"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::CACHE_CONTROL,
|
||||
axum::http::HeaderValue::from_static("no-cache"),
|
||||
);
|
||||
return Ok((headers, axum::body::Body::from_stream(stream)).into_response());
|
||||
// streaming: provider SSE is re-normalised into strict OpenAI chunks
|
||||
if stream {
|
||||
let (info, raw) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let headers = sse_headers(&info);
|
||||
let normalized = normalize_sse(raw, requested_model);
|
||||
return Ok((headers, axum::body::Body::from_stream(normalized)).into_response());
|
||||
}
|
||||
|
||||
let resp = state.pipeline.run(canonical, &state).await?;
|
||||
let headers = cortiq_headers(&resp.cortiq);
|
||||
|
||||
let mut body = serde_json::json!({
|
||||
"id": resp.id,
|
||||
"object": "chat.completion",
|
||||
"created": std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs(),
|
||||
"model": resp.model_used,
|
||||
"choices": resp.choices.iter().map(|c| {
|
||||
let choices: Vec<serde_json::Value> = resp
|
||||
.choices
|
||||
.iter()
|
||||
.map(|c| {
|
||||
let mut message = serde_json::Map::new();
|
||||
message.insert("role".into(), serde_json::json!(c.message.role));
|
||||
message.insert("content".into(), serde_json::json!(c.message.content));
|
||||
// Absent tool calls are omitted entirely — `"tool_calls": null` is
|
||||
// not in the schema and trips strict clients.
|
||||
if !c.message.tool_calls.is_empty() {
|
||||
message.insert(
|
||||
"tool_calls".into(),
|
||||
serde_json::Value::Array(c.message.tool_calls.clone()),
|
||||
);
|
||||
}
|
||||
let finish = if c.message.tool_calls.is_empty() {
|
||||
normalize_finish_reason(&c.finish_reason)
|
||||
} else {
|
||||
"tool_calls"
|
||||
};
|
||||
serde_json::json!({
|
||||
"index": c.index,
|
||||
"message": {
|
||||
"role": c.message.role,
|
||||
"content": c.message.content,
|
||||
"tool_calls": if c.message.tool_calls.is_empty() { serde_json::Value::Null } else { serde_json::Value::Array(c.message.tool_calls.clone()) },
|
||||
},
|
||||
"finish_reason": c.finish_reason,
|
||||
"message": serde_json::Value::Object(message),
|
||||
"finish_reason": finish,
|
||||
})
|
||||
}).collect::<Vec<_>>(),
|
||||
})
|
||||
.collect();
|
||||
|
||||
let mut body = serde_json::json!({
|
||||
"id": response_id(&resp.id),
|
||||
"object": "chat.completion",
|
||||
"created": now_secs(),
|
||||
"model": requested_model,
|
||||
"choices": choices,
|
||||
"usage": {
|
||||
"prompt_tokens": resp.usage.prompt_tokens,
|
||||
"completion_tokens": resp.usage.completion_tokens,
|
||||
@@ -148,7 +306,7 @@ async fn handler(
|
||||
});
|
||||
|
||||
if state.live().cfg.cortiq.echo {
|
||||
body["cortiq"] = serde_json::json!({
|
||||
let mut echo = serde_json::json!({
|
||||
"task_label": resp.cortiq.task_label,
|
||||
"complexity": {
|
||||
"score": resp.cortiq.complexity_score,
|
||||
@@ -156,10 +314,230 @@ async fn handler(
|
||||
},
|
||||
"selected_model": resp.cortiq.selected_model,
|
||||
"route_source": resp.cortiq.route_source,
|
||||
"router_request_id": resp.cortiq.router_request_id,
|
||||
"cost_usd": resp.cortiq.cost_usd,
|
||||
});
|
||||
if let Some(rid) = &resp.cortiq.router_request_id {
|
||||
echo["router_request_id"] = serde_json::json!(rid);
|
||||
}
|
||||
body["cortiq"] = echo;
|
||||
}
|
||||
|
||||
Ok((headers, Json(body)).into_response())
|
||||
}
|
||||
|
||||
/// Rewrite an upstream SSE stream into strict OpenAI `chat.completion.chunk`
|
||||
/// events.
|
||||
///
|
||||
/// Every chunk is reissued with an id, `created`, and the *requested* model id;
|
||||
/// `message`-shaped chunks are converted to `delta`; stop reasons are mapped to
|
||||
/// the spec's closed set; empty deltas are dropped. The stream is guaranteed to
|
||||
/// open with a `role` delta, to carry exactly one terminal `finish_reason`, and
|
||||
/// to end with `data: [DONE]` — even when the upstream dies mid-flight, in
|
||||
/// which case the failure is delivered as an SSE `error` event so the client
|
||||
/// shows a message instead of waiting forever.
|
||||
fn normalize_sse(
|
||||
stream: crate::providers::ChatStream,
|
||||
model: String,
|
||||
) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
|
||||
use futures::StreamExt;
|
||||
async_stream::stream! {
|
||||
let fallback_id = response_id("");
|
||||
let created = now_secs();
|
||||
let mut buf = String::new();
|
||||
let mut id: Option<String> = None;
|
||||
// Which choice indices already carried a `role` delta (n > 1 gets one each).
|
||||
let mut roled: std::collections::HashSet<u64> = std::collections::HashSet::new();
|
||||
let mut sent_finish = false;
|
||||
let mut failure: Option<String> = None;
|
||||
|
||||
// Emit one normalised event; `None` means "nothing worth sending".
|
||||
macro_rules! chunk {
|
||||
($choices:expr, $extra:expr) => {{
|
||||
let mut evt = serde_json::json!({
|
||||
"id": id.clone().unwrap_or_else(|| fallback_id.clone()),
|
||||
"object": "chat.completion.chunk",
|
||||
"created": created,
|
||||
"model": model,
|
||||
"choices": $choices,
|
||||
});
|
||||
for (k, v) in $extra {
|
||||
evt[k] = v;
|
||||
}
|
||||
bytes::Bytes::from(format!("data: {evt}\n\n"))
|
||||
}};
|
||||
}
|
||||
|
||||
futures::pin_mut!(stream);
|
||||
'outer: while let Some(item) = stream.next().await {
|
||||
let bytes = match item {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
failure = Some(e.to_string());
|
||||
break 'outer;
|
||||
}
|
||||
};
|
||||
buf.push_str(&String::from_utf8_lossy(&bytes));
|
||||
|
||||
while let Some(idx) = buf.find("\n\n") {
|
||||
let event: String = buf.drain(..idx + 2).collect();
|
||||
for line in event.lines() {
|
||||
let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
|
||||
let data = data.trim();
|
||||
if data.is_empty() || data == "[DONE]" {
|
||||
continue;
|
||||
}
|
||||
let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
|
||||
|
||||
// An upstream error delivered inside the stream: pass it on
|
||||
// and stop — the client needs to see it, not time out.
|
||||
if let Some(err) = v.get("error").filter(|e| !e.is_null()) {
|
||||
failure = Some(err.to_string());
|
||||
break 'outer;
|
||||
}
|
||||
|
||||
if id.is_none() {
|
||||
id = v["id"].as_str().filter(|s| !s.is_empty()).map(str::to_string);
|
||||
}
|
||||
|
||||
let mut choices = Vec::new();
|
||||
if let Some(arr) = v["choices"].as_array() {
|
||||
for (i, c) in arr.iter().enumerate() {
|
||||
// Some upstreams send a full `message` per chunk
|
||||
// instead of a `delta`; both mean the same thing here.
|
||||
let src = if c["delta"].is_object() { &c["delta"] } else { &c["message"] };
|
||||
let index = c["index"].as_u64().unwrap_or(i as u64);
|
||||
let mut delta = serde_json::Map::new();
|
||||
if roled.insert(index) {
|
||||
delta.insert("role".into(), serde_json::json!("assistant"));
|
||||
}
|
||||
if let Some(text) = src["content"].as_str() {
|
||||
if !text.is_empty() {
|
||||
delta.insert("content".into(), serde_json::json!(text));
|
||||
}
|
||||
}
|
||||
// Reasoning traces and citations are not in the base
|
||||
// schema but every client that understands them
|
||||
// ignores them safely, so they are worth keeping.
|
||||
for key in ["reasoning_content", "reasoning", "thinking", "refusal", "annotations"] {
|
||||
if let Some(val) = src.get(key).filter(|v| !v.is_null()) {
|
||||
delta.insert(key.into(), val.clone());
|
||||
}
|
||||
}
|
||||
if let Some(tc) = src["tool_calls"].as_array().filter(|a| !a.is_empty()) {
|
||||
delta.insert("tool_calls".into(), serde_json::Value::Array(tc.clone()));
|
||||
}
|
||||
|
||||
let finish = c["finish_reason"].as_str().map(normalize_finish_reason);
|
||||
if finish.is_none() && delta.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let mut choice = serde_json::json!({
|
||||
"index": index,
|
||||
"delta": serde_json::Value::Object(delta),
|
||||
});
|
||||
if let Some(f) = finish {
|
||||
choice["finish_reason"] = serde_json::json!(f);
|
||||
sent_finish = true;
|
||||
}
|
||||
choices.push(choice);
|
||||
}
|
||||
}
|
||||
|
||||
// Terminal metadata rides on a choice-less chunk, exactly
|
||||
// as OpenAI ships `usage` when `stream_options` asks for it.
|
||||
let mut extra: Vec<(&str, serde_json::Value)> = Vec::new();
|
||||
if let Some(u) = v.get("usage").filter(|u| !u.is_null()) {
|
||||
extra.push(("usage", u.clone()));
|
||||
}
|
||||
for key in ["cortiq_cost_usd", "cortiq_estimated"] {
|
||||
if let Some(val) = v.get(key).filter(|v| !v.is_null()) {
|
||||
extra.push((key, val.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
if !choices.is_empty() || !extra.is_empty() {
|
||||
yield Ok(chunk!(choices, extra));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(message) = failure {
|
||||
let evt = serde_json::json!({
|
||||
"error": { "message": message, "type": "upstream_unavailable", "code": "stream_failed" }
|
||||
});
|
||||
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
|
||||
}
|
||||
|
||||
// Close the stream properly whatever the upstream did: a client that
|
||||
// never sees a stop reason and a [DONE] keeps its spinner turning.
|
||||
if !sent_finish {
|
||||
let mut delta = serde_json::Map::new();
|
||||
if roled.insert(0) {
|
||||
delta.insert("role".into(), serde_json::json!("assistant"));
|
||||
}
|
||||
let choices = vec![serde_json::json!({
|
||||
"index": 0,
|
||||
"delta": serde_json::Value::Object(delta),
|
||||
"finish_reason": "stop",
|
||||
})];
|
||||
let extra: Vec<(&str, serde_json::Value)> = Vec::new();
|
||||
yield Ok(chunk!(choices, extra));
|
||||
}
|
||||
yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn content_parts_and_null_are_accepted() {
|
||||
let body = serde_json::json!({
|
||||
"model": "cortiq-auto",
|
||||
"messages": [
|
||||
{"role": "user", "content": [{"type": "text", "text": "hello "}, {"type": "image_url", "image_url": {"url": "x"}}, {"type": "text", "text": "world"}]},
|
||||
{"role": "assistant", "content": null},
|
||||
{"role": "tool", "content": "42", "tool_call_id": "call_1"}
|
||||
],
|
||||
"tools": null,
|
||||
"reasoning_effort": "high",
|
||||
"chat_id": "owui-123"
|
||||
});
|
||||
let req: OpenAiChatRequest = serde_json::from_value(body).expect("must parse");
|
||||
assert_eq!(req.messages[0].content, "hello world");
|
||||
assert_eq!(req.messages[1].content, "");
|
||||
assert_eq!(req.messages[2].tool_call_id.as_deref(), Some("call_1"));
|
||||
assert!(req.tools.unwrap_or_default().is_empty());
|
||||
assert_eq!(
|
||||
req.reasoning_effort.as_ref().and_then(Effort::to_budget),
|
||||
Some(8192)
|
||||
);
|
||||
assert!(!sanitize_passthrough(req.rest).contains_key("chat_id"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn finish_reasons_map_to_the_openai_set() {
|
||||
assert_eq!(normalize_finish_reason("max_tokens"), "length");
|
||||
assert_eq!(normalize_finish_reason("eos"), "stop");
|
||||
assert_eq!(normalize_finish_reason("tool_use"), "tool_calls");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_is_closed_even_when_the_upstream_stops_early() {
|
||||
use futures::StreamExt;
|
||||
let upstream = futures::stream::iter(vec![Ok(bytes::Bytes::from(
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"}}]}\n\n",
|
||||
))]);
|
||||
let out: Vec<String> = normalize_sse(Box::pin(upstream), "cortiq-auto".into())
|
||||
.map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
|
||||
.collect()
|
||||
.await;
|
||||
let joined = out.join("");
|
||||
assert!(joined.contains("\"role\":\"assistant\""));
|
||||
assert!(joined.contains("\"model\":\"cortiq-auto\""));
|
||||
assert!(joined.contains("\"finish_reason\":\"stop\""));
|
||||
assert!(joined.trim_end().ends_with("data: [DONE]"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
|
||||
use crate::protocols::openai_chat::cortiq_headers;
|
||||
use crate::protocols::openai_chat::{cortiq_headers, normalize_finish_reason, sse_headers};
|
||||
use crate::state::SharedState;
|
||||
use axum::response::{IntoResponse, Response};
|
||||
use axum::{extract::State, routing::post, Json, Router};
|
||||
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<CompletionsRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.openai_completions {
|
||||
@@ -70,37 +71,37 @@ async fn handler(
|
||||
));
|
||||
}
|
||||
|
||||
// Echoed back as-is: clients match the reply against the id they asked for.
|
||||
let requested_model = req.model.clone();
|
||||
let canonical = ChatRequest {
|
||||
routing: parse_routing(&req.model),
|
||||
messages: vec![Message {
|
||||
role: "user".into(),
|
||||
content: prompt,
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
}],
|
||||
tools: vec![],
|
||||
params: GenParams {
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_completions".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
if req.stream {
|
||||
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let mut headers = cortiq_headers(&info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
axum::http::HeaderValue::from_static("text/event-stream"),
|
||||
);
|
||||
return Ok((
|
||||
headers,
|
||||
sse_headers(&info),
|
||||
axum::body::Body::from_stream(chat_to_completions_sse(stream)),
|
||||
)
|
||||
.into_response());
|
||||
@@ -112,11 +113,12 @@ async fn handler(
|
||||
.first()
|
||||
.map(|c| c.message.content.clone())
|
||||
.unwrap_or_default();
|
||||
let finish = resp
|
||||
.choices
|
||||
.first()
|
||||
.map(|c| c.finish_reason.clone())
|
||||
.unwrap_or_else(|| "stop".into());
|
||||
let finish = normalize_finish_reason(
|
||||
resp.choices
|
||||
.first()
|
||||
.map(|c| c.finish_reason.as_str())
|
||||
.unwrap_or("stop"),
|
||||
);
|
||||
let headers = cortiq_headers(&resp.cortiq);
|
||||
let body = serde_json::json!({
|
||||
"id": resp.id,
|
||||
@@ -125,7 +127,7 @@ async fn handler(
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs(),
|
||||
"model": resp.model_used,
|
||||
"model": requested_model,
|
||||
"choices": [{ "text": answer, "index": 0, "logprobs": serde_json::Value::Null, "finish_reason": finish }],
|
||||
"usage": {
|
||||
"prompt_tokens": resp.usage.prompt_tokens,
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
//! Inbound OpenAI Models adapter: `GET /v1/models`.
|
||||
//! Lists the configured pool (plus the virtual `cortiq-auto`) for client discovery.
|
||||
//! Lists everything a client can address: the virtual `cortiq-auto`, the
|
||||
//! configured `[[models]]` pool, and the managed local `[[cmf.servers]]`.
|
||||
//!
|
||||
//! The managed CMF servers belong here because they are routable model ids like
|
||||
//! any other — omitting them left OpenAI clients (Open WebUI among them) unable
|
||||
//! to pin a local model even though the gateway would happily serve it.
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::state::SharedState;
|
||||
@@ -9,6 +14,13 @@ pub fn routes() -> Router<SharedState> {
|
||||
Router::new().route("/v1/models", get(handler))
|
||||
}
|
||||
|
||||
fn now_secs() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse> {
|
||||
let live = state.live();
|
||||
if !live.cfg.protocols.openai_models {
|
||||
@@ -16,17 +28,51 @@ async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse>
|
||||
"openai_models protocol is disabled".into(),
|
||||
));
|
||||
}
|
||||
let created = now_secs();
|
||||
let mut data = vec![serde_json::json!({
|
||||
"id": "cortiq-auto",
|
||||
"object": "model",
|
||||
"created": created,
|
||||
"owned_by": "cortiq",
|
||||
})];
|
||||
let mut seen: std::collections::HashSet<String> =
|
||||
std::collections::HashSet::from(["cortiq-auto".to_string()]);
|
||||
|
||||
for m in &live.cfg.models {
|
||||
if !seen.insert(m.id.clone()) {
|
||||
continue;
|
||||
}
|
||||
data.push(serde_json::json!({
|
||||
"id": m.id,
|
||||
"object": "model",
|
||||
"created": created,
|
||||
"owned_by": m.provider,
|
||||
}));
|
||||
}
|
||||
|
||||
// Managed local CMF servers. Health is reported alongside (a non-standard
|
||||
// field clients ignore) rather than used as a filter: a large `.cmf` takes
|
||||
// minutes to load, and a model that vanishes from the list while it warms up
|
||||
// is worse than one marked "not ready yet".
|
||||
let cmf = state.cmf.status();
|
||||
for s in live.cfg.cmf.effective_servers() {
|
||||
if !seen.insert(s.id.clone()) {
|
||||
continue; // a static [[models]] entry with the same id wins
|
||||
}
|
||||
let status = cmf.servers.iter().find(|st| st.id == s.id);
|
||||
data.push(serde_json::json!({
|
||||
"id": s.id,
|
||||
"object": "model",
|
||||
"created": created,
|
||||
"owned_by": "cortiq-cmf",
|
||||
"cortiq": {
|
||||
"kind": "cmf",
|
||||
"local": true,
|
||||
"running": status.map(|st| st.running).unwrap_or(false),
|
||||
"healthy": status.map(|st| st.healthy).unwrap_or(false),
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
Ok(Json(serde_json::json!({ "object": "list", "data": data })))
|
||||
}
|
||||
|
||||
@@ -32,7 +32,13 @@ impl AnthropicProvider {
|
||||
caps: Caps::from_list(&cfg.caps),
|
||||
price_in: cfg.price_in,
|
||||
price_out: cfg.price_out,
|
||||
http: reqwest::Client::new(),
|
||||
// connect timeout: a dead host fails over in seconds instead of
|
||||
// hanging the request; NO overall timeout — that would cut long
|
||||
// generations and streams (OpenRouter can be slow legitimately)
|
||||
http: reqwest::Client::builder()
|
||||
.connect_timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.unwrap_or_else(|_| reqwest::Client::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -140,6 +146,7 @@ impl Provider for AnthropicProvider {
|
||||
role: "assistant".into(),
|
||||
content,
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
},
|
||||
finish_reason: v["stop_reason"].as_str().unwrap_or("stop").to_string(),
|
||||
}],
|
||||
|
||||
+17
-1
@@ -28,7 +28,13 @@ impl OpenAiProvider {
|
||||
caps: Caps::from_list(&cfg.caps),
|
||||
price_in: cfg.price_in,
|
||||
price_out: cfg.price_out,
|
||||
http: reqwest::Client::new(),
|
||||
// connect timeout: a dead host fails over in seconds instead of
|
||||
// hanging the request; NO overall timeout — that would cut long
|
||||
// generations and streams (OpenRouter can be slow legitimately)
|
||||
http: reqwest::Client::builder()
|
||||
.connect_timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.unwrap_or_else(|_| reqwest::Client::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -42,9 +48,18 @@ impl OpenAiProvider {
|
||||
if let Some(t) = req.params.temperature {
|
||||
body["temperature"] = t.into();
|
||||
}
|
||||
if let Some(p) = req.params.top_p {
|
||||
body["top_p"] = p.into();
|
||||
}
|
||||
if let Some(m) = req.params.max_tokens {
|
||||
body["max_tokens"] = m.into();
|
||||
}
|
||||
if let Some(tb) = req.params.think_budget {
|
||||
body["think_budget"] = tb.into();
|
||||
body["reasoning_effort"] = tb.into();
|
||||
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
|
||||
body["enable_thinking"] = (tb > 0).into();
|
||||
}
|
||||
if !req.tools.is_empty() {
|
||||
body["tools"] = serde_json::Value::Array(req.tools.clone());
|
||||
}
|
||||
@@ -120,6 +135,7 @@ impl Provider for OpenAiProvider {
|
||||
role,
|
||||
content,
|
||||
tool_calls,
|
||||
..Default::default()
|
||||
},
|
||||
finish_reason: c["finish_reason"].as_str().unwrap_or("stop").to_string(),
|
||||
});
|
||||
|
||||
+40
-2
@@ -1,7 +1,7 @@
|
||||
//! Model pool registry: builds providers from `[[models]]` and resolves
|
||||
//! `model_id → Arc<dyn Provider>`.
|
||||
|
||||
use crate::config::Config;
|
||||
use crate::config::{Config, ModelCfg};
|
||||
use crate::providers::{anthropic::AnthropicProvider, openai::OpenAiProvider, Provider};
|
||||
use crate::secrets::SecretStore;
|
||||
use std::collections::HashMap;
|
||||
@@ -21,15 +21,53 @@ impl Registry {
|
||||
.as_ref()
|
||||
.and_then(|name| secrets.resolve(name));
|
||||
let provider: Arc<dyn Provider> = match m.provider.as_str() {
|
||||
"openai" | "ollama" | "http" => Arc::new(OpenAiProvider::new(m, api_key)?),
|
||||
// openrouter + lmstudio are OpenAI-compatible (Bearer + /chat/completions)
|
||||
"openai" | "ollama" | "http" | "openrouter" | "lmstudio" => {
|
||||
Arc::new(OpenAiProvider::new(m, api_key)?)
|
||||
}
|
||||
"anthropic" => Arc::new(AnthropicProvider::new(m, api_key)?),
|
||||
other => anyhow::bail!("unknown provider '{}' for model '{}'", other, m.id),
|
||||
};
|
||||
by_id.insert(m.id.clone(), provider);
|
||||
}
|
||||
|
||||
// Managed local CMF servers: register each as an OpenAI-compatible
|
||||
// provider so it can be routed to like any other backend — including when
|
||||
// the router is unavailable or the gateway is in local-only mode.
|
||||
for s in cfg.cmf.effective_servers() {
|
||||
if by_id.contains_key(&s.id) {
|
||||
continue; // a static [[models]] entry with the same id wins
|
||||
}
|
||||
let m = ModelCfg {
|
||||
id: s.id.clone(),
|
||||
provider: "openai".into(),
|
||||
base_url: format!("http://{}:{}/v1", cfg.cmf.local_host, s.port),
|
||||
model: "cortiq".into(),
|
||||
cost_tier: "local".into(),
|
||||
price_in: 0.0,
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: s.caps.clone(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
think_budget: s.think_budget,
|
||||
system_prompt: s.system_prompt.clone(),
|
||||
o1: s.o1.clone(),
|
||||
skip_mtp: s.skip_mtp,
|
||||
};
|
||||
by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
|
||||
}
|
||||
Ok(Self { by_id })
|
||||
}
|
||||
|
||||
/// Any model id, preferring a real registered one (used as a local-first
|
||||
/// fallback target when routing cannot pick a specific model).
|
||||
pub fn any_id(&self) -> Option<String> {
|
||||
self.by_id.keys().next().cloned()
|
||||
}
|
||||
|
||||
pub fn get(&self, model_id: &str) -> Option<Arc<dyn Provider>> {
|
||||
self.by_id.get(model_id).cloned()
|
||||
}
|
||||
|
||||
+64
-3
@@ -4,16 +4,48 @@
|
||||
|
||||
use crate::config::RouterCfg;
|
||||
use crate::model::RouteDecision;
|
||||
use crate::secrets::SecretStore;
|
||||
use std::sync::atomic::{AtomicU16, Ordering};
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Outcome of the most recent real `/v1/route` call, shared with the admin health
|
||||
/// endpoint so the panel can tell "no money / bad key" apart from "router down".
|
||||
/// 0 = ok (or no calls yet); [`STATUS_NETWORK`] / [`STATUS_TIMEOUT`] for transport
|
||||
/// failures; otherwise the upstream HTTP status code.
|
||||
pub type RouterLastStatus = Arc<AtomicU16>;
|
||||
|
||||
pub const STATUS_NETWORK: u16 = 1;
|
||||
pub const STATUS_TIMEOUT: u16 = 2;
|
||||
|
||||
/// Error kind for the admin panel.
|
||||
pub fn classify_status(code: u16) -> &'static str {
|
||||
match code {
|
||||
STATUS_NETWORK => "unreachable",
|
||||
STATUS_TIMEOUT => "timeout",
|
||||
401 | 403 => "auth",
|
||||
402 => "payment",
|
||||
429 => "quota",
|
||||
_ => "error",
|
||||
}
|
||||
}
|
||||
|
||||
pub struct RouterClient {
|
||||
http: reqwest::Client,
|
||||
url: String,
|
||||
api_key: Option<String>,
|
||||
taxonomy_id: Option<String>,
|
||||
last_status: RouterLastStatus,
|
||||
}
|
||||
|
||||
impl RouterClient {
|
||||
pub fn new(cfg: &RouterCfg) -> anyhow::Result<Self> {
|
||||
/// The key is resolved through the secret store (admin panel value first,
|
||||
/// then the environment variable) so a key pasted in the console works
|
||||
/// without a restart.
|
||||
pub fn new(
|
||||
cfg: &RouterCfg,
|
||||
secrets: &SecretStore,
|
||||
last_status: RouterLastStatus,
|
||||
) -> anyhow::Result<Self> {
|
||||
let mut builder =
|
||||
reqwest::Client::builder().timeout(std::time::Duration::from_millis(cfg.timeout_ms));
|
||||
|
||||
@@ -27,17 +59,35 @@ impl RouterClient {
|
||||
let api_key = cfg
|
||||
.api_key_env
|
||||
.as_ref()
|
||||
.and_then(|name| std::env::var(name).ok());
|
||||
.and_then(|name| secrets.resolve(name));
|
||||
Ok(Self {
|
||||
http,
|
||||
url: cfg.url.trim_end_matches('/').to_string(),
|
||||
api_key,
|
||||
taxonomy_id: cfg.taxonomy_id.clone(),
|
||||
last_status,
|
||||
})
|
||||
}
|
||||
|
||||
/// Classify text. Returns `Ok(None)` on network error/timeout —
|
||||
/// the caller treats this as "router unavailable" and uses the default tier.
|
||||
/// `GET /v1/taxonomies` — the account's taxonomies with their label sets.
|
||||
/// `Ok(None)` when the router is disabled or has no key configured.
|
||||
pub async fn taxonomies(&self) -> anyhow::Result<Option<serde_json::Value>> {
|
||||
if self.url.is_empty() {
|
||||
return Ok(None);
|
||||
}
|
||||
let mut req = self.http.get(format!("{}/v1/taxonomies", self.url));
|
||||
if let Some(k) = &self.api_key {
|
||||
req = req.bearer_auth(k);
|
||||
}
|
||||
let resp = req.send().await?;
|
||||
if !resp.status().is_success() {
|
||||
anyhow::bail!("router /v1/taxonomies: HTTP {}", resp.status());
|
||||
}
|
||||
Ok(Some(resp.json().await?))
|
||||
}
|
||||
|
||||
pub async fn route(&self, text: &str, profile: &str) -> anyhow::Result<Option<RouteDecision>> {
|
||||
let mut body = serde_json::json!({
|
||||
"input": { "text": text },
|
||||
@@ -54,11 +104,22 @@ impl RouterClient {
|
||||
|
||||
let resp = match req.send().await {
|
||||
Ok(r) => r,
|
||||
Err(_) => return Ok(None), // degradation
|
||||
Err(e) => {
|
||||
let code = if e.is_timeout() {
|
||||
STATUS_TIMEOUT
|
||||
} else {
|
||||
STATUS_NETWORK
|
||||
};
|
||||
self.last_status.store(code, Ordering::Relaxed);
|
||||
return Ok(None); // degradation
|
||||
}
|
||||
};
|
||||
if !resp.status().is_success() {
|
||||
self.last_status
|
||||
.store(resp.status().as_u16(), Ordering::Relaxed);
|
||||
return Ok(None);
|
||||
}
|
||||
self.last_status.store(0, Ordering::Relaxed);
|
||||
let v: serde_json::Value = resp.json().await?;
|
||||
let d = &v["decision"];
|
||||
Ok(Some(RouteDecision {
|
||||
|
||||
+212
-17
@@ -3,38 +3,117 @@
|
||||
|
||||
use crate::config::{Config, TierTargets};
|
||||
|
||||
fn to_list(t: &TierTargets) -> Vec<String> {
|
||||
t.models()
|
||||
}
|
||||
|
||||
/// A task rule, normalized: per-tier model lists ("which model serves this
|
||||
/// task at each complexity") plus any-tier models that lead regardless.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct TaskRule {
|
||||
pub any: Vec<String>,
|
||||
pub per: std::collections::HashMap<String, Vec<String>>,
|
||||
}
|
||||
|
||||
pub struct RoutingTable {
|
||||
tiers: std::collections::HashMap<String, Vec<String>>,
|
||||
/// Per-task rules — a matching rule's models lead the candidate chain.
|
||||
tasks: std::collections::HashMap<String, TaskRule>,
|
||||
default: String,
|
||||
}
|
||||
|
||||
impl RoutingTable {
|
||||
pub fn from_config(cfg: &Config) -> Self {
|
||||
let tiers = cfg
|
||||
.routing
|
||||
.tiers
|
||||
.iter()
|
||||
.map(|(tier, targets)| {
|
||||
let list = match targets {
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
(tier.clone(), list)
|
||||
})
|
||||
.collect();
|
||||
Self {
|
||||
tiers,
|
||||
tiers: cfg
|
||||
.routing
|
||||
.tiers
|
||||
.iter()
|
||||
.map(|(tier, targets)| (tier.clone(), to_list(targets)))
|
||||
.collect(),
|
||||
tasks: cfg
|
||||
.routing
|
||||
.tasks
|
||||
.iter()
|
||||
.map(|(task, targets)| {
|
||||
let (any, per) = targets.task_rule();
|
||||
(task.clone(), TaskRule { any, per })
|
||||
})
|
||||
.filter(|(_, r)| !r.any.is_empty() || r.per.values().any(|v| !v.is_empty()))
|
||||
.collect(),
|
||||
default: cfg.routing.default.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Candidate chain for `(task, tier)` and whether a task rule shaped it.
|
||||
///
|
||||
/// Order of preference:
|
||||
/// 1. the task's models FOR THIS TIER (the user said exactly which model
|
||||
/// serves this task at this complexity);
|
||||
/// 2. at tier=high — the global high chain (escalation: any-tier task
|
||||
/// models must not undercut it), then the task's any-tier models;
|
||||
/// otherwise the task's any-tier models, then the global tier chain;
|
||||
/// 3. `default` as the last resort. Deduplicated, empties dropped.
|
||||
pub fn candidates_for(&self, task: &str, tier: &str) -> (Vec<String>, bool) {
|
||||
fn push(out: &mut Vec<String>, id: &str) {
|
||||
if !id.is_empty() && !out.iter().any(|x| x == id) {
|
||||
out.push(id.to_string());
|
||||
}
|
||||
}
|
||||
let mut out: Vec<String> = Vec::new();
|
||||
let rule = self.tasks.get(task);
|
||||
let tier_chain = self
|
||||
.tiers
|
||||
.get(tier)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| vec![self.default.clone()]);
|
||||
if let Some(r) = rule {
|
||||
if let Some(list) = r.per.get(tier) {
|
||||
for id in list {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
}
|
||||
let escalate = tier == "high" && self.tiers.contains_key("high");
|
||||
if escalate {
|
||||
// high verdicts escalate over any-tier task pins
|
||||
for id in &tier_chain {
|
||||
push(&mut out, id);
|
||||
}
|
||||
if let Some(r) = rule {
|
||||
for id in &r.any {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if let Some(r) = rule {
|
||||
for id in &r.any {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
for id in &tier_chain {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
let d = self.default.clone();
|
||||
push(&mut out, &d);
|
||||
// "task-rule" must mean "the rule chose the head of this chain" — a
|
||||
// rule that exists but lost to the tier chain (math has a high pick,
|
||||
// the request came in low) must not claim the credit.
|
||||
let matched = match (rule, out.first()) {
|
||||
(Some(r), Some(head)) => {
|
||||
r.per.get(tier).map(|v| v.contains(head)).unwrap_or(false)
|
||||
|| r.any.first() == Some(head)
|
||||
}
|
||||
_ => false,
|
||||
};
|
||||
(out, matched)
|
||||
}
|
||||
|
||||
/// Candidates for the given complexity tier, in preference order.
|
||||
/// If the tier is not defined, the sole candidate is `default`.
|
||||
pub fn candidates(&self, tier: &str) -> Vec<String> {
|
||||
self.tiers
|
||||
.get(tier)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| vec![self.default.clone()])
|
||||
self.candidates_for("", tier).0
|
||||
}
|
||||
|
||||
/// Default model (used when the router is unavailable).
|
||||
@@ -46,3 +125,119 @@ impl RoutingTable {
|
||||
// TODO(v0.2): cost_aware mode and circuit breaker — pick the cheapest available
|
||||
// model no lower than min_class[tier], respecting max_cost_usd_per_request,
|
||||
// and skip models whose circuit breaker is open.
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn table(tasks: &[(&str, &[&str])], tiers: &[(&str, &[&str])], default: &str) -> RoutingTable {
|
||||
RoutingTable {
|
||||
tiers: tiers
|
||||
.iter()
|
||||
.map(|(k, v)| (k.to_string(), v.iter().map(|s| s.to_string()).collect()))
|
||||
.collect(),
|
||||
tasks: tasks
|
||||
.iter()
|
||||
.map(|(k, v)| {
|
||||
(
|
||||
k.to_string(),
|
||||
TaskRule {
|
||||
any: v.iter().map(|s| s.to_string()).collect(),
|
||||
per: Default::default(),
|
||||
},
|
||||
)
|
||||
})
|
||||
.collect(),
|
||||
default: default.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn with_task_tier_models(
|
||||
mut t: RoutingTable,
|
||||
task: &str,
|
||||
tier: &str,
|
||||
models: &[&str],
|
||||
) -> RoutingTable {
|
||||
t.tasks.entry(task.to_string()).or_default().per.insert(
|
||||
tier.to_string(),
|
||||
models.iter().map(|s| s.to_string()).collect(),
|
||||
);
|
||||
t
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn task_rule_leads_tier_chain_follows() {
|
||||
let t = table(
|
||||
&[("translation", &["local"])],
|
||||
&[("low", &["cheap", "local"]), ("high", &["big"])],
|
||||
"cheap",
|
||||
);
|
||||
// dedup: rule model already in the tier chain appears once, first
|
||||
let (c, matched) = t.candidates_for("translation", "low");
|
||||
assert!(matched);
|
||||
assert_eq!(c, vec!["local", "cheap"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn high_tier_escalates_over_task_rule() {
|
||||
let t = table(
|
||||
&[("code", &["cheap-coder"])],
|
||||
&[("low", &["cheap"]), ("high", &["big"])],
|
||||
"cheap",
|
||||
);
|
||||
// medium (no chain defined → default): task rule still leads
|
||||
let (c, m) = t.candidates_for("code", "medium");
|
||||
assert!(m);
|
||||
assert_eq!(c[0], "cheap-coder");
|
||||
// high: tier chain leads, task rule follows as fallback
|
||||
let (c, m) = t.candidates_for("code", "high");
|
||||
// escalation put the global high chain first → the rule did not pick
|
||||
// the head, so no task-rule credit
|
||||
assert!(!m);
|
||||
// default rides along as the very last fallback
|
||||
assert_eq!(c, vec!["big", "cheap-coder", "cheap"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn per_tier_models_lead_their_tier() {
|
||||
let base = table(
|
||||
&[("code", &["generic-coder"])],
|
||||
&[("low", &["cheap"]), ("high", &["big"])],
|
||||
"cheap",
|
||||
);
|
||||
let t = with_task_tier_models(base, "code", "high", &["best-coder"]);
|
||||
// at high: the task's HIGH pick leads, escalation chain next,
|
||||
// any-tier task models after it, default last
|
||||
let (c, m) = t.candidates_for("code", "high");
|
||||
assert!(m);
|
||||
assert_eq!(c, vec!["best-coder", "big", "generic-coder", "cheap"]);
|
||||
// at low: any-tier task models lead as before (and the badge is
|
||||
// honest — the head came from the rule)
|
||||
let (c, m) = t.candidates_for("code", "low");
|
||||
assert!(m);
|
||||
assert_eq!(c, vec!["generic-coder", "cheap"]);
|
||||
// a tier with an explicit pick but no any-models
|
||||
let t2 = with_task_tier_models(
|
||||
table(&[], &[("low", &["cheap"]), ("high", &["big"])], "cheap"),
|
||||
"legal",
|
||||
"high",
|
||||
&["strong"],
|
||||
);
|
||||
let (c, m) = t2.candidates_for("legal", "high");
|
||||
assert!(m);
|
||||
assert_eq!(c, vec!["strong", "big", "cheap"]);
|
||||
// the same rule at LOW did not shape the head → no task-rule credit
|
||||
let (c, m) = t2.candidates_for("legal", "low");
|
||||
assert!(!m);
|
||||
assert_eq!(c, vec!["cheap"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_rule_falls_back_to_tier_then_default() {
|
||||
let t = table(&[], &[("low", &["a"])], "d");
|
||||
assert_eq!(t.candidates_for("qa", "low").0, vec!["a", "d"]);
|
||||
let (c, matched) = t.candidates_for("qa", "unknown-tier");
|
||||
assert!(!matched);
|
||||
assert_eq!(c, vec!["d"]);
|
||||
}
|
||||
}
|
||||
|
||||
+151
-6
@@ -8,7 +8,7 @@
|
||||
use crate::cache::SemanticCache;
|
||||
use crate::config::Config;
|
||||
use crate::registry::Registry;
|
||||
use crate::router_client::RouterClient;
|
||||
use crate::router_client::{RouterClient, RouterLastStatus};
|
||||
use crate::routing::RoutingTable;
|
||||
use crate::secrets::SecretStore;
|
||||
use crate::stats::Stats;
|
||||
@@ -27,8 +27,12 @@ pub struct Live {
|
||||
}
|
||||
|
||||
impl Live {
|
||||
pub fn build(cfg: Config, secrets: &SecretStore) -> anyhow::Result<Self> {
|
||||
let router = RouterClient::new(&cfg.router)?;
|
||||
pub fn build(
|
||||
cfg: Config,
|
||||
secrets: &SecretStore,
|
||||
router_status: RouterLastStatus,
|
||||
) -> anyhow::Result<Self> {
|
||||
let router = RouterClient::new(&cfg.router, secrets, router_status)?;
|
||||
let registry = Registry::from_config(&cfg, secrets)?;
|
||||
let routing = RoutingTable::from_config(&cfg);
|
||||
Ok(Self {
|
||||
@@ -38,15 +42,110 @@ impl Live {
|
||||
router,
|
||||
})
|
||||
}
|
||||
|
||||
/// Local-first fallback targets, in order: the managed local CMF model (if
|
||||
/// configured), then the routing default, then any registered model — only
|
||||
/// registered ids are returned. Used whenever the router is bypassed
|
||||
/// (local-only / disabled) or unavailable, so CMF models serve with no router.
|
||||
pub fn local_candidates(&self) -> Vec<String> {
|
||||
let mut v = Vec::new();
|
||||
for s in self.cfg.cmf.effective_servers() {
|
||||
v.push(s.id);
|
||||
}
|
||||
let def = self.routing.default_model().to_string();
|
||||
if !def.is_empty() && !v.contains(&def) {
|
||||
v.push(def);
|
||||
}
|
||||
v.retain(|id| self.registry.get(id).is_some());
|
||||
if v.is_empty() {
|
||||
if let Some(a) = self.registry.any_id() {
|
||||
v.push(a);
|
||||
}
|
||||
}
|
||||
v
|
||||
}
|
||||
}
|
||||
|
||||
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
|
||||
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
|
||||
/// fine for office-grade fairness and costs one HashMap lookup per request.
|
||||
#[derive(Default)]
|
||||
pub struct RateWindows {
|
||||
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
|
||||
}
|
||||
|
||||
impl RateWindows {
|
||||
pub fn allow(&self, key: &str, per_min: u32) -> bool {
|
||||
let minute = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
/ 60;
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
let e = g.entry(key.to_string()).or_insert((minute, 0));
|
||||
if e.0 != minute {
|
||||
*e = (minute, 0);
|
||||
}
|
||||
if e.1 >= per_min {
|
||||
return false;
|
||||
}
|
||||
e.1 += 1;
|
||||
true
|
||||
}
|
||||
/// Seconds until the current minute window rolls over.
|
||||
pub fn retry_after_secs(&self) -> u64 {
|
||||
60 - (std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
% 60)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct AppState {
|
||||
pub live: ArcSwap<Live>,
|
||||
pub stats: Arc<Stats>,
|
||||
pub promotion: Arc<crate::promotion::Promotion>,
|
||||
pub imports: Arc<crate::import::JobStore>,
|
||||
/// Image/video generation jobs (cortiq imagine / animate).
|
||||
pub media: Arc<crate::media::MediaStore>,
|
||||
/// One-click local model benchmarks (cortiq bench --json).
|
||||
pub bench: Arc<crate::bench::BenchStore>,
|
||||
/// Host metrics sampler (CPU / RAM / disk) for the dashboard.
|
||||
pub sysmon: Arc<crate::sysmon::SysMon>,
|
||||
/// Newest cortiq-gateway version on crates.io, when newer than ours.
|
||||
pub gateway_update: Arc<std::sync::Mutex<Option<String>>>,
|
||||
/// Admin console reachable from other machines without a token — the
|
||||
/// footgun banner flag (set at startup, shown by the SPA).
|
||||
pub admin_open_public: std::sync::atomic::AtomicBool,
|
||||
/// Managed local CMF model server (install/update/spawn lifecycle + status).
|
||||
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
|
||||
pub cache: Arc<SemanticCache>,
|
||||
/// Per-API-key fixed-window rate limiter.
|
||||
pub rate: RateWindows,
|
||||
pub secrets: SecretStore,
|
||||
pub config_path: String,
|
||||
pub pipeline: crate::pipeline::Pipeline,
|
||||
/// Outcome of the most recent router call (survives config reloads) —
|
||||
/// lets the admin panel distinguish a bad/expired key from a down router.
|
||||
pub router_status: RouterLastStatus,
|
||||
}
|
||||
|
||||
/// Map the user-facing `[shadow]` config to the promotion-table tuning.
|
||||
fn promotion_cfg(s: &crate::config::ShadowCfg) -> crate::promotion::PromotionCfg {
|
||||
crate::promotion::PromotionCfg {
|
||||
enabled: s.enabled && !s.local_model_id.is_empty(),
|
||||
window: s.window,
|
||||
n_min: s.n_min,
|
||||
promote_lb: s.promote_lb,
|
||||
soak: s.soak,
|
||||
file: if s.file.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(s.file.clone())
|
||||
},
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
impl std::ops::Deref for SharedState {
|
||||
@@ -61,15 +160,53 @@ impl SharedState {
|
||||
let secrets_path = sibling(&config_path, "secrets.toml");
|
||||
let secrets = SecretStore::load(&secrets_path);
|
||||
let stats = Stats::new(&cfg.stats);
|
||||
let promotion = crate::promotion::Promotion::new(promotion_cfg(&cfg.shadow));
|
||||
let imports = crate::import::JobStore::new();
|
||||
let media = crate::media::MediaStore::new();
|
||||
let bench = crate::bench::BenchStore::new();
|
||||
// jobs survive restarts: files under <data>/, reconciled on load
|
||||
let data = crate::config::data_dir();
|
||||
let _ = std::fs::create_dir_all(&data);
|
||||
imports.attach_persistence(data.join("jobs-import.json"));
|
||||
media.attach_persistence(data.join("jobs-media.json"));
|
||||
let sysmon = crate::sysmon::spawn(cfg.cmf.models_dir.clone());
|
||||
let gateway_update: Arc<std::sync::Mutex<Option<String>>> =
|
||||
Arc::new(std::sync::Mutex::new(None));
|
||||
{
|
||||
// check crates.io for a newer gateway once at start and twice a day
|
||||
let slot = gateway_update.clone();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
if let Some(latest) = crate::cmf_runtime::latest_gateway_version().await {
|
||||
let newer =
|
||||
crate::cmf_runtime::version_lt(env!("CARGO_PKG_VERSION"), &latest);
|
||||
*slot.lock().unwrap() = newer.then_some(latest);
|
||||
}
|
||||
tokio::time::sleep(std::time::Duration::from_secs(12 * 3600)).await;
|
||||
}
|
||||
});
|
||||
}
|
||||
let cmf = crate::cmf_runtime::CmfRuntime::new();
|
||||
let cache = SemanticCache::new(&cfg.cache);
|
||||
let live = Live::build(cfg, &secrets)?;
|
||||
let router_status = RouterLastStatus::default();
|
||||
let live = Live::build(cfg, &secrets, router_status.clone())?;
|
||||
Ok(SharedState(Arc::new(AppState {
|
||||
live: ArcSwap::from_pointee(live),
|
||||
stats,
|
||||
promotion,
|
||||
imports,
|
||||
media,
|
||||
bench,
|
||||
sysmon,
|
||||
gateway_update,
|
||||
admin_open_public: std::sync::atomic::AtomicBool::new(false),
|
||||
cmf,
|
||||
cache,
|
||||
rate: RateWindows::default(),
|
||||
secrets,
|
||||
config_path,
|
||||
pipeline: crate::pipeline::Pipeline::new(),
|
||||
router_status,
|
||||
})))
|
||||
}
|
||||
|
||||
@@ -84,9 +221,17 @@ impl AppState {
|
||||
/// If any step fails, the active state and file are left unchanged.
|
||||
pub fn reload(&self, new_cfg: Config) -> anyhow::Result<()> {
|
||||
new_cfg.validate()?;
|
||||
let live = Live::build(new_cfg.clone(), &self.secrets)?;
|
||||
let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
|
||||
new_cfg.save(&self.config_path)?;
|
||||
self.live.store(Arc::new(live));
|
||||
|
||||
let cmf_rt = self.cmf.clone();
|
||||
let cmf_cfg = new_cfg.cmf.clone();
|
||||
tokio::spawn(async move {
|
||||
cmf_rt.stop().await;
|
||||
crate::cmf_runtime::manage(cmf_rt, cmf_cfg).await;
|
||||
});
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -94,7 +239,7 @@ impl AppState {
|
||||
/// (e.g. after a secret change — only provider keys changed).
|
||||
pub fn rebuild(&self) -> anyhow::Result<()> {
|
||||
let cfg = self.live.load_full().cfg.clone();
|
||||
let live = Live::build(cfg, &self.secrets)?;
|
||||
let live = Live::build(cfg, &self.secrets, self.router_status.clone())?;
|
||||
self.live.store(Arc::new(live));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
+350
-42
@@ -55,7 +55,7 @@ pub struct RequestRecord {
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize)]
|
||||
#[derive(Clone, Debug, Default, Serialize, serde::Deserialize)]
|
||||
pub struct Totals {
|
||||
pub requests: u64,
|
||||
pub ok: u64,
|
||||
@@ -83,9 +83,23 @@ impl Totals {
|
||||
self.cost_usd += r.cost_usd;
|
||||
self.latency_ms_sum += r.latency_ms;
|
||||
}
|
||||
/// Accumulate another Totals into this one (for windowed aggregation).
|
||||
fn merge(&mut self, o: &Totals) {
|
||||
self.requests += o.requests;
|
||||
self.ok += o.ok;
|
||||
self.errors += o.errors;
|
||||
self.failovers += o.failovers;
|
||||
self.prompt_tokens += o.prompt_tokens;
|
||||
self.completion_tokens += o.completion_tokens;
|
||||
self.cost_usd += o.cost_usd;
|
||||
self.latency_ms_sum += o.latency_ms_sum;
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-minute bucket for time-series charts.
|
||||
/// Per-minute bucket. Beyond the chart fields it carries the per-group breakdown
|
||||
/// and failovers so the admin snapshot can be computed for ANY time window by
|
||||
/// summing the in-window buckets (the internal fields are not serialized into
|
||||
/// the `series` payload).
|
||||
#[derive(Clone, Debug, Default, Serialize)]
|
||||
pub struct Bucket {
|
||||
pub minute: u64, // unix seconds, rounded down to the minute
|
||||
@@ -95,17 +109,41 @@ pub struct Bucket {
|
||||
pub completion_tokens: u64,
|
||||
pub cost_usd: f64,
|
||||
pub latency_ms_sum: u64,
|
||||
#[serde(skip)]
|
||||
pub failovers: u64,
|
||||
#[serde(skip)]
|
||||
pub by_model: HashMap<String, Totals>,
|
||||
#[serde(skip)]
|
||||
pub by_account: HashMap<String, Totals>,
|
||||
#[serde(skip)]
|
||||
pub by_tier: HashMap<String, Totals>,
|
||||
#[serde(skip)]
|
||||
pub by_label: HashMap<String, Totals>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct Inner {
|
||||
/// All-time totals + per-model (used by the Prometheus counters, which must
|
||||
/// be monotonic — the windowed admin snapshot is derived from the buckets).
|
||||
total: Totals,
|
||||
by_model: HashMap<String, Totals>,
|
||||
by_account: HashMap<String, Totals>,
|
||||
by_tier: HashMap<String, Totals>,
|
||||
by_label: HashMap<String, Totals>,
|
||||
/// Per-minute buckets — the single source for the windowed admin snapshot.
|
||||
buckets: VecDeque<Bucket>,
|
||||
recent: VecDeque<RequestRecord>,
|
||||
/// Per-account totals over EVERY record still in the JSONL file (no
|
||||
/// retention gate) — per-key accounting for the keys page.
|
||||
by_account_alltime: HashMap<String, Totals>,
|
||||
/// Per-account totals folded out of the file by compaction. alltime(key)
|
||||
/// = folded + by_account_alltime, with no double counting: a record is
|
||||
/// either in the file or folded, never both.
|
||||
folded: HashMap<String, Totals>,
|
||||
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
/// Records with ts ≤ this are already reflected in the persisted
|
||||
/// month_spend (watermark against double counting on replay).
|
||||
month_upto_ts: u64,
|
||||
/// Appends since the last compaction size check.
|
||||
writes_since_check: u32,
|
||||
}
|
||||
|
||||
pub struct Stats {
|
||||
@@ -113,6 +151,7 @@ pub struct Stats {
|
||||
file: Option<String>,
|
||||
ring_size: usize,
|
||||
retention_secs: u64,
|
||||
max_file_bytes: u64,
|
||||
inner: Mutex<Inner>,
|
||||
}
|
||||
|
||||
@@ -128,9 +167,12 @@ impl Stats {
|
||||
file,
|
||||
ring_size: cfg.ring_size.max(1),
|
||||
retention_secs: parse_duration_secs(&cfg.retention).unwrap_or(7 * 86_400),
|
||||
max_file_bytes: cfg.max_file_mb.saturating_mul(1024 * 1024),
|
||||
inner: Mutex::new(Inner::default()),
|
||||
};
|
||||
s.load_folded();
|
||||
s.replay();
|
||||
s.maybe_compact(true);
|
||||
std::sync::Arc::new(s)
|
||||
}
|
||||
|
||||
@@ -151,6 +193,12 @@ impl Stats {
|
||||
// push everything to the "recent" ring (it self-limits by size),
|
||||
// but apply to aggregates/series only entries within the window.
|
||||
Self::push_recent(&mut inner, &rec, self.ring_size);
|
||||
inner
|
||||
.by_account_alltime
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
if rec.ts >= cutoff {
|
||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||
}
|
||||
@@ -158,6 +206,33 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
|
||||
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
|
||||
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
|
||||
return;
|
||||
}
|
||||
let m = month_of(rec.ts);
|
||||
let e = inner
|
||||
.month_spend
|
||||
.entry(rec.account.clone())
|
||||
.or_insert((m.clone(), 0.0));
|
||||
if e.0 != m {
|
||||
*e = (m, 0.0);
|
||||
}
|
||||
e.1 += rec.cost_usd;
|
||||
}
|
||||
|
||||
/// Month-to-date spend for an account (budget checks).
|
||||
pub fn month_spend(&self, account: &str) -> f64 {
|
||||
let inner = self.inner.lock().unwrap();
|
||||
let m = month_of(now_secs());
|
||||
inner
|
||||
.month_spend
|
||||
.get(account)
|
||||
.filter(|(mm, _)| *mm == m)
|
||||
.map(|(_, v)| *v)
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
|
||||
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
|
||||
inner.recent.push_back(rec.clone());
|
||||
while inner.recent.len() > ring_size {
|
||||
@@ -166,31 +241,14 @@ impl Stats {
|
||||
}
|
||||
|
||||
fn apply_aggregates(inner: &mut Inner, rec: &RequestRecord, retention_secs: u64) {
|
||||
// all-time counters for Prometheus; the windowed snapshot is derived
|
||||
// from the buckets below.
|
||||
inner.total.apply(rec);
|
||||
inner
|
||||
.by_model
|
||||
.entry(rec.model_id.clone())
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
inner
|
||||
.by_account
|
||||
.entry(if rec.account.is_empty() {
|
||||
"anonymous".to_string()
|
||||
} else {
|
||||
rec.account.clone()
|
||||
})
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
inner
|
||||
.by_tier
|
||||
.entry(rec.tier.clone())
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
inner
|
||||
.by_label
|
||||
.entry(rec.task_label.clone())
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
|
||||
let minute = rec.ts - (rec.ts % 60);
|
||||
match inner.buckets.back_mut() {
|
||||
@@ -226,6 +284,13 @@ impl Stats {
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||
Self::push_recent(&mut inner, &rec, self.ring_size);
|
||||
inner
|
||||
.by_account_alltime
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
inner.writes_since_check += 1;
|
||||
}
|
||||
if let Some(path) = &self.file {
|
||||
if let Ok(line) = serde_json::to_string(&rec) {
|
||||
@@ -238,6 +303,141 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
}
|
||||
self.maybe_compact(false);
|
||||
}
|
||||
|
||||
fn totals_path(&self) -> Option<String> {
|
||||
self.file.as_ref().map(|f| format!("{f}.totals.json"))
|
||||
}
|
||||
|
||||
fn load_folded(&self) {
|
||||
let Some(path) = self.totals_path() else {
|
||||
return;
|
||||
};
|
||||
let Ok(content) = std::fs::read_to_string(&path) else {
|
||||
return;
|
||||
};
|
||||
#[derive(serde::Deserialize)]
|
||||
struct Sidecar {
|
||||
#[serde(default)]
|
||||
by_account: HashMap<String, Totals>,
|
||||
#[serde(default)]
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
#[serde(default)]
|
||||
month_upto_ts: u64,
|
||||
}
|
||||
// new format {by_account, month_spend, month_upto_ts}; the original
|
||||
// file held the bare by_account map — accept both
|
||||
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
inner.folded = sc.by_account;
|
||||
inner.month_spend = sc.month_spend;
|
||||
inner.month_upto_ts = sc.month_upto_ts;
|
||||
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||
self.inner.lock().unwrap().folded = m;
|
||||
}
|
||||
}
|
||||
|
||||
fn write_sidecar(&self, inner: &Inner) {
|
||||
let Some(tpath) = self.totals_path() else {
|
||||
return;
|
||||
};
|
||||
let payload = serde_json::json!({
|
||||
"by_account": inner.folded,
|
||||
"month_spend": inner.month_spend,
|
||||
"month_upto_ts": now_secs(),
|
||||
});
|
||||
let tmp = format!("{tpath}.tmp");
|
||||
if std::fs::write(&tmp, payload.to_string()).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &tpath);
|
||||
}
|
||||
}
|
||||
|
||||
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
|
||||
/// older than the retention window fold into the cumulative per-account
|
||||
/// sidecar and are dropped from the file (tmp + rename on both files, so
|
||||
/// a crash mid-compaction loses nothing). `force` checks size regardless
|
||||
/// of the write counter (startup).
|
||||
fn maybe_compact(&self, force: bool) {
|
||||
if self.max_file_bytes == 0 {
|
||||
return;
|
||||
}
|
||||
let Some(path) = &self.file else { return };
|
||||
{
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
if !force {
|
||||
if inner.writes_since_check < 256 {
|
||||
return;
|
||||
}
|
||||
inner.writes_since_check = 0;
|
||||
// piggyback: persist month-to-date spend so budgets survive
|
||||
// restarts even between compactions
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
}
|
||||
}
|
||||
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
|
||||
if size <= self.max_file_bytes {
|
||||
return;
|
||||
}
|
||||
let Ok(content) = std::fs::read_to_string(path) else {
|
||||
return;
|
||||
};
|
||||
let cutoff = now_secs().saturating_sub(self.retention_secs);
|
||||
let mut kept = String::with_capacity(content.len() / 2);
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
for line in content.lines() {
|
||||
let line = line.trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
match serde_json::from_str::<RequestRecord>(line) {
|
||||
Ok(rec) if rec.ts < cutoff => {
|
||||
inner
|
||||
.folded
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
// fold = leave by_account_alltime (record leaves the file,
|
||||
// its share moves from "alltime" into "folded")
|
||||
if let Some(t) = inner.by_account_alltime.get_mut(&rec.account) {
|
||||
// subtract by rebuilding is costly; instead track the
|
||||
// invariant additively: remove the record's share
|
||||
t.requests = t.requests.saturating_sub(1);
|
||||
if rec.outcome == "ok" {
|
||||
t.ok = t.ok.saturating_sub(1);
|
||||
} else {
|
||||
t.errors = t.errors.saturating_sub(1);
|
||||
}
|
||||
if rec.failover {
|
||||
t.failovers = t.failovers.saturating_sub(1);
|
||||
}
|
||||
t.prompt_tokens = t.prompt_tokens.saturating_sub(rec.prompt_tokens as u64);
|
||||
t.completion_tokens = t
|
||||
.completion_tokens
|
||||
.saturating_sub(rec.completion_tokens as u64);
|
||||
t.cost_usd = (t.cost_usd - rec.cost_usd).max(0.0);
|
||||
t.latency_ms_sum = t.latency_ms_sum.saturating_sub(rec.latency_ms);
|
||||
}
|
||||
}
|
||||
Ok(_) => {
|
||||
kept.push_str(line);
|
||||
kept.push('\n');
|
||||
}
|
||||
Err(_) => { /* unparseable line — drop */ }
|
||||
}
|
||||
}
|
||||
// persist the folded totals FIRST, then swap the trimmed file in
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
let tmp = format!("{path}.tmp");
|
||||
if std::fs::write(&tmp, kept).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, path);
|
||||
}
|
||||
tracing::info!(
|
||||
new_size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0),
|
||||
"stats log compacted (older records folded into cumulative totals)"
|
||||
);
|
||||
}
|
||||
|
||||
/// Snapshot for the admin API: aggregates + time series for `range_secs` + breakdown by `groupby`.
|
||||
@@ -250,13 +450,31 @@ impl Stats {
|
||||
.filter(|b| b.minute >= cutoff)
|
||||
.collect();
|
||||
|
||||
let breakdown_src = match groupby {
|
||||
"account" => &inner.by_account,
|
||||
"tier" => &inner.by_tier,
|
||||
"label" => &inner.by_label,
|
||||
_ => &inner.by_model,
|
||||
};
|
||||
let mut breakdown: Vec<serde_json::Value> = breakdown_src
|
||||
// Windowed totals + breakdown = sum of the in-window buckets, so the KPIs
|
||||
// and the breakdown recompute whenever the range changes.
|
||||
let mut win = Totals::default();
|
||||
let mut group: HashMap<String, Totals> = HashMap::new();
|
||||
for b in &series {
|
||||
win.requests += b.requests;
|
||||
win.errors += b.errors;
|
||||
win.ok += b.requests.saturating_sub(b.errors);
|
||||
win.failovers += b.failovers;
|
||||
win.prompt_tokens += b.prompt_tokens;
|
||||
win.completion_tokens += b.completion_tokens;
|
||||
win.cost_usd += b.cost_usd;
|
||||
win.latency_ms_sum += b.latency_ms_sum;
|
||||
let src = match groupby {
|
||||
"account" => &b.by_account,
|
||||
"tier" => &b.by_tier,
|
||||
"label" => &b.by_label,
|
||||
_ => &b.by_model,
|
||||
};
|
||||
for (k, t) in src {
|
||||
group.entry(k.clone()).or_default().merge(t);
|
||||
}
|
||||
}
|
||||
|
||||
let mut breakdown: Vec<serde_json::Value> = group
|
||||
.iter()
|
||||
.map(|(k, t)| {
|
||||
serde_json::json!({
|
||||
@@ -278,19 +496,39 @@ impl Stats {
|
||||
.cmp(&a["requests"].as_u64().unwrap_or(0))
|
||||
});
|
||||
|
||||
let mut alltime: HashMap<String, Totals> = inner.folded.clone();
|
||||
for (k, t) in &inner.by_account_alltime {
|
||||
alltime.entry(k.clone()).or_default().merge(t);
|
||||
}
|
||||
let alltime_by_account: serde_json::Map<String, serde_json::Value> = alltime
|
||||
.iter()
|
||||
.map(|(k, t)| {
|
||||
(
|
||||
k.clone(),
|
||||
serde_json::json!({
|
||||
"requests": t.requests,
|
||||
"prompt_tokens": t.prompt_tokens,
|
||||
"completion_tokens": t.completion_tokens,
|
||||
"cost_usd": t.cost_usd,
|
||||
}),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
|
||||
serde_json::json!({
|
||||
"alltime_by_account": alltime_by_account,
|
||||
"totals": {
|
||||
"requests": inner.total.requests,
|
||||
"ok": inner.total.ok,
|
||||
"errors": inner.total.errors,
|
||||
"failovers": inner.total.failovers,
|
||||
"prompt_tokens": inner.total.prompt_tokens,
|
||||
"completion_tokens": inner.total.completion_tokens,
|
||||
"total_tokens": inner.total.prompt_tokens + inner.total.completion_tokens,
|
||||
"cost_usd": inner.total.cost_usd,
|
||||
"avg_latency_ms": avg(inner.total.latency_ms_sum, inner.total.requests),
|
||||
"success_rate": if inner.total.requests > 0 {
|
||||
inner.total.ok as f64 / inner.total.requests as f64
|
||||
"requests": win.requests,
|
||||
"ok": win.ok,
|
||||
"errors": win.errors,
|
||||
"failovers": win.failovers,
|
||||
"prompt_tokens": win.prompt_tokens,
|
||||
"completion_tokens": win.completion_tokens,
|
||||
"total_tokens": win.prompt_tokens + win.completion_tokens,
|
||||
"cost_usd": win.cost_usd,
|
||||
"avg_latency_ms": avg(win.latency_ms_sum, win.requests),
|
||||
"success_rate": if win.requests > 0 {
|
||||
win.ok as f64 / win.requests as f64
|
||||
} else { 0.0 },
|
||||
},
|
||||
"groupby": groupby,
|
||||
@@ -299,6 +537,21 @@ impl Stats {
|
||||
})
|
||||
}
|
||||
|
||||
/// Reset all in-memory stats and truncate the JSONL log — the "clear logs"
|
||||
/// admin action.
|
||||
pub fn clear(&self) {
|
||||
if let Some(tpath) = self.totals_path() {
|
||||
let _ = std::fs::remove_file(tpath);
|
||||
}
|
||||
{
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
*inner = Inner::default();
|
||||
}
|
||||
if let Some(path) = &self.file {
|
||||
let _ = std::fs::write(path, "");
|
||||
}
|
||||
}
|
||||
|
||||
/// Recent requests (newest first), with pagination.
|
||||
pub fn recent(&self, limit: usize, offset: usize) -> Vec<RequestRecord> {
|
||||
let inner = self.inner.lock().unwrap();
|
||||
@@ -358,10 +611,65 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
|
||||
if r.outcome != "ok" {
|
||||
b.errors += 1;
|
||||
}
|
||||
if r.failover {
|
||||
b.failovers += 1;
|
||||
}
|
||||
b.prompt_tokens += r.prompt_tokens as u64;
|
||||
b.completion_tokens += r.completion_tokens as u64;
|
||||
b.cost_usd += r.cost_usd;
|
||||
b.latency_ms_sum += r.latency_ms;
|
||||
// per-group breakdown, so the snapshot can be windowed by summing buckets
|
||||
b.by_model.entry(r.model_id.clone()).or_default().apply(r);
|
||||
b.by_account
|
||||
.entry(if r.account.is_empty() {
|
||||
"anonymous".to_string()
|
||||
} else {
|
||||
r.account.clone()
|
||||
})
|
||||
.or_default()
|
||||
.apply(r);
|
||||
b.by_tier.entry(r.tier.clone()).or_default().apply(r);
|
||||
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
|
||||
}
|
||||
|
||||
fn month_of(ts: u64) -> String {
|
||||
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
|
||||
let days = ts / 86_400;
|
||||
let mut year = 1970u64;
|
||||
let mut d = days;
|
||||
loop {
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let len = if leap { 366 } else { 365 };
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
year += 1;
|
||||
}
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let ml = [
|
||||
31,
|
||||
if leap { 29 } else { 28 },
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
];
|
||||
let mut month = 1;
|
||||
for len in ml {
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
month += 1;
|
||||
}
|
||||
format!("{year:04}-{month:02}")
|
||||
}
|
||||
|
||||
fn avg(sum: u64, n: u64) -> u64 {
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
//! Host metrics sampler for the admin dashboard: CPU load, RAM, free disk at
|
||||
//! the models dir, and the gateway's own RSS. One background task refreshes a
|
||||
//! snapshot every 2 s; the API endpoint only reads it.
|
||||
|
||||
use serde::Serialize;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[derive(Clone, Default, Serialize)]
|
||||
pub struct Snapshot {
|
||||
pub ts: u64,
|
||||
/// Whole-machine CPU usage 0..100 (average over all cores).
|
||||
pub cpu_percent: f32,
|
||||
pub cores: usize,
|
||||
pub load1: f64,
|
||||
pub ram_total: u64,
|
||||
pub ram_used: u64,
|
||||
/// Disk that holds models_dir.
|
||||
pub disk_total: u64,
|
||||
pub disk_free: u64,
|
||||
/// Resident memory of the gateway process itself.
|
||||
pub process_rss: u64,
|
||||
}
|
||||
|
||||
pub struct SysMon {
|
||||
snap: Mutex<Snapshot>,
|
||||
}
|
||||
|
||||
impl SysMon {
|
||||
pub fn snapshot(&self) -> Snapshot {
|
||||
self.snap.lock().unwrap().clone()
|
||||
}
|
||||
}
|
||||
|
||||
fn now() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Start the sampler. `models_dir` picks which disk's free space is reported.
|
||||
pub fn spawn(models_dir: String) -> Arc<SysMon> {
|
||||
let mon = Arc::new(SysMon {
|
||||
snap: Mutex::new(Snapshot::default()),
|
||||
});
|
||||
let ret = mon.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
use sysinfo::{Disks, ProcessesToUpdate, System};
|
||||
let mut sys = System::new();
|
||||
let pid = sysinfo::Pid::from_u32(std::process::id());
|
||||
let models_abs = std::fs::canonicalize(&models_dir)
|
||||
.unwrap_or_else(|_| std::path::PathBuf::from(&models_dir));
|
||||
loop {
|
||||
sys.refresh_cpu_usage();
|
||||
sys.refresh_memory();
|
||||
let _ = sys.refresh_processes(ProcessesToUpdate::Some(&[pid]), true);
|
||||
let disks = Disks::new_with_refreshed_list();
|
||||
// the disk whose mount point is the longest prefix of models_dir
|
||||
let mut best: Option<(usize, u64, u64)> = None;
|
||||
for d in disks.list() {
|
||||
let mp = d.mount_point();
|
||||
if models_abs.starts_with(mp) {
|
||||
let len = mp.as_os_str().len();
|
||||
if best.map(|(l, _, _)| len > l).unwrap_or(true) {
|
||||
best = Some((len, d.total_space(), d.available_space()));
|
||||
}
|
||||
}
|
||||
}
|
||||
let (disk_total, disk_free) = best.map(|(_, t, f)| (t, f)).unwrap_or((0, 0));
|
||||
let snap = Snapshot {
|
||||
ts: now(),
|
||||
cpu_percent: sys.global_cpu_usage(),
|
||||
cores: sys.cpus().len(),
|
||||
load1: System::load_average().one,
|
||||
ram_total: sys.total_memory(),
|
||||
ram_used: sys.used_memory(),
|
||||
disk_total,
|
||||
disk_free,
|
||||
process_rss: sys.process(pid).map(|p| p.memory()).unwrap_or(0),
|
||||
};
|
||||
*mon.snap.lock().unwrap() = snap;
|
||||
std::thread::sleep(std::time::Duration::from_secs(2));
|
||||
}
|
||||
});
|
||||
ret
|
||||
}
|
||||
Vendored
+404
@@ -177,6 +177,18 @@ textarea { resize: vertical; min-height: 64px; }
|
||||
.check input { width: auto; }
|
||||
.hint { color: var(--faint); font-size: 12px; margin-top: 4px; }
|
||||
|
||||
/* "?" help badge — hover/focus shows the native title tooltip explaining a setting */
|
||||
.help {
|
||||
display: inline-flex; align-items: center; justify-content: center;
|
||||
width: 15px; height: 15px; border-radius: 50%;
|
||||
border: 1px solid var(--border); background: var(--panel);
|
||||
color: var(--muted); font-size: 10px; font-weight: 700; line-height: 1;
|
||||
cursor: help; vertical-align: middle; user-select: none;
|
||||
transition: 0.12s;
|
||||
}
|
||||
.help:hover, .help:focus { color: var(--accent); border-color: var(--accent); outline: none; }
|
||||
label.field > span .help { text-transform: none; }
|
||||
|
||||
/* ---- buttons ------------------------------------------------------------- */
|
||||
.btn {
|
||||
display: inline-flex; align-items: center; justify-content: center; gap: 8px;
|
||||
@@ -307,3 +319,395 @@ tbody tr:hover td { background: var(--hover); }
|
||||
.callout { border: 1px solid var(--border); border-left: 3px solid var(--warn); background: color-mix(in srgb, var(--warn) 6%, transparent); border-radius: 10px; padding: 13px 15px; color: var(--muted); margin-bottom: 16px; }
|
||||
.callout.info { border-left-color: var(--accent-2); background: color-mix(in srgb, var(--accent-2) 6%, transparent); }
|
||||
.callout b { color: var(--text); }
|
||||
|
||||
/* ── Import model (HuggingFace → CMF) ── */
|
||||
.view-head { margin-bottom: 14px; }
|
||||
.view-head h2 { margin: 0 0 2px; }
|
||||
.card-title { font-weight: 600; margin-bottom: 10px; }
|
||||
.actions { margin-top: 12px; display: flex; gap: 8px; }
|
||||
.pad { padding: 18px; }
|
||||
.err { color: #e5534b; }
|
||||
.btn.sm { padding: 4px 10px; font-size: 12px; }
|
||||
|
||||
.hf-search {
|
||||
width: 100%; box-sizing: border-box; margin-bottom: 12px;
|
||||
padding: 10px 12px; font-size: 15px;
|
||||
border: 1px solid var(--border, #2a2f3a); border-radius: 8px;
|
||||
background: var(--bg-2, #12151c); color: inherit;
|
||||
}
|
||||
.hf-grid {
|
||||
display: grid; grid-template-columns: repeat(auto-fill, minmax(230px, 1fr));
|
||||
gap: 10px; max-height: 420px; overflow-y: auto;
|
||||
}
|
||||
.hf-card {
|
||||
border: 1px solid var(--border, #2a2f3a); border-radius: 8px;
|
||||
padding: 10px 12px; cursor: pointer; transition: border-color .12s, background .12s;
|
||||
background: var(--bg-2, #12151c);
|
||||
}
|
||||
.hf-card:hover { border-color: var(--accent, #5b8def); }
|
||||
.hf-card.sel { border-color: var(--accent, #5b8def); background: rgba(91,141,239,.10); }
|
||||
.hf-id { font-weight: 600; font-size: 13px; word-break: break-all; margin-bottom: 6px; }
|
||||
.hf-meta { display: flex; gap: 8px; align-items: center; flex-wrap: wrap; font-size: 12px; }
|
||||
.badge.bad { background: rgba(229,83,75,.15); color: #e5534b; }
|
||||
|
||||
.import-form { margin-top: 14px; }
|
||||
.adv { margin-top: 8px; }
|
||||
.adv-body { margin-top: 10px; padding-top: 10px; border-top: 1px dashed var(--border, #2a2f3a); }
|
||||
|
||||
.pbar {
|
||||
height: 8px; border-radius: 6px; overflow: hidden; margin: 10px 0;
|
||||
background: var(--bg-2, #12151c); border: 1px solid var(--border, #2a2f3a);
|
||||
}
|
||||
.pbar-fill { height: 100%; width: 100%; border-radius: 6px; }
|
||||
.pbar-fill.run {
|
||||
width: 40%;
|
||||
background: linear-gradient(90deg, transparent, var(--accent, #5b8def), transparent);
|
||||
animation: pbar-slide 1.2s infinite ease-in-out;
|
||||
}
|
||||
.pbar-fill.ok { background: #3fb950; }
|
||||
.pbar-fill.err { background: #e5534b; }
|
||||
@keyframes pbar-slide { 0% { margin-left: -40%; } 100% { margin-left: 100%; } }
|
||||
|
||||
.conv-log {
|
||||
margin-top: 10px; max-height: 260px; overflow-y: auto;
|
||||
background: var(--bg-2, #0d1017); border: 1px solid var(--border, #2a2f3a);
|
||||
border-radius: 8px; padding: 10px 12px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px;
|
||||
white-space: pre-wrap; word-break: break-word; color: var(--muted, #9aa4b2);
|
||||
}
|
||||
|
||||
/* Router-key field: input grows, inline "Save key" button sits beside it */
|
||||
.key-row { display: flex; gap: 8px; align-items: stretch; }
|
||||
.key-row input { flex: 1 1 auto; min-width: 0; }
|
||||
.key-row .btn { flex: 0 0 auto; white-space: nowrap; }
|
||||
|
||||
/* Determinate progress fill (known %); .run remains the indeterminate slider */
|
||||
.pbar-fill.det {
|
||||
background: linear-gradient(90deg, var(--accent, #5b8def), #7aa2f7);
|
||||
transition: width .4s ease;
|
||||
}
|
||||
/* Import job cards (multiple conversions, live) */
|
||||
.job { padding: 12px 0; border-top: 1px solid var(--border-soft, #232833); }
|
||||
.job:first-of-type { border-top: 0; }
|
||||
.job-head { display: flex; align-items: center; justify-content: space-between; gap: 12px; }
|
||||
.job-head b { word-break: break-all; }
|
||||
.small { font-size: 12px; }
|
||||
/* Router usage: readable, wrapping key/value rows (no horizontal overflow) */
|
||||
.usage-box { max-width: 100%; overflow-x: hidden; }
|
||||
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
|
||||
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
|
||||
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
|
||||
|
||||
/* ---- responsive shell + touch layout ----------------------------------- */
|
||||
.menu-toggle, .nav-close, .mobile-nav-backdrop { display: none; }
|
||||
.hamburger { width: 18px; display: flex; flex-direction: column; gap: 4px; }
|
||||
.hamburger i { display: block; height: 2px; border-radius: 2px; background: currentColor; }
|
||||
body.modal-open { overflow: hidden; }
|
||||
|
||||
@media (max-width: 980px) {
|
||||
.app { display: block; min-height: 100dvh; }
|
||||
.main { min-height: 100dvh; }
|
||||
.menu-toggle { display: inline-flex; flex: 0 0 auto; }
|
||||
.nav-close { display: inline-flex; margin-left: auto; flex: 0 0 auto; font-size: 24px; line-height: 1; }
|
||||
.brand-copy { min-width: 0; }
|
||||
|
||||
.sidebar {
|
||||
display: flex;
|
||||
position: fixed;
|
||||
inset: 0 auto 0 0;
|
||||
width: min(84vw, 310px);
|
||||
height: 100dvh;
|
||||
padding: max(18px, env(safe-area-inset-top)) 14px max(14px, env(safe-area-inset-bottom));
|
||||
z-index: 70;
|
||||
box-shadow: 18px 0 45px rgba(0, 0, 0, .42);
|
||||
transform: translateX(-104%);
|
||||
visibility: hidden;
|
||||
transition: transform .22s ease, visibility 0s linear .22s;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.app.nav-open .sidebar {
|
||||
transform: translateX(0);
|
||||
visibility: visible;
|
||||
transition: transform .22s ease;
|
||||
}
|
||||
.mobile-nav-backdrop {
|
||||
display: block;
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 65;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border: 0;
|
||||
background: rgba(3, 7, 18, .62);
|
||||
opacity: 0;
|
||||
visibility: hidden;
|
||||
pointer-events: none;
|
||||
transition: opacity .22s ease, visibility 0s linear .22s;
|
||||
}
|
||||
.app.nav-open .mobile-nav-backdrop {
|
||||
opacity: 1;
|
||||
visibility: visible;
|
||||
pointer-events: auto;
|
||||
transition: opacity .22s ease;
|
||||
}
|
||||
body.mobile-nav-open { overflow: hidden; }
|
||||
|
||||
.topbar { padding: 10px 18px; min-height: 58px; gap: 10px; }
|
||||
.topbar .title { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.topbar .right { gap: 8px; min-width: 0; }
|
||||
.content { max-width: none; padding: 22px 20px 52px; }
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
body { font-size: 14px; }
|
||||
.status-pills { display: none; }
|
||||
.topbar { padding: 9px 12px; }
|
||||
.content { padding: 18px 14px 42px; }
|
||||
|
||||
.page-head { margin-bottom: 16px; }
|
||||
.page-head > .flex { align-items: stretch; flex-direction: column; }
|
||||
.page-head > .flex > .btn { width: 100%; }
|
||||
.page-head p { line-height: 1.45; }
|
||||
|
||||
.card { padding: 15px; margin-bottom: 14px; border-radius: 12px; }
|
||||
.card .card-head { align-items: flex-start; flex-wrap: wrap; }
|
||||
.card .card-head .right { width: 100%; margin-left: 0; flex-wrap: wrap; }
|
||||
.grid { gap: 14px; }
|
||||
.kpis { gap: 10px; margin-bottom: 14px; }
|
||||
.kpi { padding: 12px; border-radius: 11px; }
|
||||
.kpi .big { font-size: 21px; }
|
||||
|
||||
.row { display: grid; grid-template-columns: minmax(0, 1fr); gap: 0; }
|
||||
.row > * { min-width: 0; width: 100%; }
|
||||
input, select, textarea { min-height: 44px; font-size: 16px; }
|
||||
input[type="checkbox"], input[type="radio"] { min-height: 0; }
|
||||
.check { min-height: 36px; margin-bottom: 8px; }
|
||||
.btn, .icon-btn { min-height: 44px; }
|
||||
.btn { padding: 10px 14px; }
|
||||
.btn.sm, .btn.small { min-height: 38px; padding: 8px 11px; }
|
||||
|
||||
.table-wrap {
|
||||
width: 100%;
|
||||
overflow-x: auto;
|
||||
overscroll-behavior-x: contain;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: thin;
|
||||
}
|
||||
.table-wrap table { min-width: max-content; }
|
||||
th, td { padding: 10px; }
|
||||
|
||||
.tierrow { align-items: flex-start; flex-direction: column; gap: 8px; }
|
||||
.tierrow .tname { width: auto; }
|
||||
.chips { width: 100%; }
|
||||
.togglerow { align-items: flex-start; }
|
||||
|
||||
.pg { gap: 14px; }
|
||||
.routecard .kv { flex-wrap: wrap; }
|
||||
.routecard .kv .v { max-width: 100%; overflow-wrap: anywhere; }
|
||||
.answer { padding: 12px; }
|
||||
.seg { max-width: 100%; overflow-x: auto; }
|
||||
.seg button { min-height: 40px; white-space: nowrap; }
|
||||
|
||||
.hf-grid { grid-template-columns: 1fr; max-height: 56dvh; }
|
||||
.job-head, .actions { align-items: stretch; flex-wrap: wrap; }
|
||||
.actions .btn { flex: 1 1 150px; }
|
||||
.key-row { flex-direction: column; }
|
||||
.key-row .btn { width: 100%; }
|
||||
.cmf-server-row { align-items: stretch !important; padding: 10px; border: 1px solid var(--border-soft); border-radius: 10px; }
|
||||
.cmf-server-row > input, .cmf-server-row > select { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
|
||||
.cmf-server-row > .btn { flex: 1 1 calc(50% - 6px); }
|
||||
.cmf-server-row > .small { flex: 1 1 100%; }
|
||||
|
||||
.modal-backdrop { align-items: flex-end; padding: 8px 0 0; }
|
||||
.modal {
|
||||
max-width: none;
|
||||
max-height: calc(100dvh - 8px);
|
||||
overflow-y: auto;
|
||||
padding: 18px 15px max(14px, env(safe-area-inset-bottom));
|
||||
border-radius: 18px 18px 0 0;
|
||||
}
|
||||
.modal .modal-foot {
|
||||
position: sticky;
|
||||
bottom: calc(-1 * max(14px, env(safe-area-inset-bottom)));
|
||||
margin: 12px -15px 0;
|
||||
padding: 12px 15px max(14px, env(safe-area-inset-bottom));
|
||||
background: color-mix(in srgb, var(--card) 94%, transparent);
|
||||
backdrop-filter: blur(10px);
|
||||
border-top: 1px solid var(--border-soft);
|
||||
}
|
||||
.modal .modal-foot .btn { flex: 1; }
|
||||
|
||||
.toasts { left: 12px; right: 12px; bottom: max(12px, env(safe-area-inset-bottom)); }
|
||||
.toast { min-width: 0; width: 100%; }
|
||||
.login-wrap { align-items: flex-start; padding: max(20px, env(safe-area-inset-top)) 14px; }
|
||||
.login-card { padding: 22px 18px; margin-top: 8dvh; }
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.topbar .title { font-size: 14px; }
|
||||
.topbar .right { gap: 6px; }
|
||||
.lang-select { width: 54px; min-height: 44px; padding: 6px; }
|
||||
.icon-btn { width: 42px; height: 42px; }
|
||||
.kpis { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.kpi .lbl { line-height: 1.25; }
|
||||
.barlist .b { grid-template-columns: minmax(0, 1fr) auto; gap: 6px 10px; }
|
||||
.barlist .b .track { grid-column: 1 / -1; grid-row: 2; }
|
||||
.barlist .b .val { min-width: 0; }
|
||||
.empty { padding: 28px 12px; }
|
||||
}
|
||||
|
||||
@media (max-width: 360px) {
|
||||
.topbar .title { display: none; }
|
||||
.kpis { grid-template-columns: 1fr; }
|
||||
}
|
||||
|
||||
/* ---- onboarding ------------------------------------------------------------ */
|
||||
.onb-shell { min-height: 100vh; display: flex; align-items: flex-start; justify-content: center; padding: 32px 20px; background: var(--bg-grad); }
|
||||
.onb-content { width: 100%; max-width: 860px; }
|
||||
.onb-wrap { max-width: 860px; margin: 0 auto; }
|
||||
.onb-header { display: flex; align-items: center; justify-content: space-between; margin-bottom: 18px; }
|
||||
.onb-dots { display: flex; gap: 8px; }
|
||||
.onb-dots .dot { width: 28px; height: 6px; border-radius: 999px; background: var(--border); }
|
||||
.onb-dots .dot.on { background: var(--accent); }
|
||||
.onb-dots .dot.done { background: color-mix(in srgb, var(--accent) 55%, transparent); }
|
||||
.onb-title { font-size: 12px; font-weight: 700; color: var(--muted); letter-spacing: 0.04em; text-transform: uppercase; }
|
||||
.onb-step h2 { font-size: 22px; margin-bottom: 6px; }
|
||||
.onb-langs { display: grid; grid-template-columns: repeat(auto-fill, minmax(132px, 1fr)); gap: 12px; margin: 18px 0; }
|
||||
.onb-lang {
|
||||
display: flex; flex-direction: column; align-items: center; gap: 10px;
|
||||
padding: 18px 10px; border-radius: 14px; border: 1.5px solid var(--border);
|
||||
background: var(--card); cursor: pointer; transition: .15s; font-family: inherit;
|
||||
}
|
||||
.onb-lang:hover { border-color: color-mix(in srgb, var(--accent) 45%, transparent); transform: translateY(-1px); }
|
||||
.onb-lang.sel { border-color: var(--accent); background: color-mix(in srgb, var(--accent) 10%, var(--card)); box-shadow: 0 6px 18px rgba(99,102,241,.18); }
|
||||
.onb-lang .flag { font-size: 32px; line-height: 1; }
|
||||
.onb-lang b { font-size: 14px; font-weight: 700; color: var(--text); }
|
||||
/* Featured gallery: the list grows as new models are published, so give it more
|
||||
room than the search grid before it starts scrolling. */
|
||||
.featured-section .hf-grid { max-height: min(62vh, 760px); }
|
||||
/* Money: 4 decimals, last two de-emphasized */
|
||||
.money-tail { font-size: .72em; opacity: .75; margin-left: 1px; }
|
||||
|
||||
.seg { flex-wrap: wrap; }
|
||||
|
||||
.ok { color: var(--good); }
|
||||
|
||||
.pairs { display: flex; flex-direction: column; gap: 6px; align-items: flex-start; }
|
||||
.pair-row { display: flex; gap: 6px; align-items: center; }
|
||||
.pair-model { min-width: 220px; max-width: 340px; padding: 4px 8px; font-size: 12.5px; }
|
||||
.pair-tier { max-width: 150px; padding: 4px 8px; font-size: 12.5px; }
|
||||
|
||||
/* Complexity bands slider */
|
||||
.bands-wrap { margin: 4px 0 16px; }
|
||||
/* one-line command chip (install commands etc.) — theme-aware */
|
||||
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
|
||||
|
||||
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
|
||||
.bands-track.disabled { opacity: .45; pointer-events: none; }
|
||||
.bands-zone { position: absolute; top: 0; bottom: 0; }
|
||||
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
|
||||
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
|
||||
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
|
||||
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
|
||||
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
|
||||
.bands-handle:active { cursor: grabbing; }
|
||||
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
|
||||
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
|
||||
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
|
||||
.bands-val.raised { top: -46px; }
|
||||
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
|
||||
|
||||
/* Benchmark view */
|
||||
@keyframes bm-in { from { opacity: 0; transform: translateY(-3px); } to { opacity: 1; transform: none; } }
|
||||
tbody tr:last-child { animation: bm-in .25s ease; }
|
||||
.bm-sum { margin-bottom: 10px; }
|
||||
.bm-cell { cursor: pointer; white-space: nowrap; }
|
||||
.bm-cell:hover { background: var(--hover); }
|
||||
.badge.ok { background: rgba(52,211,153,.15); color: var(--good); }
|
||||
|
||||
/* Routing: per-task rule rows */
|
||||
.taskrow { align-items: flex-start; }
|
||||
.taskname { min-width: 190px; line-height: 1.35; }
|
||||
.taskname .mono { font-size: 10.5px; opacity: .75; }
|
||||
.chips-empty { padding: 4px 0; font-style: italic; }
|
||||
|
||||
/* Shell notices (open admin warning, update available) */
|
||||
.notice {
|
||||
display: flex; align-items: center; gap: 8px;
|
||||
padding: 8px 18px; font-size: 13px;
|
||||
border-bottom: 1px solid var(--border-soft, #232833);
|
||||
}
|
||||
.notice.warn { background: color-mix(in srgb, #e5a54b 12%, var(--bg)); }
|
||||
.notice.info { background: color-mix(in srgb, var(--accent) 10%, var(--bg)); }
|
||||
.notice .icon-btn { margin-left: auto; }
|
||||
|
||||
/* Dashboard host metrics */
|
||||
.sysgrid { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 16px; }
|
||||
.sysitem .k { color: var(--muted); font-size: 12px; }
|
||||
.sysitem .v { font-weight: 600; }
|
||||
.sysbar {
|
||||
height: 8px; border-radius: 6px; overflow: hidden; margin: 6px 0;
|
||||
background: var(--bg-2, #12151c); border: 1px solid var(--border, #2a2f3a);
|
||||
}
|
||||
.sysbar i { display: block; height: 100%; background: var(--accent, #5b8def); }
|
||||
.sysbar i.bad { background: #e5534b; }
|
||||
|
||||
/* Log viewer */
|
||||
.logs-view { max-height: 68vh; }
|
||||
|
||||
/* Chat dialogue bubbles */
|
||||
.chat-log { max-height: 56vh; overflow-y: auto; display: flex; flex-direction: column; gap: 8px; }
|
||||
.chat-msg { display: flex; }
|
||||
.chat-msg.user { justify-content: flex-end; }
|
||||
.chat-bubble {
|
||||
max-width: 85%; padding: 8px 12px; border-radius: 12px;
|
||||
white-space: pre-wrap; word-break: break-word; font-size: 13.5px;
|
||||
background: var(--bg-2, #12151c); border: 1px solid var(--border-soft, #232833);
|
||||
}
|
||||
.chat-msg.user .chat-bubble {
|
||||
background: color-mix(in srgb, var(--accent) 16%, var(--bg-2));
|
||||
border-color: color-mix(in srgb, var(--accent) 35%, transparent);
|
||||
}
|
||||
|
||||
/* Generation gallery */
|
||||
.gal-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(110px, 1fr)); gap: 8px; }
|
||||
.gal-item {
|
||||
position: relative; cursor: pointer; border: 1px solid var(--border, #2a2f3a);
|
||||
border-radius: 8px; overflow: hidden; background: var(--bg-2, #12151c);
|
||||
}
|
||||
.gal-item:hover { border-color: var(--accent, #5b8def); }
|
||||
.gal-thumb { width: 100%; display: block; min-height: 64px; }
|
||||
.gal-video { display: flex; align-items: center; justify-content: center; font-size: 28px; height: 72px; }
|
||||
.gal-cap { font-size: 10px; color: var(--muted); padding: 3px 6px; }
|
||||
.gal-del {
|
||||
position: absolute; top: 2px; right: 2px; z-index: 2;
|
||||
border: 0; border-radius: 50%; width: 18px; height: 18px; line-height: 16px;
|
||||
background: rgba(0,0,0,.55); color: #fff; cursor: pointer; font-size: 12px;
|
||||
}
|
||||
|
||||
/* Bench + badges */
|
||||
.bench-row { gap: 10px; padding: 6px 0; border-top: 1px solid var(--border-soft, #232833); }
|
||||
.bench-row:first-of-type { border-top: 0; }
|
||||
.badge.warn { background: rgba(229,165,75,.15); color: #e5a54b; }
|
||||
.snippet { max-height: 260px; }
|
||||
|
||||
/* Playground media tabs: generated image / video frame */
|
||||
.media-out { display: flex; flex-direction: column; gap: 8px; }
|
||||
.media-canvas {
|
||||
max-width: 100%; height: auto; border-radius: 8px;
|
||||
border: 1px solid var(--border, #2a2f3a); background: #000;
|
||||
}
|
||||
/* Live download progress inside a featured card (empty until a job runs) */
|
||||
.card-prog:empty { display: none; }
|
||||
.card-prog { margin-top: 6px; }
|
||||
.card-prog .pbar { margin: 4px 0; }
|
||||
.card-prog .btn.sm { padding: 0 6px; line-height: 18px; }
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sidebar, .mobile-nav-backdrop { transition: none; }
|
||||
}
|
||||
|
||||
/* full-screen farewell after "stop everything & power off" */
|
||||
.shutdown-screen { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100vh; gap: 12px; text-align: center; padding: 24px; }
|
||||
.shutdown-screen .shutdown-icon { font-size: 64px; color: var(--bad); }
|
||||
.shutdown-screen .hint { max-width: 480px; }
|
||||
|
||||
Vendored
+1
-1
@@ -19,6 +19,6 @@
|
||||
<noscript>This console requires JavaScript.</noscript>
|
||||
<div id="app"><div class="boot">Loading…</div></div>
|
||||
<div id="toasts" class="toasts"></div>
|
||||
<script type="module" src="/admin/js/app.js"></script>
|
||||
<script type="module" src="/admin/js/app.js?v=3"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Vendored
+124
-5
@@ -19,7 +19,10 @@ export function setToken(tok) {
|
||||
}
|
||||
|
||||
async function j(method, path, body) {
|
||||
const opt = { method, headers: { Authorization: "Bearer " + getToken() } };
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const opt = { method, headers };
|
||||
if (body !== undefined) {
|
||||
opt.headers["content-type"] = "application/json";
|
||||
opt.body = JSON.stringify(body);
|
||||
@@ -50,6 +53,7 @@ export const api = {
|
||||
// meta / health
|
||||
meta: () => j("GET", "/meta"),
|
||||
health: () => j("GET", "/health"),
|
||||
probeRouter: () => j("POST", "/router/probe"),
|
||||
|
||||
// config
|
||||
getConfig: () => j("GET", "/config"),
|
||||
@@ -64,10 +68,13 @@ export const api = {
|
||||
|
||||
// routing
|
||||
getRouting: () => j("GET", "/routing"),
|
||||
routingLabels: () => j("GET", "/routing/labels"),
|
||||
putRouting: (b) => j("PUT", "/routing", b),
|
||||
|
||||
// protocols
|
||||
getProtocols: () => j("GET", "/protocols"),
|
||||
harnesses: () => j("GET", "/harnesses"),
|
||||
balance: () => j("GET", "/balance"),
|
||||
putProtocols: (b) => j("PUT", "/protocols", b),
|
||||
|
||||
// settings
|
||||
@@ -86,18 +93,91 @@ export const api = {
|
||||
|
||||
// stats
|
||||
stats: (range, groupby) => j("GET", `/stats?range=${range}&groupby=${groupby}`),
|
||||
clearStats: () => j("DELETE", "/stats"),
|
||||
requests: (limit = 50, offset = 0) => j("GET", `/requests?limit=${limit}&offset=${offset}`),
|
||||
|
||||
// playground
|
||||
test: (b) => j("POST", "/test", b),
|
||||
|
||||
// shadow / self-warming loop
|
||||
shadow: () => j("GET", "/shadow"),
|
||||
|
||||
// CMF runtime (the `cortiq` CLI that runs the .cmf format)
|
||||
cmfStatus: () => j("GET", "/cmf"),
|
||||
cmfInstall: () => j("POST", "/cmf/install"),
|
||||
cmfPortCheck: (port) => j("GET", "/cmf/port?port=" + encodeURIComponent(port)),
|
||||
cmfFiles: () => j("GET", "/cmf/files"),
|
||||
providerModels: (b) => j("POST", "/provider/models", b),
|
||||
|
||||
// CMF model factory (HuggingFace → local .cmf)
|
||||
hfSearch: (q, limit = 24) =>
|
||||
j("GET", `/hf/search?q=${encodeURIComponent(q || "")}&limit=${limit}`),
|
||||
startImport: (b) => j("POST", "/import", b),
|
||||
listImports: () => j("GET", "/import"),
|
||||
importStatus: (job) => j("GET", "/import/" + encodeURIComponent(job)),
|
||||
cancelImport: (job) =>
|
||||
j("POST", "/import/" + encodeURIComponent(job) + "/cancel"),
|
||||
deleteImport: (job) => j("DELETE", "/import/" + encodeURIComponent(job)),
|
||||
registerImport: (job) =>
|
||||
j("POST", "/import/" + encodeURIComponent(job) + "/register"),
|
||||
|
||||
// media generation (cortiq imagine / animate over local .cmf)
|
||||
mediaModels: () => j("GET", "/media/models"),
|
||||
mediaGenerate: (b) => j("POST", "/media", b),
|
||||
mediaJobs: () => j("GET", "/media"),
|
||||
mediaJob: (id) => j("GET", "/media/" + encodeURIComponent(id)),
|
||||
mediaCancel: (id) => j("POST", "/media/" + encodeURIComponent(id) + "/cancel"),
|
||||
mediaDelete: (id) => j("DELETE", "/media/" + encodeURIComponent(id)),
|
||||
|
||||
// system / logs / lifecycle
|
||||
system: () => j("GET", "/system"),
|
||||
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
|
||||
clearLogs: () => j("DELETE", "/logs"),
|
||||
restart: () => j("POST", "/restart"),
|
||||
shutdown: () => j("POST", "/shutdown"),
|
||||
benchStart: (model) => j("POST", "/bench", { model }),
|
||||
benchList: () => j("GET", "/bench"),
|
||||
|
||||
// first-run onboarding
|
||||
setupStatus: () => j("GET", "/setup/status"),
|
||||
// Connects every finished download as a local model unless register === false.
|
||||
setupComplete: (lang, register) => j("POST", "/setup/complete", { lang, register }),
|
||||
setupFeatured: () => j("GET", "/setup/featured"),
|
||||
// `name` is the exact output stem from setup/featured — it wins over the
|
||||
// repo/variant guess so the delete always hits the file the download wrote.
|
||||
deleteSetupFile: (repo, variant, name) =>
|
||||
j(
|
||||
"DELETE",
|
||||
`/setup/file?repo=${encodeURIComponent(repo)}` +
|
||||
(variant ? `&variant=${encodeURIComponent(variant)}` : "") +
|
||||
(name ? `&name=${encodeURIComponent(name)}` : "")
|
||||
),
|
||||
};
|
||||
|
||||
// Raw-body POST (no JSON encoding) — the config import takes TOML text as-is.
|
||||
export async function postRaw(path, text) {
|
||||
const tok = getToken();
|
||||
const headers = { "content-type": "application/toml" };
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(BASE + path, { method: "POST", headers, body: text });
|
||||
const body = await r.text();
|
||||
if (!r.ok) {
|
||||
let msg = body;
|
||||
try { msg = JSON.parse(body).error?.message || body; } catch { /* raw */ }
|
||||
throw new Error(msg || r.statusText);
|
||||
}
|
||||
return body ? JSON.parse(body) : null;
|
||||
}
|
||||
|
||||
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
|
||||
// returns the routing decision read from X-Cortiq-* response headers.
|
||||
export async function testStream(body, onDelta) {
|
||||
const tok = getToken();
|
||||
const headers = { "content-type": "application/json" };
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(BASE + "/test/stream", {
|
||||
method: "POST",
|
||||
headers: { Authorization: "Bearer " + getToken(), "content-type": "application/json" },
|
||||
headers,
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (r.status === 401) {
|
||||
@@ -110,6 +190,11 @@ export async function testStream(body, onDelta) {
|
||||
const reader = r.body.getReader();
|
||||
const dec = new TextDecoder();
|
||||
let buf = "";
|
||||
const t0 = performance.now();
|
||||
let ttftMs = null; // time to first content token
|
||||
let usage = null; // OpenAI-style final-chunk usage, when the provider sends it
|
||||
let usageEstimated = false; // gateway-injected estimate rather than tokenizer truth
|
||||
let costUsd = null; // final cost from the gateway's closing event (headers carry 0)
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
@@ -122,8 +207,17 @@ export async function testStream(body, onDelta) {
|
||||
const data = line.slice(5).trim();
|
||||
if (!data || data === "[DONE]") continue;
|
||||
try {
|
||||
const d = JSON.parse(data).choices?.[0]?.delta?.content;
|
||||
if (d) onDelta(d);
|
||||
const chunk = JSON.parse(data);
|
||||
if (chunk.usage) {
|
||||
usage = chunk.usage;
|
||||
usageEstimated = !!chunk.cortiq_estimated;
|
||||
}
|
||||
if (chunk.cortiq_cost_usd != null) costUsd = chunk.cortiq_cost_usd;
|
||||
const d = chunk.choices?.[0]?.delta?.content;
|
||||
if (d) {
|
||||
if (ttftMs == null) ttftMs = Math.round(performance.now() - t0);
|
||||
onDelta(d);
|
||||
}
|
||||
} catch {
|
||||
/* ignore malformed chunk */
|
||||
}
|
||||
@@ -136,6 +230,31 @@ export async function testStream(body, onDelta) {
|
||||
score: parseFloat(h.get("x-cortiq-complexity-score") || "0"),
|
||||
selected_model: h.get("x-cortiq-selected-model") || "",
|
||||
route_source: h.get("x-cortiq-route-source") || "",
|
||||
cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"),
|
||||
cost_usd: costUsd != null ? costUsd : parseFloat(h.get("x-cortiq-cost-usd") || "0"),
|
||||
usage,
|
||||
usage_estimated: usageEstimated,
|
||||
ttft_ms: ttftMs,
|
||||
};
|
||||
}
|
||||
|
||||
// Fetch a media job's output with the admin token and hand back an object URL
|
||||
// (plain <img src>/<video src> cannot carry the Authorization header).
|
||||
export async function mediaFileUrl(job, kind) {
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
|
||||
if (!r.ok) throw new Error((await r.text()) || r.statusText);
|
||||
return { url: URL.createObjectURL(await r.blob()), size: +(r.headers.get("content-length") || 0) };
|
||||
}
|
||||
|
||||
// Same fetch, raw bytes — for outputs the browser cannot display natively
|
||||
// (P6 PPM, MJPEG AVI) that we decode in JS.
|
||||
export async function mediaFileBytes(job, kind) {
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
|
||||
if (!r.ok) throw new Error((await r.text()) || r.statusText);
|
||||
return r.arrayBuffer();
|
||||
}
|
||||
|
||||
Vendored
+139
-14
@@ -4,25 +4,47 @@ import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
|
||||
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
|
||||
import { renderLogin } from "./views/login.js";
|
||||
import { renderDashboard } from "./views/dashboard.js";
|
||||
import { renderModels } from "./views/models.js";
|
||||
import { renderModels } from "./views/models.js?v=3";
|
||||
import { renderImport } from "./views/import.js";
|
||||
import { renderRouting } from "./views/routing.js";
|
||||
import { renderProtocols } from "./views/protocols.js";
|
||||
import { renderKeys } from "./views/keys.js";
|
||||
import { renderPlayground } from "./views/playground.js";
|
||||
import { renderSettings } from "./views/settings.js";
|
||||
import { renderLogs } from "./views/logs.js";
|
||||
import { renderBenchmark } from "./views/benchmark.js";
|
||||
import { renderOnboarding } from "./views/onboarding.js";
|
||||
|
||||
const ROUTES = [
|
||||
{ id: "dashboard", icon: "▤", view: renderDashboard },
|
||||
{ id: "models", icon: "▦", view: renderModels },
|
||||
{ id: "import", icon: "⬇", view: renderImport },
|
||||
{ id: "routing", icon: "⇄", view: renderRouting },
|
||||
{ id: "protocols", icon: "⇆", view: renderProtocols },
|
||||
{ id: "keys", icon: "🔑", view: renderKeys },
|
||||
{ id: "playground", icon: "✦", view: renderPlayground },
|
||||
{ id: "benchmark", icon: "⏱", view: renderBenchmark },
|
||||
{ id: "logs", icon: "📜", view: renderLogs },
|
||||
{ id: "settings", icon: "⚙", view: renderSettings },
|
||||
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
|
||||
];
|
||||
|
||||
export const appState = { meta: null, health: null, version: "" };
|
||||
|
||||
// allaigate site — where router keys are issued and topped up
|
||||
export const SITE_URL = "https://api.allaigate.com/";
|
||||
|
||||
let mobileNavOpen = false;
|
||||
|
||||
function setMobileNav(open) {
|
||||
mobileNavOpen = !!open;
|
||||
const app = document.querySelector(".app");
|
||||
if (app) app.classList.toggle("nav-open", mobileNavOpen);
|
||||
document.body.classList.toggle("mobile-nav-open", mobileNavOpen);
|
||||
const toggle = document.querySelector(".menu-toggle");
|
||||
if (toggle) toggle.setAttribute("aria-expanded", String(mobileNavOpen));
|
||||
}
|
||||
|
||||
function currentRoute() {
|
||||
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
|
||||
return ROUTES.find((r) => r.id === id) || ROUTES[0];
|
||||
@@ -34,25 +56,55 @@ const FAVICON =
|
||||
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="%233b82f6"/><stop offset=".5" stop-color="%238b5cf6"/><stop offset="1" stop-color="%2306b6d4"/></linearGradient></defs><rect width="32" height="32" rx="8" fill="url(%23g)"/><path d="M9 16h14M17 10l6 6-6 6" stroke="white" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>'
|
||||
);
|
||||
|
||||
// One-line banners under the topbar: open-admin warning, update available.
|
||||
function shellNotices() {
|
||||
const host = h("div", {});
|
||||
const m = appState.meta || {};
|
||||
if (m.admin_open_public && !sessionStorage.getItem("hide_open_admin")) {
|
||||
const close = h("button", { class: "icon-btn", title: "×", onClick: (e) => { sessionStorage.setItem("hide_open_admin", "1"); e.target.closest(".notice").remove(); } }, "×");
|
||||
host.appendChild(
|
||||
h("div", { class: "notice warn" }, "⚠ " + t("notice.openAdmin"), " ",
|
||||
h("a", { href: "#/settings" }, t("nav.settings") + " →"), close)
|
||||
);
|
||||
}
|
||||
if (m.update_available) {
|
||||
host.appendChild(
|
||||
h("div", { class: "notice info" },
|
||||
"⬆ " + t("notice.update", { v: m.update_available }),
|
||||
" ", h("code", { class: "mono" }, "cargo install cortiq-gateway"))
|
||||
);
|
||||
}
|
||||
return host;
|
||||
}
|
||||
|
||||
function sidebar(activeId) {
|
||||
const visibleRoutes = ROUTES.filter((r) => !r.hidden);
|
||||
return h(
|
||||
"aside",
|
||||
{ class: "sidebar", id: "sidebar" },
|
||||
{ class: "sidebar", id: "sidebar", "aria-label": "Main navigation" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "brand" },
|
||||
h("img", { src: FAVICON, alt: "" }),
|
||||
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
|
||||
h("div", { class: "brand-copy" }, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub"))),
|
||||
h("button", { class: "icon-btn nav-close", type: "button", title: "Close menu", "aria-label": "Close menu", onClick: () => setMobileNav(false) }, "×")
|
||||
),
|
||||
...ROUTES.map((r) =>
|
||||
...visibleRoutes.map((r) =>
|
||||
h(
|
||||
"a",
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id, onClick: () => setMobileNav(false) },
|
||||
h("span", { class: "ico" }, r.icon),
|
||||
t("nav." + r.id)
|
||||
)
|
||||
),
|
||||
h("div", { class: "nav-spacer" }),
|
||||
h(
|
||||
"a",
|
||||
{ class: "nav-item", href: SITE_URL, target: "_blank", rel: "noopener" },
|
||||
h("span", { class: "ico" }, "◈"),
|
||||
t("nav.billing"),
|
||||
h("span", { style: "margin-left:auto;opacity:.55" }, "↗")
|
||||
),
|
||||
h("div", { class: "nav-foot" }, t("nav.foot", { v: appState.version || "—" }))
|
||||
);
|
||||
}
|
||||
@@ -61,9 +113,19 @@ function statusPills() {
|
||||
const h_ = appState.health;
|
||||
const gwOk = !!appState.meta;
|
||||
const routerOk = h_?.router?.reachable;
|
||||
// reachable but no key resolves → warn: routing silently degrades to default
|
||||
const keyMissing = h_?.router?.key_source === "missing";
|
||||
const routerDot = h_ ? (routerOk ? (keyMissing ? "warn" : "ok") : "bad") : "warn";
|
||||
const routerText = h_
|
||||
? routerOk
|
||||
? keyMissing
|
||||
? t("status.noKey")
|
||||
: t("status.online")
|
||||
: t("status.offline")
|
||||
: t("status.checking");
|
||||
return h(
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
{ class: "flex status-pills" },
|
||||
h(
|
||||
"span",
|
||||
{ class: "pill" },
|
||||
@@ -73,8 +135,8 @@ function statusPills() {
|
||||
h(
|
||||
"span",
|
||||
{ class: "pill", title: h_?.router?.url || "" },
|
||||
h("span", { class: "dot " + (h_ ? (routerOk ? "ok" : "bad") : "warn") }),
|
||||
t("status.router") + ": " + (h_ ? (routerOk ? t("status.online") : t("status.offline")) : t("status.checking"))
|
||||
h("span", { class: "dot " + routerDot }),
|
||||
t("status.router") + ": " + routerText
|
||||
)
|
||||
);
|
||||
}
|
||||
@@ -87,7 +149,7 @@ function langSelect() {
|
||||
title: "Language",
|
||||
onChange: (e) => setLang(e.target.value),
|
||||
},
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, l.label))
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, `${l.flag} ${l.label}`))
|
||||
);
|
||||
return sel;
|
||||
}
|
||||
@@ -114,6 +176,19 @@ function topbar(activeId) {
|
||||
return h(
|
||||
"header",
|
||||
{ class: "topbar" },
|
||||
h(
|
||||
"button",
|
||||
{
|
||||
class: "icon-btn menu-toggle",
|
||||
type: "button",
|
||||
title: "Menu",
|
||||
"aria-label": "Open menu",
|
||||
"aria-controls": "sidebar",
|
||||
"aria-expanded": String(mobileNavOpen),
|
||||
onClick: () => setMobileNav(!mobileNavOpen),
|
||||
},
|
||||
h("span", { class: "hamburger", "aria-hidden": "true" }, h("i"), h("i"), h("i"))
|
||||
),
|
||||
h("div", { class: "title" }, t("nav." + activeId)),
|
||||
h(
|
||||
"div",
|
||||
@@ -153,13 +228,25 @@ async function renderView() {
|
||||
|
||||
function renderShell() {
|
||||
const route = currentRoute();
|
||||
mobileNavOpen = false;
|
||||
document.body.classList.remove("mobile-nav-open");
|
||||
// onboarding — full screen without sidebar/topbar
|
||||
if (route.id === "onboarding") {
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h("div", { class: "onb-shell" }, h("div", { class: "onb-content", id: "view" }))
|
||||
);
|
||||
renderView();
|
||||
return;
|
||||
}
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "app" },
|
||||
sidebar(route.id),
|
||||
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
|
||||
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
|
||||
h("main", { class: "main" }, topbar(route.id), shellNotices(), h("div", { class: "content", id: "view" }))
|
||||
)
|
||||
);
|
||||
renderView();
|
||||
@@ -169,12 +256,30 @@ function showLogin() {
|
||||
mount(document.getElementById("app"), renderLogin(boot));
|
||||
}
|
||||
|
||||
let staleNotified = false;
|
||||
async function checkUiFresh() {
|
||||
// the SPA lives in the tab until reload; if the gateway restarted (dev
|
||||
// rebuild, update, config restart) the embedded UI may have changed —
|
||||
// tell the user instead of letting them stare at stale views
|
||||
try {
|
||||
const m = await api.meta();
|
||||
if (appState.meta?.started && m.started && m.started !== appState.meta.started && !staleNotified) {
|
||||
staleNotified = true;
|
||||
const bar = h("div", { class: "callout warn", style: "position:fixed; left:50%; transform:translateX(-50%); bottom:18px; z-index:99; box-shadow:0 8px 24px rgba(0,0,0,.35)" },
|
||||
h("span", {}, "🔄 " + t("shell.uiStale")), " ",
|
||||
h("button", { class: "btn sm primary", onClick: () => location.reload() }, t("shell.reload")));
|
||||
document.body.append(bar);
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
|
||||
async function refreshHealth() {
|
||||
try {
|
||||
appState.health = await api.health();
|
||||
} catch {
|
||||
appState.health = null;
|
||||
}
|
||||
checkUiFresh();
|
||||
// re-render topbar only if shell present
|
||||
const route = currentRoute();
|
||||
const main = document.querySelector(".main");
|
||||
@@ -191,20 +296,34 @@ export async function boot() {
|
||||
setToken(urlTok);
|
||||
history.replaceState({}, "", location.pathname + location.hash);
|
||||
}
|
||||
if (!getToken()) {
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// try without token first — open mode (no CORTIQ_ADMIN_TOKEN) succeeds,
|
||||
// protected mode returns 401 and we show login
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch (e) {
|
||||
if (String(e.message) === "unauthorized") {
|
||||
// if we have no token stored, show login; with a stale token also show login
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// gateway reachable but error — still show shell
|
||||
// if meta failed for other reason but token was present, fall through
|
||||
if (!getToken()) {
|
||||
// open mode but gateway unreachable — still try shell; health will show
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch {}
|
||||
}
|
||||
}
|
||||
// first-run check — redirect to onboarding if needed (unless already there)
|
||||
try {
|
||||
const st = await api.setupStatus();
|
||||
if (st.needs_setup && location.hash !== "#/onboarding") {
|
||||
location.hash = "#/onboarding";
|
||||
}
|
||||
} catch {}
|
||||
if (!location.hash) location.hash = "#/dashboard";
|
||||
renderShell();
|
||||
refreshHealth();
|
||||
@@ -221,6 +340,12 @@ onLangChange(() => {
|
||||
window.addEventListener("hashchange", () => {
|
||||
if (document.querySelector(".app")) renderShell();
|
||||
});
|
||||
window.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Escape" && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
window.addEventListener("resize", () => {
|
||||
if (window.innerWidth > 980 && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
setInterval(() => {
|
||||
if (document.querySelector(".app")) refreshHealth();
|
||||
}, 20000);
|
||||
|
||||
Vendored
+12
-7
@@ -10,13 +10,13 @@ import tr from "./i18n/tr.js";
|
||||
|
||||
const DICTS = { en, ru, de, fr, es, zh, tr };
|
||||
export const LANGS = [
|
||||
{ code: "en", label: "English" },
|
||||
{ code: "ru", label: "Русский" },
|
||||
{ code: "de", label: "Deutsch" },
|
||||
{ code: "fr", label: "Français" },
|
||||
{ code: "es", label: "Español" },
|
||||
{ code: "zh", label: "中文" },
|
||||
{ code: "tr", label: "Türkçe" },
|
||||
{ code: "en", label: "English", flag: "🇬🇧" },
|
||||
{ code: "ru", label: "Русский", flag: "🇷🇺" },
|
||||
{ code: "de", label: "Deutsch", flag: "🇩🇪" },
|
||||
{ code: "fr", label: "Français", flag: "🇫🇷" },
|
||||
{ code: "es", label: "Español", flag: "🇪🇸" },
|
||||
{ code: "zh", label: "中文", flag: "🇨🇳" },
|
||||
{ code: "tr", label: "Türkçe", flag: "🇹🇷" },
|
||||
];
|
||||
|
||||
const listeners = new Set();
|
||||
@@ -48,6 +48,11 @@ export function onLangChange(fn) {
|
||||
return () => listeners.delete(fn);
|
||||
}
|
||||
|
||||
// Human name for a complexity tier; custom tier names pass through as-is.
|
||||
export function trTier(tier) {
|
||||
return ["low", "medium", "high"].includes(tier) ? t("tier." + tier) : tier;
|
||||
}
|
||||
|
||||
export function t(key, vars) {
|
||||
const d = DICTS[lang] || en;
|
||||
let s = d[key] ?? en[key] ?? key;
|
||||
|
||||
Vendored
+388
-3
@@ -30,9 +30,10 @@ export default {
|
||||
"nav.routing": "Routing",
|
||||
"nav.protocols": "Protokolle",
|
||||
"nav.keys": "API-Schlüssel",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Testbereich",
|
||||
"nav.settings": "Einstellungen",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Router-Key & Abrechnung",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "Router",
|
||||
@@ -40,6 +41,7 @@ export default {
|
||||
"status.online": "online",
|
||||
"status.offline": "offline",
|
||||
"status.checking": "Prüfe…",
|
||||
"status.noKey": "kein Key",
|
||||
"theme.toggle": "Design",
|
||||
"logout": "Abmelden",
|
||||
|
||||
@@ -74,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "nach Aufgabe",
|
||||
"dash.groupby.account": "nach Account",
|
||||
"dash.recent.title": "Letzte Anfragen",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Playground ausprobieren.",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Testbereich ausprobieren.",
|
||||
"dash.col.time": "Zeit",
|
||||
"dash.col.model": "Modell",
|
||||
"dash.col.tier": "Tier",
|
||||
@@ -88,6 +90,7 @@ export default {
|
||||
"dash.health.title": "Status",
|
||||
"dash.health.reachable": "erreichbar",
|
||||
"dash.health.unreachable": "nicht erreichbar",
|
||||
"dash.health.getKey": "Router-Key holen",
|
||||
|
||||
// models
|
||||
"models.title": "Modelle",
|
||||
@@ -186,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "alle",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Testbereich",
|
||||
"pg.subtitle": "Prompt durch die Live-Pipeline senden und Routing-Entscheidung prüfen.",
|
||||
"pg.system": "System-Prompt",
|
||||
"pg.systemPlaceholder": "optionale Systemnachricht",
|
||||
@@ -225,6 +228,18 @@ export default {
|
||||
"settings.router.timeout": "Timeout (ms)",
|
||||
"settings.router.verifyTls": "TLS-Zertifikat prüfen",
|
||||
"settings.router.taxonomy": "Taxonomie-ID",
|
||||
"settings.router.getKey": "Router-Key holen oder aufladen:",
|
||||
"settings.router.secret": "Router-Key (Wert)",
|
||||
"settings.router.test": "Router testen",
|
||||
"router.probe.ok": "Router OK ({ms} ms)",
|
||||
"router.probe.noKey": "Router-Key ist nicht gesetzt",
|
||||
"router.probe.auth": "Key abgelehnt — ungültig oder widerrufen",
|
||||
"router.probe.payment": "Zahlung erforderlich — Tarif abgelaufen oder Kontingent erschöpft",
|
||||
"router.probe.quota": "Ratenlimit überschritten",
|
||||
"router.probe.timeout": "Router-Zeitüberschreitung",
|
||||
"router.probe.unreachable": "Router nicht erreichbar",
|
||||
"router.probe.error": "Router-Fehler",
|
||||
"router.usage": "Verbrauch",
|
||||
"settings.route": "Routing-Eingabe",
|
||||
"settings.route.strategy": "Textstrategie",
|
||||
"settings.route.maxChars": "Max. Zeichen",
|
||||
@@ -265,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "Eintrags-TTL",
|
||||
"settings.cache.embed": "Embedding-Modell",
|
||||
"settings.cache.note": "Cache-Einstellungen werden nach einem Neustart wirksam.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Modell importieren",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Logs leeren",
|
||||
"dash.clearConfirm": "Alle erfassten Statistiken und Logs löschen? Das lässt sich nicht rückgängig machen.",
|
||||
"dash.cleared": "Logs geleert",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Gib eine ID oder einen Modellnamen ein",
|
||||
"models.form.apiKey": "API-Schlüssel",
|
||||
"models.form.apiKeyPlaceholder": "API-Schlüssel einfügen",
|
||||
"models.form.apiKeyNotNeeded": "nicht erforderlich (lokaler Server)",
|
||||
"models.form.apiKeyHint": "Wird sicher im Gateway gespeichert und nicht erneut angezeigt.",
|
||||
"models.form.keyStored": "Ein Schlüssel ist gespeichert — leer lassen, um ihn zu behalten, oder einen neuen einfügen.",
|
||||
"models.form.keyFromEnv": "Es wird ein Schlüssel aus einer Umgebungsvariable verwendet.",
|
||||
"models.form.paramsTitle": "Generierungs- und Denkparameter (O(1), MTP, Temperatur)",
|
||||
"models.form.temperature": "Temperatur",
|
||||
"models.form.temperatureHint": "Generierungstemperatur (0.0 = deterministisch, 1.0+ = kreativ).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling-Schwelle (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max. Tokens",
|
||||
"models.form.maxTokensHint": "Maximale Anzahl generierter Tokens.",
|
||||
"models.form.thinkBudget": "Denkmodus / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Token-Budget fürs Nachdenken (0 = aus, N > 0 = max. Reasoning-Tokens).",
|
||||
"models.form.systemPrompt": "Standard-System-Prompt",
|
||||
"models.form.systemPromptHint": "Wird automatisch vorangestellt, wenn die Anfrage keinen System-Prompt enthält.",
|
||||
"models.form.systemPromptPlaceholder": "Du bist ein hilfreicher Assistent…",
|
||||
"models.form.testKey": "Testen & Modelle laden",
|
||||
"models.form.testing": "wird geprüft…",
|
||||
"models.form.foundModels": "{n} Modelle geladen",
|
||||
"models.form.modelPlaceholder": "tippen zum Suchen…",
|
||||
"models.form.modelHint": "Klicke „Testen & Modelle laden“, um die Liste mit deinem Schlüssel abzurufen, und wähle dann eines aus.",
|
||||
"models.managed": "verwaltet",
|
||||
"models.managedHint": "Wird vom verwalteten CMF-Server bereitgestellt (Einstellungen → Lokale Modelle). Dort konfigurieren.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Router verwenden (aus = nur lokale Modelle)",
|
||||
"settings.cmf": "Lokale Modelle (CMF)",
|
||||
"settings.cmf.localOnly": "Nur lokale Modelle verwenden",
|
||||
"settings.cmf.manage": "Lokalen Modellserver starten",
|
||||
"settings.cmf.autoInstall": "cortiq automatisch installieren (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq automatisch aktualisieren",
|
||||
"settings.cmf.model": "Pfad zum lokalen .cmf-Modell",
|
||||
"settings.cmf.port": "Port des lokalen Servers",
|
||||
"settings.cmf.modelId": "Modell-ID im Pool",
|
||||
"settings.cmf.note": "Stellt ein lokales .cmf über cortiq bereit (github.com/infosave2007/cmf). Funktioniert auch, wenn der Router nicht erreichbar ist.",
|
||||
"settings.cmf.threads": "Worker-Threads",
|
||||
"settings.cmf.gpu": "GPU verwenden (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Threads parallelisieren die Decode-Matvecs (~3× schneller; 0 = Standard der Runtime). GPU-Offload hilft vor allem beim Prefill langer Prompts — das Decoding ist speicherbandbreitenbegrenzt und daher auf Apple Silicon ungefähr neutral.",
|
||||
"settings.cmf.pick": "Modell auswählen",
|
||||
"settings.cmf.models": "Lokale Modelle",
|
||||
"settings.cmf.modelsHelp": "Jede Zeile ist ein Modell, das das Gateway ausführt (eigenes cortiq serve auf eigenem Port) und im Pool registriert. Füge so viele hinzu, wie du .cmf-Dateien und freie Ports hast.",
|
||||
"settings.cmf.rowHint": "ID · .cmf-Datei · Port · Threads · GPU. Änderungen greifen nach einem Neustart; das Löschen einer Zeile stoppt diesen Server.",
|
||||
"settings.cmf.addServer": "+ Lokales Modell hinzufügen",
|
||||
"settings.cmf.checkPort": "Port prüfen",
|
||||
"settings.cmf.portFree": "Port ist frei",
|
||||
"settings.cmf.portBusy": "Port ist belegt",
|
||||
"settings.cmf.useSuggested": "{port} verwenden",
|
||||
"settings.cmf.serving": "{model} läuft auf :{port}",
|
||||
"settings.cmf.intro": "Modelle lokal mit der cortiq-Engine ausführen — funktioniert auch offline oder wenn der Router ausfällt.",
|
||||
"settings.cmf.runtime": "cortiq-Runtime",
|
||||
"settings.cmf.runtimeHelp": "Das Kommandozeilenwerkzeug `cortiq`, das .cmf-Modelle lädt und ausführt. Einmal installieren, bevor du einen lokalen Server startest.",
|
||||
"settings.cmf.install": "cortiq installieren",
|
||||
"settings.cmf.reinstall": "Neu installieren / aktualisieren",
|
||||
"settings.cmf.installing": "wird installiert… (kompiliert, einige Minuten)",
|
||||
"settings.cmf.installedOk": "cortiq installiert",
|
||||
"settings.cmf.notInstalled": "nicht installiert",
|
||||
"settings.cmf.addModel": "Modell hinzufügen",
|
||||
"settings.cmf.addModelHelp": "Modelle werden im Tab „Modell importieren“ heruntergeladen und konvertiert (HuggingFace → .cmf). Richte den lokalen Server danach unten auf die .cmf-Datei aus.",
|
||||
"settings.cmf.importNote": "Noch kein Modell? Importiere eines:",
|
||||
"settings.cmf.importLink": "Import öffnen",
|
||||
"settings.cmf.modelHelp": "Pfad zur .cmf-Datei, die bereitgestellt werden soll (z. B. models/qwen.cmf), erzeugt im Import-Tab. Mehrere Modelle für verschiedene Aufgaben? Importiere sie und wechsle diesen Pfad.",
|
||||
"settings.cmf.portHelp": "TCP-Port, auf dem der lokale Modellserver lauscht (Standard 8081). Nur ändern, wenn er mit einem anderen Dienst kollidiert.",
|
||||
"settings.cmf.threadsHelp": "Wie viele CPU-Kerne für die Generierung genutzt werden. Mehr = schneller (~3× gegenüber einem Kern); 0 überlässt die Wahl der Runtime.",
|
||||
"settings.cmf.gpuHelp": "Matrixberechnungen auf die GPU auslagern (Metal/wgpu). Auf Apple Silicon beschleunigt das vor allem lange Prompts — das Decoding ist speicherbandbreitenbegrenzt.",
|
||||
"settings.cmf.localOnlyHelp": "Jede Anfrage an das lokale Modell leiten und dabei Router sowie Cloud-Anbieter ignorieren.",
|
||||
"settings.cmf.manageHelp": "Das Gateway startet beim Hochfahren `cortiq serve` für die untenstehende .cmf-Datei und routet dorthin. Erfordert ein installiertes cortiq (Schritt 1). Greift nach einem Neustart.",
|
||||
"settings.cmf.autoInstallHelp": "Wenn cortiq beim Start fehlt, automatisch von crates.io installieren.",
|
||||
"settings.cmf.autoUpdateHelp": "Beim Start cortiq auf die neueste Version aktualisieren, falls eine neuere existiert.",
|
||||
"settings.router.enabledHelp": "Den cortiq-Router jede Anfrage klassifizieren lassen (Aufgabentyp + Komplexität) und ein Modell wählen. Aus = nur die lokale Routing-Tabelle verwenden.",
|
||||
"settings.router.verifyTlsHelp": "Das TLS-Zertifikat des Routers prüfen. Nur bei einem selbstsignierten oder IP-gebundenen Zertifikat abschalten.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus-Metriken unter GET /metrics bereitstellen.",
|
||||
"settings.cortiq.echoHelp": "Ein Feld `cortiq` (Aufgabe, Modell, Kosten) zu jedem Antwort-Body hinzufügen. Response-Header werden immer gesendet.",
|
||||
"settings.stats.enabledHelp": "Aktuelle Anfragen aufzeichnen, um Dashboard und Diagramme zu speisen.",
|
||||
"settings.cache.enabledHelp": "Semantischer Cache: eine gespeicherte Antwort zurückgeben, wenn ein neuer Prompt einem früheren sehr ähnlich ist, und den Modellaufruf überspringen. Erfordert ein Embedding-Modell im Pool.",
|
||||
"settings.log.bodiesHelp": "Vollständige Nachrichteninhalte von Anfrage und Antwort loggen. Aus Datenschutzgründen standardmäßig aus.",
|
||||
"settings.router.saveKey": "Schlüssel speichern",
|
||||
"settings.router.keySaved": "Router-Schlüssel gespeichert",
|
||||
"settings.router.keyEmpty": "Füge zuerst den Schlüssel ein",
|
||||
"settings.router.secretHelp": "Füge den Schlüssel aus deiner E-Mail von api.allaigate.com ein und klicke auf „Schlüssel speichern“. Das ist KEIN Wert aus „API-Schlüssel“ — die sind für Clients, die dein Gateway aufrufen.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — Schlüssel gespeichert (zum Ersetzen erneut einfügen)",
|
||||
"settings.route.strategyHelp": "Welcher Text zur Klassifizierung an den Router geht: last_user = nur die letzte Nutzernachricht; last_user_plus_system = letzte Nutzernachricht + System-Prompt; concat_all = die gesamte Unterhaltung.",
|
||||
"settings.route.maxCharsHelp": "Maximale Zeichenzahl dieses Textes, die an den Router gesendet wird (längerer Text wird gekürzt). Mehr = genauere Klassifizierung, aber langsamere und teurere Router-Aufrufe.",
|
||||
"settings.route.cacheTtlHelp": "Wie lange eine Routing-Entscheidung für eine identische Anfrage zwischengespeichert wird, bevor der Router erneut gefragt wird (z. B. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Standard-Routing-Profil, wenn eine Anfrage keines angibt: cost-saver (am günstigsten), balanced oder quality-first (bestes Modell). Beeinflusst, welches Modell der Router wählt.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Suche auf HuggingFace, konvertiere in ein lokales .cmf und stelle es bereit.",
|
||||
"import.searchPlaceholder": "HuggingFace-Modelle suchen (z. B. qwen3, llama)…",
|
||||
"import.searching": "Suche läuft…",
|
||||
"import.noResults": "Keine Modelle gefunden.",
|
||||
"import.configure": "Konvertierung einrichten",
|
||||
"import.outName": "Ausgabename",
|
||||
"import.outNameHint": "Name der .cmf-Datei (Buchstaben, Ziffern, - und _).",
|
||||
"import.quant": "Quantisierung",
|
||||
"import.convert": "Konvertieren & laden",
|
||||
"import.converting": "Konvertierung",
|
||||
"import.started": "Konvertierung gestartet",
|
||||
"import.failed": "Konvertierung fehlgeschlagen — siehe Log.",
|
||||
"import.register": "Als lokales Modell registrieren",
|
||||
"import.registered": "Als '{id}' registriert. Starte cortiq-server, um es bereitzustellen.",
|
||||
"import.jobs": "Downloads & Konvertierungen",
|
||||
"import.cancel": "Abbrechen",
|
||||
"import.cancelling": "Wird abgebrochen — unvollständige Ausgabe wird entfernt…",
|
||||
"import.delete": "Löschen",
|
||||
"import.deleteConfirm": "Dieses Modell und seine konvertierte .cmf-Datei von der Festplatte löschen?",
|
||||
"import.deleted": "Mitsamt Dateien gelöscht",
|
||||
"import.state.running": "läuft",
|
||||
"import.state.done": "fertig",
|
||||
"import.state.error": "Fehler",
|
||||
"import.state.cancelled": "abgebrochen",
|
||||
"import.featuredTitle": "Empfohlene fertige .cmf-Modelle",
|
||||
"import.featuredSub": "Vorkonvertierte .cmf-Modelle vom infosave-Konto auf HuggingFace — die Liste wird live gelesen, neue Modelle erscheinen von selbst.",
|
||||
"import.readyCmfBadge": "Fertiges .cmf",
|
||||
"import.directDownload": "Herunterladen",
|
||||
"import.q.q8_2f": "8-Bit Zwei-Feld (𝒲×θ) — empfohlen: bestes Verhältnis von Qualität und Größe.",
|
||||
"import.q.q8_row": "8-Bit pro Zeile — einfach und robust.",
|
||||
"import.q.q4": "4-Bit Block — am kleinsten, geringere Qualität.",
|
||||
"import.q.q4_tiled": "4-Bit gekachelt (Q4_TILED) — 1,66× schneller auf der Metal-GPU.",
|
||||
"import.q.vbit": "Variabel 3–8 Bit — Bit-Budget pro Experte.",
|
||||
"import.q.q1t": "1,58-Bit ternär (Q1T) — BitNet b1.58 mit spärlichen Ausreißern.",
|
||||
"import.q.q1s": "1-Bit spärlich (Q1S) — 1-Bit-Basis mit holografischen Ausreißern.",
|
||||
"import.q.q1p": "1-Bit PTQ (Q1P) — 1 Bit mit Fehlerdiffusion.",
|
||||
"import.q.q1": "1-Bit (Q1) — rein binäres Format.",
|
||||
"import.q.f16": "16-Bit — keine Quantisierung.",
|
||||
"import.q.f32": "32-Bit — volle Präzision (große Datei).",
|
||||
"import.adv.title": "Erweiterte Einstellungen",
|
||||
"import.adv.auto": "automatisch",
|
||||
"import.adv.linearCore": "Linearer Kern",
|
||||
"import.adv.linearHint": "Wie Layer mit linearer Attention übernommen bzw. gefaltet werden.",
|
||||
"import.adv.nphase": "vmf_phase Phasen/Kopf",
|
||||
"import.adv.vbitShape": "VBIT-Budgetform",
|
||||
"import.adv.meanBits": "VBIT mittlere Bits",
|
||||
"import.adv.shard": "Shard max. (GB)",
|
||||
"import.adv.noShard": "kein Sharding",
|
||||
"import.adv.skipMtp": "MTP-Kopf überspringen",
|
||||
"import.adv.o1": "O(1)-Attention",
|
||||
"import.adv.o1Hint": "Streaming-Attention mit konstantem Speicher: der KV-Cache wird nach dem Prefill freigegeben, lange Kontexte lassen den Speicher also nicht wachsen. Die Gewichte bleiben Byte-identisch — gespeichert wird nur ein Hinweis für die Ladezeit; du kannst jederzeit zurückwechseln (mit --o1 off starten). Auf Qwen3-0.6B validiert: alle Layer ≈ ×1,18 Perplexität.",
|
||||
"import.adv.o1Off": "aus — exakte Attention (Standard)",
|
||||
"import.adv.o1All": "alle — jeden Softmax-Layer konvertieren",
|
||||
"import.adv.o1Deep": "deep N — nur die N tiefsten Layer",
|
||||
"import.adv.o1Custom": "benutzerdefiniert — explizite Layer-Liste",
|
||||
"import.adv.o1DeepN": "N tiefe Layer",
|
||||
"import.adv.o1Layers": "Layer (kommagetrennt)",
|
||||
"import.adv.o1mHint": "Landmark-Budget (Standard 32)",
|
||||
"import.adv.o1windowHint": "Breite des exakten Fensters (Standard 128)",
|
||||
"import.adv.o1sinkHint": "permanente exakte Sink-Keys (Standard 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Ersteinrichtung",
|
||||
"onboarding.welcome": "Willkommen bei Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Wähle deine Sprache — du kannst sie jederzeit in der Kopfzeile ändern.",
|
||||
"onboarding.next": "Weiter",
|
||||
"onboarding.back": "Zurück",
|
||||
"onboarding.skip": "Vorerst überspringen",
|
||||
"onboarding.skipSetup": "Überspringen",
|
||||
"onboarding.finish": "Einrichtung abschließen",
|
||||
"onboarding.featuredTitle": "Fertige Modelle (ein Klick)",
|
||||
"onboarding.featuredSub": "Das sind bereits konvertierte .cmf-Modelle von infosave auf HuggingFace — einfach herunterladen, keine Konvertierung nötig. Wähle eines oder mehrere für den Sofortstart.",
|
||||
"onboarding.download": "Herunterladen",
|
||||
"onboarding.hfTitle": "Jedes HuggingFace-Modell — automatisch konvertiert",
|
||||
"onboarding.hfSub": "Suche ein beliebiges Modell, wähle eine Quantisierung, und wir konvertieren es automatisch nach .cmf. Keine manuellen Config-Änderungen.",
|
||||
"onboarding.noJobs": "Noch keine Downloads.",
|
||||
"onboarding.doneTitle": "Alles bereit",
|
||||
"onboarding.doneSub": "Heruntergeladene Modelle werden automatisch registriert. Weitere kannst du unter Import oder Modelle hinzufügen.",
|
||||
"onboarding.doneHint": "„Einrichtung abschließen“ verbindet jeden fertigen Download als lokales Modell, startet dessen .cmf-Server und richtet das Routing darauf aus. Alles lässt sich später unter Modelle ändern.",
|
||||
"onboarding.willRegister": "Werden als lokale Modelle verbunden",
|
||||
"onboarding.nothingToRegister": "Noch keine abgeschlossenen Downloads — nichts zu verbinden. Geh einen Schritt zurück und wähle ein Modell.",
|
||||
"onboarding.registeredN": "{n} Modell(e) verbunden: {ids}",
|
||||
"onboarding.runtimeMissing": "Die cortiq-Runtime ist nicht installiert — installiere sie unter Einstellungen → Lokale Modelle, um diese Modelle bereitzustellen.",
|
||||
"onboarding.downloaded": "heruntergeladen",
|
||||
"onboarding.update": "Aktualisieren",
|
||||
"onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen",
|
||||
"onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.",
|
||||
"onboarding.unregistered": "Ebenfalls abgemeldet: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Bild",
|
||||
"pg.tab.video": "Video",
|
||||
"pg.generate": "Generieren",
|
||||
"pg.generating": "Wird generiert…",
|
||||
"pg.genFailed": "Generierung fehlgeschlagen — siehe Log.",
|
||||
"pg.result": "Ergebnis",
|
||||
"pg.genMetrics": "Generierungsmetriken",
|
||||
"pg.imagePromptPlaceholder": "ein Rotfuchs im Schnee bei Sonnenuntergang, fotorealistisch, detailliertes Fell",
|
||||
"pg.videoPromptPlaceholder": "ein Corgi mit Kochmütze wirft einen Pfannkuchen, Brutzeln und ein fröhliches Bellen",
|
||||
"pg.imageEmpty": "Das generierte Bild erscheint hier.",
|
||||
"pg.videoEmpty": "Der generierte Clip (mit Ton) erscheint hier.",
|
||||
"pg.width": "Breite",
|
||||
"pg.height": "Höhe",
|
||||
"pg.steps": "Schritte",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Zufälliger Seed",
|
||||
"pg.cfgHint": "Guidance-Stärke; ≤ 1 schaltet CFG ab und halbiert die Arbeit.",
|
||||
"pg.frames": "Frames",
|
||||
"pg.framesHint": "Bei 24 fps; auf das Raster des Modells gerundet.",
|
||||
"pg.quality": "JPEG-Qualität",
|
||||
"pg.videoSizeHint": "Vielfache von 32; die trainierte kurze Kante ist 768.",
|
||||
"pg.videoStepsHint": "Die Turbo-LoRA ist auf 4 Schritte trainiert.",
|
||||
"pg.firstFrame": "Erstes Frame (optional)",
|
||||
"pg.lastFrame": "Letztes Frame (optional)",
|
||||
"pg.keyframeHint": "Macht den Lauf zu Keyframe-to-Video.",
|
||||
"pg.resolution": "Auflösung",
|
||||
"pg.genTime": "Generierungszeit",
|
||||
"pg.timePerStep": "Pro Schritt",
|
||||
"pg.timePerFrame": "Pro Frame",
|
||||
"pg.fileSize": "Dateigröße",
|
||||
"pg.duration": "Dauer",
|
||||
"pg.play": "Abspielen",
|
||||
"pg.noMediaModels": "Noch kein lokales {kind}-Modell. Lade eines (z. B. aus der infosave-Sammlung) unter Import herunter:",
|
||||
"pg.tokSpeed": "Geschwindigkeit",
|
||||
"pg.ttft": "Erstes Token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Logs",
|
||||
"logs.subtitle": "Jüngstes Gateway-Log — das, was das Terminal zeigen würde.",
|
||||
"logs.filterPlaceholder": "filtern…",
|
||||
"logs.pause": "Pause",
|
||||
"logs.resume": "Fortsetzen",
|
||||
"logs.empty": "Nichts gefunden.",
|
||||
"notice.openAdmin": "Die Konsole ist ohne Token offen und von anderen Rechnern erreichbar. Setze einen Token, wenn diese Maschine nicht privat ist.",
|
||||
"notice.update": "Update verfügbar: v{v}.",
|
||||
"dash.sys.title": "System",
|
||||
"dash.sys.cores": "Kerne",
|
||||
"dash.sys.disk": "Festplatte (Modelle)",
|
||||
"dash.sys.free": "frei",
|
||||
"dash.sys.gateway": "Gateway",
|
||||
"fit.disk": "zu wenig Speicherplatz",
|
||||
"fit.diskTitle": "Der Download ist größer als der freie Platz auf der Modell-Festplatte.",
|
||||
"fit.ram": "passt evtl. nicht in den RAM",
|
||||
"fit.ramTitle": "Das Modell ist nahe an oder größer als der RAM dieser Maschine — es läuft womöglich nicht.",
|
||||
"pg.eta": "übrig",
|
||||
"pg.min": "Min",
|
||||
"pg.queued": "In der Warteschlange — Position {n}. Startet, sobald die aktuelle Generierung fertig ist.",
|
||||
"pg.gallery": "Letzte Generierungen",
|
||||
"pg.galDeleteConfirm": "Diese Generierung samt Dateien löschen?",
|
||||
"pg.clearChat": "Leeren",
|
||||
"keys.snip.title": "Deine App verbinden",
|
||||
"keys.snip.hint": "Füge eines der Beispiele in dein Projekt ein — nur der Schlüssel-Platzhalter muss ersetzt werden.",
|
||||
"keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
|
||||
"keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
|
||||
"keys.snip.keyPh": "dein API-Schlüssel",
|
||||
"settings.restart": "Alles neu starten",
|
||||
"settings.restartConfirm": "Gateway und alle lokalen Modellserver neu starten? Laufende Anfragen werden abgebrochen.",
|
||||
"settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
|
||||
"settings.shutdown": "Alles stoppen & ausschalten",
|
||||
"settings.shutdownConfirm": "Alle lokalen Modellserver stoppen und das Gateway ausschalten? API und dieses Panel antworten erst wieder, wenn Sie das Gateway neu starten.",
|
||||
"settings.shutdownDone": "Das Gateway ist ausgeschaltet.",
|
||||
"settings.shutdownHint": "Alle lokalen Modellserver sind gestoppt, der Prozess wurde beendet. Starten Sie das Gateway auf dem Server neu, um das Panel zurückzubekommen.",
|
||||
"bench.title": "Geschwindigkeit lokaler Modelle",
|
||||
"bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
|
||||
"bench.run": "Messen",
|
||||
"bench.running": "misst…",
|
||||
"bench.failed": "fehlgeschlagen",
|
||||
"bench.none": "Noch keine lokalen .cmf-Chat-Modelle.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Smalltalk",
|
||||
"task.code": "Code",
|
||||
"task.creative-writing": "Kreatives Schreiben",
|
||||
"task.extraction": "Datenextraktion",
|
||||
"task.math": "Mathematik",
|
||||
"task.qa": "Fragen & Antworten",
|
||||
"task.summarization": "Zusammenfassung",
|
||||
"task.translation": "Übersetzung",
|
||||
"routing.tasks.title": "Aufgabenbasiertes Routing",
|
||||
"routing.tasks.hint": "Die Modelle einer Regel kommen für ihre Aufgabe zuerst; die Komplexitätskette bleibt als Fallback. Leer = nur nach Komplexität.",
|
||||
"routing.tasks.inherit": "nach Komplexität (keine Regel)",
|
||||
"routing.tasks.share": "{p}% der Anfragen (7 T)",
|
||||
"routing.tasks.fromRouter": "Labels live vom Router",
|
||||
"routing.tasks.builtin": "Router nicht erreichbar — eingebauter Labelsatz",
|
||||
"routing.tasks.addCustom": "Aufgabe hinzufügen",
|
||||
"routing.tasks.addPlaceholder": "eigenes Label…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Qualitäts- und Tempo-Proben durch die Live-Pipeline: Antworten per Schlüsselwort geprüft, Schleifen erkannt, tok/s und Zeit bis zum ersten Token je Modell.",
|
||||
"bm.models": "Zu testende Modelle",
|
||||
"bm.quickMode": "Schnelltest",
|
||||
"bm.quickHint": "Die schnelle Teilmenge mit kurzen Antworten — etwa eine Minute pro lokalem Modell.",
|
||||
"bm.suiteInfo": "Eingebaute Suite: {q} schnelle / {f} volle Tests, plus Ihre eigenen.",
|
||||
"bm.run": "Starten",
|
||||
"bm.pickModel": "Wählen Sie mindestens ein Modell.",
|
||||
"bm.testOf": "Test {i} von {n}",
|
||||
"bm.results": "Ergebnisse",
|
||||
"bm.quick": "schnell",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Klick auf eine Zelle zeigt die vollständige Antwort.",
|
||||
"bm.history": "Frühere Läufe (in diesem Browser gespeichert)",
|
||||
"bm.custom": "Eigene Tests",
|
||||
"bm.customHint": "Prompt + kommagetrennte Schlüsselwörter, die die Antwort enthalten muss (| für Synonyme). Im Browser gespeichert.",
|
||||
"bm.promptPh": "Prompt…",
|
||||
"bm.checkPh": "Schlüsselwörter, kommagetrennt",
|
||||
"bm.addTest": "Test hinzufügen",
|
||||
"bm.emptyAnswer": "leere Antwort",
|
||||
"bm.loopDetected": "Schleife erkannt",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Smartes Routing",
|
||||
"bm.mode.models": "Einzelne Modelle",
|
||||
"bm.mode.routingDesc": "Das Kernfeature des Produkts im Test: jede Probe läuft über cortiq-auto — Sie sehen, welche Aufgabe der Router erkannt hat, welches Modell er wählte, ob das den Erwartungen entspricht und was es gekostet hat.",
|
||||
"bm.mode.modelsDesc": "Dieselbe Suite, an gewählte Modelle gepinnt — Qualität, Tempo und Kosten im direkten Vergleich, am Router vorbei.",
|
||||
"bm.answersOk": "Antworten korrekt",
|
||||
"bm.classAcc": "Aufgaben-Klassifikation",
|
||||
"bm.modelDist": "Anfragen gingen an",
|
||||
"bm.expectedLbl": "erwartet:",
|
||||
"bm.answer": "Antwort",
|
||||
"bm.liveRunning": "läuft…",
|
||||
"bm.spent": "ausgegeben",
|
||||
"bm.verdict": "🏁 Fertig: Antworten {p}, ausgegeben {s}",
|
||||
"bm.clearHistory": "Alle löschen",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — ins Terminal einfügen (das Gateway spricht das Anthropic-Protokoll nativ):",
|
||||
"keys.h.codex": "Codex — hängt den Provider an ~/.codex/config.toml an und startet:",
|
||||
"keys.h.opencode": "OpenCode — schreibt ~/.config/opencode/opencode.json (überschreibt!) und startet:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — hängt das Modell an ~/.continue/config.yaml an:",
|
||||
"keys.h.cline1": "Cline (VS Code) — in den Erweiterungseinstellungen wählen:",
|
||||
"keys.h.cline2": "Einstellungen → API Provider → OpenAI Compatible, dann die Werte oben einfügen.",
|
||||
"keys.h.note": "cortiq-auto = smartes Routing: der Router wählt das Modell pro Anfrage. Für ein festes Modell den exakten Namen angeben. Unter Windows die Skripte in Git Bash / WSL ausführen oder dieselben Variablen über die Systemumgebung setzen.",
|
||||
"keys.h.protoOff": "Das Anthropic-Protokoll (/v1/messages) ist deaktiviert — Claude Code kann sich nicht verbinden.",
|
||||
"keys.h.enableProto": "Aktivieren",
|
||||
"keys.h.st.installed": "Auf der Gateway-Maschine installiert",
|
||||
"keys.h.st.configured": "Bereits für dieses Gateway konfiguriert",
|
||||
"keys.h.st.notFound": "Auf der Gateway-Maschine nicht gefunden",
|
||||
"bm.degraded": "Router degradiert",
|
||||
"bm.routerDegraded": "Der Router hat nicht rechtzeitig geantwortet — die Anfrage nutzte die Fallback-Kette; die Klassifikation wird nicht gewertet.",
|
||||
"bm.cutByCap": "Budget ging fürs Denken drauf — keine Antwort mehr",
|
||||
"shell.uiStale": "Das Gateway wurde aktualisiert — diese Seite zeigt eine ältere Oberfläche.",
|
||||
"shell.reload": "Neu laden",
|
||||
"keys.col.requests": "Anfragen",
|
||||
"keys.col.tokens": "Tokens ein/aus",
|
||||
"keys.col.costAll": "$ gesamt",
|
||||
"routing.tasks.tier": "Komplexität",
|
||||
"routing.tasks.tierAuto": "Auto (Router)",
|
||||
"routing.tasks.tierHint": "Welches Modell diese Aufgabe bei welcher Komplexität bedient; \"jede\" führt auf allen Stufen. Reihenfolge = Fallback-Reihenfolge. High eskaliert weiterhin über \"jede\"-Einträge.",
|
||||
"routing.tasks.tierAny": "jede Komplexität",
|
||||
"routing.tasks.addModel": "Modell",
|
||||
"routing.bands.enable": "Eigene Komplexitätsgrenzen",
|
||||
"routing.bands.hint": "Ziehen Sie die Griffe: wo low endet und high beginnt — die Griffe schieben einander, die Zahlen aktualisieren sich live. Der kontinuierliche Router-Score wird mit IHREN Grenzen neu eingeteilt; ohne Häkchen gelten die Grenzen des Routers.",
|
||||
"tier.low": "leicht",
|
||||
"tier.medium": "mittel",
|
||||
"tier.high": "schwer",
|
||||
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
|
||||
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
|
||||
"settings.cache.entries": "Einträge",
|
||||
"settings.cache.hits": "Treffer",
|
||||
"settings.cache.saved": "gespart",
|
||||
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
|
||||
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
|
||||
"keys.budgetOf": "Budget {b}/Mon.",
|
||||
"dash.kpi.orBalance": "OpenRouter-Guthaben",
|
||||
"keys.h.install": "Installieren:",
|
||||
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
|
||||
"keys.snip.apply": "Schlüssel einsetzen",
|
||||
};
|
||||
|
||||
Vendored
+377
@@ -28,12 +28,14 @@ export default {
|
||||
// nav
|
||||
"nav.dashboard": "Dashboard",
|
||||
"nav.models": "Models",
|
||||
"nav.import": "Import model",
|
||||
"nav.routing": "Routing",
|
||||
"nav.protocols": "Protocols",
|
||||
"nav.keys": "API keys",
|
||||
"nav.playground": "Playground",
|
||||
"nav.settings": "Settings",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Router key & billing",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "Router",
|
||||
@@ -41,6 +43,7 @@ export default {
|
||||
"status.online": "online",
|
||||
"status.offline": "offline",
|
||||
"status.checking": "checking…",
|
||||
"status.noKey": "no key",
|
||||
"theme.toggle": "Theme",
|
||||
"logout": "Log out",
|
||||
|
||||
@@ -56,6 +59,9 @@ export default {
|
||||
// dashboard
|
||||
"dash.title": "Dashboard",
|
||||
"dash.subtitle": "Live traffic, cost and routing analytics for your model pool.",
|
||||
"dash.clear": "Clear logs",
|
||||
"dash.clearConfirm": "Clear all recorded stats and logs? This cannot be undone.",
|
||||
"dash.cleared": "Logs cleared",
|
||||
"dash.range": "Range",
|
||||
"dash.range.1h": "1h",
|
||||
"dash.range.24h": "24h",
|
||||
@@ -89,6 +95,7 @@ export default {
|
||||
"dash.health.title": "Health",
|
||||
"dash.health.reachable": "reachable",
|
||||
"dash.health.unreachable": "unreachable",
|
||||
"dash.health.getKey": "Get a router key",
|
||||
|
||||
// models
|
||||
"models.title": "Models",
|
||||
@@ -125,6 +132,32 @@ export default {
|
||||
"models.form.secretMissing": "not set",
|
||||
"models.form.secretPlaceholder": "set or replace key (write-only)",
|
||||
"models.deleteConfirm": "Delete model \"{id}\"?",
|
||||
"models.form.idOrModel": "Enter an id or a model name",
|
||||
"models.form.apiKey": "API Key",
|
||||
"models.form.apiKeyPlaceholder": "paste your API key",
|
||||
"models.form.apiKeyNotNeeded": "not required (local server)",
|
||||
"models.form.apiKeyHint": "Stored securely on the gateway; it is not shown again.",
|
||||
"models.form.keyStored": "A key is saved — leave blank to keep it, or paste a new one.",
|
||||
"models.form.keyFromEnv": "Using a key from an environment variable.",
|
||||
"models.form.paramsTitle": "Generation & Thinking Parameters (O(1), MTP, Temperature)",
|
||||
"models.form.temperature": "Temperature",
|
||||
"models.form.temperatureHint": "Generation temperature (0.0 = deterministic, 1.0+ = creative).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling threshold (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max Tokens",
|
||||
"models.form.maxTokensHint": "Maximum completion tokens to generate.",
|
||||
"models.form.thinkBudget": "Thinking Mode / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Reasoning token budget (0 = disabled, N > 0 = max reasoning tokens).",
|
||||
"models.form.systemPrompt": "Default System Prompt",
|
||||
"models.form.systemPromptHint": "Prepend automatically if request lacks a system prompt.",
|
||||
"models.form.systemPromptPlaceholder": "You are a helpful assistant...",
|
||||
"models.form.testKey": "Test & load models",
|
||||
"models.form.testing": "checking…",
|
||||
"models.form.foundModels": "{n} models loaded",
|
||||
"models.form.modelPlaceholder": "type to search…",
|
||||
"models.form.modelHint": "Click “Test & load models” to pull the list from your key, then pick one.",
|
||||
"models.managed": "managed",
|
||||
"models.managedHint": "Provided by the managed CMF server (Settings → Local models). Configure it there.",
|
||||
|
||||
// routing
|
||||
"routing.title": "Routing",
|
||||
@@ -225,12 +258,83 @@ export default {
|
||||
"settings.router.keyEnv": "Router key env name",
|
||||
"settings.router.timeout": "Timeout (ms)",
|
||||
"settings.router.verifyTls": "Verify TLS certificate",
|
||||
"settings.router.enabled": "Use the router (off = local models only)",
|
||||
"settings.cmf": "Local models (CMF)",
|
||||
"settings.cmf.localOnly": "Use only local models",
|
||||
"settings.cmf.manage": "Run a local model server",
|
||||
"settings.cmf.autoInstall": "Auto-install cortiq (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Auto-update cortiq",
|
||||
"settings.cmf.model": "Local .cmf model path",
|
||||
"settings.cmf.port": "Local server port",
|
||||
"settings.cmf.modelId": "Model id in the pool",
|
||||
"settings.cmf.note": "Serves a local .cmf via cortiq (github.com/infosave2007/cmf). Works even when the router is unavailable.",
|
||||
"settings.cmf.threads": "Worker threads",
|
||||
"settings.cmf.gpu": "Use GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Threads parallelise the decode matvecs (~3x faster; 0 = runtime default). GPU offload mainly helps prefill of long prompts — decode is memory-bandwidth-bound, so on Apple Silicon it is roughly neutral.",
|
||||
"settings.cmf.pick": "select a model",
|
||||
"settings.cmf.models": "Local models",
|
||||
"settings.cmf.modelsHelp": "Each row is one model the gateway runs (its own cortiq serve on its own port) and registers in the pool. Add as many as you have .cmf files and free ports.",
|
||||
"settings.cmf.rowHint": "id · .cmf file · port · threads · GPU. Changes apply on restart; deleting a row stops that server.",
|
||||
"settings.cmf.addServer": "+ Add local model",
|
||||
"settings.cmf.checkPort": "Check port",
|
||||
"settings.cmf.portFree": "port is free",
|
||||
"settings.cmf.portBusy": "port is in use",
|
||||
"settings.cmf.useSuggested": "use {port}",
|
||||
"settings.cmf.serving": "serving {model} on :{port}",
|
||||
"settings.cmf.intro": "Run models locally with the cortiq engine — works even offline or when the router is down.",
|
||||
"settings.cmf.runtime": "cortiq runtime",
|
||||
"settings.cmf.runtimeHelp": "The `cortiq` command-line tool that loads and runs .cmf models. Install it once before starting a local server.",
|
||||
"settings.cmf.install": "Install cortiq",
|
||||
"settings.cmf.reinstall": "Reinstall / update",
|
||||
"settings.cmf.installing": "installing… (compiles, a few minutes)",
|
||||
"settings.cmf.installedOk": "cortiq installed",
|
||||
"settings.cmf.notInstalled": "not installed",
|
||||
"settings.cmf.addModel": "Add a model",
|
||||
"settings.cmf.addModelHelp": "Models are downloaded and converted in the Import tab (HuggingFace → .cmf). Then point the local server at the .cmf file below.",
|
||||
"settings.cmf.importNote": "No model yet? Import one:",
|
||||
"settings.cmf.importLink": "Open Import",
|
||||
"settings.cmf.modelHelp": "Path to the .cmf file to serve (e.g. models/qwen.cmf), produced by the Import tab. Running several models for different tasks? Import them and switch this path.",
|
||||
"settings.cmf.portHelp": "TCP port the local model server listens on (default 8081). Change only if it clashes with another service.",
|
||||
"settings.cmf.threadsHelp": "How many CPU cores to use for generation. More = faster (~3× vs 1 core); 0 lets the runtime decide.",
|
||||
"settings.cmf.gpuHelp": "Offload matrix math to the GPU (Metal/wgpu). On Apple Silicon this mostly speeds up long prompts — decode is memory-bandwidth-bound.",
|
||||
"settings.cmf.localOnlyHelp": "Route every request to the local model, ignoring the router and any cloud providers.",
|
||||
"settings.cmf.manageHelp": "The gateway starts `cortiq serve` for the .cmf below at launch and routes to it. Needs cortiq installed (step 1). Applies on restart.",
|
||||
"settings.cmf.autoInstallHelp": "If cortiq is missing at startup, install it automatically from crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "On startup, upgrade cortiq to the newest release when a newer one exists.",
|
||||
"settings.router.enabledHelp": "Use the cortiq-router to classify each request (task type + complexity) and pick a model. Off = use the local routing table only.",
|
||||
"settings.router.verifyTlsHelp": "Verify the router's TLS certificate. Turn off only for a self-signed / IP-bound certificate.",
|
||||
"settings.telemetry.metricsHelp": "Expose Prometheus metrics at GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Add a `cortiq` field (task, model, cost) to each response body. Response headers are always sent.",
|
||||
"settings.stats.enabledHelp": "Record recent requests to power the dashboard and charts.",
|
||||
"settings.cache.enabledHelp": "Semantic cache: return a saved answer when a new prompt is very similar to a past one, skipping the model call. Needs an embedding model in the pool.",
|
||||
"settings.log.bodiesHelp": "Log full request/response message bodies. Off by default for privacy.",
|
||||
"settings.router.taxonomy": "Taxonomy id",
|
||||
"settings.router.getKey": "Get or top up a router key at",
|
||||
"settings.router.secret": "Router key (value)",
|
||||
"settings.router.saveKey": "Save key",
|
||||
"settings.router.keySaved": "Router key saved",
|
||||
"settings.router.keyEmpty": "Paste the key first",
|
||||
"settings.router.secretHelp": "Paste the key from your api.allaigate.com email, then Save key. This is NOT a “Keys” value — those are for clients calling your gateway.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — key saved (paste again to replace)",
|
||||
"settings.router.test": "Test router",
|
||||
"router.probe.ok": "Router OK ({ms} ms)",
|
||||
"router.probe.noKey": "Router key is not set",
|
||||
"router.probe.auth": "Key rejected — invalid or revoked",
|
||||
"router.probe.payment": "Payment required — plan expired or quota exhausted",
|
||||
"router.probe.quota": "Rate limit exceeded",
|
||||
"router.probe.timeout": "Router timed out",
|
||||
"router.probe.unreachable": "Router unreachable",
|
||||
"router.probe.error": "Router error",
|
||||
"router.usage": "Usage",
|
||||
"settings.route": "Routing input",
|
||||
"settings.route.strategy": "Text strategy",
|
||||
"settings.route.maxChars": "Max chars",
|
||||
"settings.route.cacheTtl": "Cache TTL",
|
||||
"settings.route.profile": "Default profile",
|
||||
"settings.route.strategyHelp": "What text is sent to the router for classification: last_user = only the last user message; last_user_plus_system = last user message + system prompt; concat_all = the whole conversation.",
|
||||
"settings.route.maxCharsHelp": "Max characters of that text sent to the router (longer is truncated). More = more accurate classification, but slower/pricier router calls.",
|
||||
"settings.route.cacheTtlHelp": "How long a routing decision is cached for an identical request before the router is asked again (e.g. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Default routing profile when a request doesn't specify one: cost-saver (cheapest), balanced, or quality-first (best model). Biases which model the router picks.",
|
||||
"settings.breaker": "Circuit breaker",
|
||||
"settings.breaker.threshold": "Error threshold",
|
||||
"settings.breaker.cooldown": "Cooldown",
|
||||
@@ -266,4 +370,277 @@ export default {
|
||||
"settings.cache.ttl": "Entry TTL",
|
||||
"settings.cache.embed": "Embedding model",
|
||||
"settings.cache.note": "Cache settings take effect after a restart.",
|
||||
|
||||
// import (HuggingFace → CMF)
|
||||
"import.subtitle": "Search HuggingFace, convert to a local .cmf, and serve it.",
|
||||
"import.searchPlaceholder": "Search HuggingFace models (e.g. qwen3, llama)…",
|
||||
"import.searching": "Searching…",
|
||||
"import.noResults": "No models found.",
|
||||
"import.configure": "Configure conversion",
|
||||
"import.outName": "Output name",
|
||||
"import.outNameHint": "The .cmf file name (letters, digits, - and _).",
|
||||
"import.quant": "Quantization",
|
||||
"import.convert": "Convert & load",
|
||||
"import.converting": "Converting",
|
||||
"import.started": "Conversion started",
|
||||
"import.failed": "Conversion failed — see the log.",
|
||||
"import.register": "Register as local model",
|
||||
"import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
|
||||
"import.jobs": "Downloads & Conversions",
|
||||
"import.cancel": "Cancel",
|
||||
"import.cancelling": "Cancelling — cleaning up partial output…",
|
||||
"import.delete": "Delete",
|
||||
"import.deleteConfirm": "Delete this model and its converted .cmf file from disk?",
|
||||
"import.deleted": "Deleted with its files",
|
||||
"import.state.running": "running",
|
||||
"import.state.done": "done",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelled",
|
||||
"import.featuredTitle": "Featured Ready-to-Run .cmf Models",
|
||||
"import.featuredSub": "Pre-converted .cmf models from the infosave account on HuggingFace — the list is read live, so newly published models appear here on their own.",
|
||||
"import.readyCmfBadge": "Ready .cmf",
|
||||
"import.directDownload": "Download",
|
||||
"import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
|
||||
"import.q.q8_row": "8-bit per-row — simple, robust.",
|
||||
"import.q.q4": "4-bit block — smallest, lower quality.",
|
||||
"import.q.q4_tiled": "4-bit tiled (Q4_TILED) — 1.66x faster on Metal GPU.",
|
||||
"import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
|
||||
"import.q.q1t": "1.58-bit ternary (Q1T) — BitNet b1.58 with sparse outliers.",
|
||||
"import.q.q1s": "1-bit sparse (Q1S) — 1-bit base with holographic outliers.",
|
||||
"import.q.q1p": "1-bit PTQ (Q1P) — 1-bit with error diffusion.",
|
||||
"import.q.q1": "1-bit (Q1) — pure binary format.",
|
||||
"import.q.f16": "16-bit — no quantization.",
|
||||
"import.q.f32": "32-bit — full precision (large file).",
|
||||
|
||||
"import.adv.title": "Advanced settings",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Linear core",
|
||||
"import.adv.linearHint": "How linear-attention layers are carried/folded.",
|
||||
"import.adv.nphase": "vmf_phase phases/head",
|
||||
"import.adv.vbitShape": "VBIT budget shape",
|
||||
"import.adv.meanBits": "VBIT mean bits",
|
||||
"import.adv.shard": "Shard max (GB)",
|
||||
"import.adv.noShard": "no sharding",
|
||||
"import.adv.skipMtp": "skip MTP head",
|
||||
"import.adv.o1": "O(1) attention",
|
||||
"import.adv.o1Hint": "Constant-memory streaming attention: the KV cache is released after prefill, so long contexts don't grow memory. Weights stay byte-identical — only a load-time hint is recorded; you can switch back anytime (serve with --o1 off). Validated on Qwen3-0.6B: all layers ≈ ×1.18 perplexity.",
|
||||
"import.adv.o1Off": "off — exact attention (default)",
|
||||
"import.adv.o1All": "all — convert every softmax layer",
|
||||
"import.adv.o1Deep": "deep N — only the N deepest layers",
|
||||
"import.adv.o1Custom": "custom — explicit layer list",
|
||||
"import.adv.o1DeepN": "N deep layers",
|
||||
"import.adv.o1Layers": "Layers (comma-separated)",
|
||||
"import.adv.o1mHint": "landmark budget (default 32)",
|
||||
"import.adv.o1windowHint": "exact-window width (default 128)",
|
||||
"import.adv.o1sinkHint": "permanent exact sink keys (default 4)",
|
||||
|
||||
// onboarding (first run)
|
||||
"onboarding.title": "Welcome setup",
|
||||
"onboarding.welcome": "Welcome to Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choose your language — you can change it anytime in the header.",
|
||||
"onboarding.next": "Next",
|
||||
"onboarding.back": "Back",
|
||||
"onboarding.skip": "Skip for now",
|
||||
"onboarding.skipSetup": "Skip",
|
||||
"onboarding.finish": "Finish setup",
|
||||
"onboarding.featuredTitle": "Ready-made models (one click)",
|
||||
"onboarding.featuredSub": "These are pre-converted .cmf models from infosave on HuggingFace — just Download, no conversion needed. Pick one or more to get started instantly.",
|
||||
"onboarding.download": "Download",
|
||||
"onboarding.hfTitle": "Any HuggingFace model — auto convert",
|
||||
"onboarding.hfSub": "Search any model, pick a quantization, and we convert it to .cmf automatically. No manual config edits.",
|
||||
"onboarding.noJobs": "No downloads yet.",
|
||||
"onboarding.doneTitle": "You're all set",
|
||||
"onboarding.doneSub": "Models you downloaded will be registered automatically. You can add more in Import or Models.",
|
||||
"onboarding.doneHint": "Finish connects every completed download as a local model, starts its .cmf server and points routing at it. You can change all of it later in Models.",
|
||||
"onboarding.willRegister": "Will be connected as local models",
|
||||
"onboarding.nothingToRegister": "No finished downloads yet — nothing to connect. Go back a step to pick a model.",
|
||||
"onboarding.registeredN": "Connected {n} model(s): {ids}",
|
||||
"onboarding.runtimeMissing": "The cortiq runtime isn't installed — install it in Settings → Local models to serve these.",
|
||||
"onboarding.downloaded": "downloaded",
|
||||
"onboarding.update": "Update",
|
||||
"onboarding.deleteFile": "Delete the .cmf file from disk",
|
||||
"onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.",
|
||||
"onboarding.unregistered": "Also unregistered: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Image",
|
||||
"pg.tab.video": "Video",
|
||||
"pg.generate": "Generate",
|
||||
"pg.generating": "Generating…",
|
||||
"pg.genFailed": "Generation failed — see the log.",
|
||||
"pg.result": "Result",
|
||||
"pg.genMetrics": "Generation metrics",
|
||||
"pg.imagePromptPlaceholder": "a red fox sitting in snow at sunset, photorealistic, detailed fur",
|
||||
"pg.videoPromptPlaceholder": "a corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark",
|
||||
"pg.imageEmpty": "The generated image will appear here.",
|
||||
"pg.videoEmpty": "The generated clip (with audio) will appear here.",
|
||||
"pg.width": "Width",
|
||||
"pg.height": "Height",
|
||||
"pg.steps": "Steps",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Random seed",
|
||||
"pg.cfgHint": "Guidance scale; ≤ 1 disables CFG and halves the work.",
|
||||
"pg.frames": "Frames",
|
||||
"pg.framesHint": "At 24 fps; snapped to the model's grid.",
|
||||
"pg.quality": "JPEG quality",
|
||||
"pg.videoSizeHint": "Multiples of 32; the trained short edge is 768.",
|
||||
"pg.videoStepsHint": "The Turbo LoRA is trained for 4 steps.",
|
||||
"pg.firstFrame": "First frame (optional)",
|
||||
"pg.lastFrame": "Last frame (optional)",
|
||||
"pg.keyframeHint": "Turns the run into keyframe-to-video.",
|
||||
"pg.resolution": "Resolution",
|
||||
"pg.genTime": "Generation time",
|
||||
"pg.timePerStep": "Per step",
|
||||
"pg.timePerFrame": "Per frame",
|
||||
"pg.fileSize": "File size",
|
||||
"pg.duration": "Duration",
|
||||
"pg.play": "Play",
|
||||
"pg.noMediaModels": "No local {kind} model yet. Download one (e.g. from the infosave collection) in Import:",
|
||||
"pg.tokSpeed": "Speed",
|
||||
"pg.ttft": "First token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Logs",
|
||||
"logs.subtitle": "Recent gateway log — what the terminal would show.",
|
||||
"logs.filterPlaceholder": "filter…",
|
||||
"logs.pause": "Pause",
|
||||
"logs.resume": "Resume",
|
||||
"logs.empty": "Nothing matches.",
|
||||
"notice.openAdmin": "The admin console is open without a token and reachable from other machines. Set a token if this box is not private.",
|
||||
"notice.update": "Update available: v{v}.",
|
||||
"dash.sys.title": "System",
|
||||
"dash.sys.cores": "cores",
|
||||
"dash.sys.disk": "Disk (models)",
|
||||
"dash.sys.free": "free",
|
||||
"dash.sys.gateway": "gateway",
|
||||
"fit.disk": "not enough disk",
|
||||
"fit.diskTitle": "The download is larger than the free space on the models disk.",
|
||||
"fit.ram": "may not fit in RAM",
|
||||
"fit.ramTitle": "The model is close to or larger than this machine's RAM — it may not run.",
|
||||
"pg.eta": "left",
|
||||
"pg.min": "min",
|
||||
"pg.queued": "In queue — position {n}. It starts as soon as the current generation finishes.",
|
||||
"pg.gallery": "Recent generations",
|
||||
"pg.galDeleteConfirm": "Delete this generation and its files?",
|
||||
"pg.clearChat": "Clear",
|
||||
"keys.snip.title": "Connect your app",
|
||||
"keys.snip.hint": "Paste one of these into your project — only the API key placeholder needs replacing.",
|
||||
"keys.snip.withKey": "Snippets below include the key you just created.",
|
||||
"keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
|
||||
"keys.snip.keyPh": "your API key",
|
||||
"settings.restart": "Restart everything",
|
||||
"settings.restartConfirm": "Restart the gateway and all local model servers? Requests in flight will be dropped.",
|
||||
"settings.restarting": "Restarting — the page reloads when the gateway is back…",
|
||||
"settings.shutdown": "Stop everything & power off",
|
||||
"settings.shutdownConfirm": "Stop all local model servers and power the gateway off? The API and this panel stop answering until you start the gateway again.",
|
||||
"settings.shutdownDone": "The gateway is off.",
|
||||
"settings.shutdownHint": "All local model servers are stopped and the process has exited. Start the gateway again on the server to bring the panel back.",
|
||||
"bench.title": "Local model speed",
|
||||
"bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
|
||||
"bench.run": "Measure",
|
||||
"bench.running": "measuring…",
|
||||
"bench.failed": "failed",
|
||||
"bench.none": "No local .cmf chat models yet.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Small talk",
|
||||
"task.code": "Code",
|
||||
"task.creative-writing": "Creative writing",
|
||||
"task.extraction": "Data extraction",
|
||||
"task.math": "Math",
|
||||
"task.qa": "Q&A",
|
||||
"task.summarization": "Summarization",
|
||||
"task.translation": "Translation",
|
||||
"routing.tasks.title": "Task-based routing",
|
||||
"routing.tasks.hint": "A rule's models go first for that task; the complexity chain below still applies as fallback. Empty = route by complexity only.",
|
||||
"routing.tasks.inherit": "by complexity (no rule)",
|
||||
"routing.tasks.share": "{p}% of requests (7d)",
|
||||
"routing.tasks.fromRouter": "labels live from the router",
|
||||
"routing.tasks.builtin": "router unreachable — built-in label set",
|
||||
"routing.tasks.addCustom": "Add task",
|
||||
"routing.tasks.addPlaceholder": "custom label…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Quality and speed probes through the live pipeline: keyword-checked answers, anti-loop detection, tok/s and time-to-first-token per model.",
|
||||
"bm.models": "Models to test",
|
||||
"bm.quickMode": "Quick test",
|
||||
"bm.quickHint": "The fast subset with short answers — about a minute per local model.",
|
||||
"bm.suiteInfo": "Built-in suite: {q} quick / {f} full tests, plus your custom ones.",
|
||||
"bm.run": "Run",
|
||||
"bm.pickModel": "Pick at least one model.",
|
||||
"bm.testOf": "test {i} of {n}",
|
||||
"bm.results": "Results",
|
||||
"bm.quick": "quick",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Click a cell to see the model's full answer.",
|
||||
"bm.history": "Previous runs (stored in this browser)",
|
||||
"bm.custom": "Custom tests",
|
||||
"bm.customHint": "Prompt + comma-separated keywords the answer must contain (use | for synonyms). Stored in this browser.",
|
||||
"bm.promptPh": "prompt…",
|
||||
"bm.checkPh": "keywords, comma-separated",
|
||||
"bm.addTest": "Add test",
|
||||
"bm.emptyAnswer": "empty answer",
|
||||
"bm.loopDetected": "loop detected",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Smart routing",
|
||||
"bm.mode.models": "Individual models",
|
||||
"bm.mode.routingDesc": "The product's core feature under test: every probe goes through cortiq-auto — you see which task the router detected, which model it picked, whether that matches expectations, and what it cost.",
|
||||
"bm.mode.modelsDesc": "The same suite pinned to selected models — compare their quality, speed and cost head-to-head, bypassing the router.",
|
||||
"bm.answersOk": "answers correct",
|
||||
"bm.classAcc": "task classification",
|
||||
"bm.modelDist": "Requests went to",
|
||||
"bm.expectedLbl": "expected:",
|
||||
"bm.answer": "Answer",
|
||||
"bm.liveRunning": "running…",
|
||||
"bm.spent": "spent",
|
||||
"bm.verdict": "🏁 Done: answers {p}, spent {s}",
|
||||
"bm.clearHistory": "Clear all",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — paste into the terminal (the gateway speaks the Anthropic protocol natively):",
|
||||
"keys.h.codex": "Codex — appends the provider to ~/.codex/config.toml and starts:",
|
||||
"keys.h.opencode": "OpenCode — writes ~/.config/opencode/opencode.json (overwrites!) and starts:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — appends the model to ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline (VS Code) — in extension settings pick:",
|
||||
"keys.h.cline2": "Settings → API Provider → OpenAI Compatible, then paste the values above.",
|
||||
"keys.h.note": "cortiq-auto = smart routing: the router picks the model per request. Use an exact model name to pin one. On Windows run the scripts in Git Bash / WSL, or set the same variables via the system environment settings.",
|
||||
"keys.h.protoOff": "The Anthropic protocol (/v1/messages) is disabled — Claude Code will not connect.",
|
||||
"keys.h.enableProto": "Enable",
|
||||
"keys.h.st.installed": "Installed on the gateway machine",
|
||||
"keys.h.st.configured": "Already configured for this gateway",
|
||||
"keys.h.st.notFound": "Not found on the gateway machine",
|
||||
"bm.degraded": "router degraded",
|
||||
"bm.routerDegraded": "The router did not answer in time — the request used the fallback chain; classification is not judged.",
|
||||
"bm.cutByCap": "budget spent on reasoning — answer never came",
|
||||
"shell.uiStale": "The gateway was updated — this page is running an older interface.",
|
||||
"shell.reload": "Reload",
|
||||
"keys.col.requests": "Requests",
|
||||
"keys.col.tokens": "Tokens in/out",
|
||||
"keys.col.costAll": "$ all-time",
|
||||
"routing.tasks.tier": "Complexity",
|
||||
"routing.tasks.tierAuto": "Auto (router)",
|
||||
"routing.tasks.tierHint": "Which model serves this task at which complexity; \"any\" leads at every tier. Order = fallback order. High verdicts still escalate over any-tier pins.",
|
||||
"routing.tasks.tierAny": "any complexity",
|
||||
"routing.tasks.addModel": "model",
|
||||
"routing.bands.enable": "My complexity boundaries",
|
||||
"routing.bands.hint": "Drag the handles to set where low ends and where high begins — the handles push each other, the numbers update live. The router's continuous score is re-mapped with YOUR boundaries; unchecked = the router's own bands.",
|
||||
"tier.low": "low",
|
||||
"tier.medium": "medium",
|
||||
"tier.high": "high",
|
||||
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
|
||||
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
|
||||
"settings.cache.entries": "entries",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "saved",
|
||||
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
|
||||
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
|
||||
"keys.budgetOf": "budget {b}/mo",
|
||||
"dash.kpi.orBalance": "OpenRouter balance",
|
||||
"keys.h.install": "Install:",
|
||||
"keys.snip.keyInputPh": "paste your API key",
|
||||
"keys.snip.apply": "Insert key",
|
||||
};
|
||||
|
||||
Vendored
+388
-3
@@ -30,9 +30,10 @@ export default {
|
||||
"nav.routing": "Enrutamiento",
|
||||
"nav.protocols": "Protocolos",
|
||||
"nav.keys": "Claves API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Zona de pruebas",
|
||||
"nav.settings": "Configuración",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clave del router y pagos",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "Router",
|
||||
@@ -40,6 +41,7 @@ export default {
|
||||
"status.online": "en línea",
|
||||
"status.offline": "sin conexión",
|
||||
"status.checking": "verificando…",
|
||||
"status.noKey": "sin clave",
|
||||
"theme.toggle": "Tema",
|
||||
"logout": "Cerrar sesión",
|
||||
|
||||
@@ -74,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "por tarea",
|
||||
"dash.groupby.account": "por cuenta",
|
||||
"dash.recent.title": "Solicitudes recientes",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba el Playground.",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba la Zona de pruebas.",
|
||||
"dash.col.time": "Hora",
|
||||
"dash.col.model": "Modelo",
|
||||
"dash.col.tier": "Nivel",
|
||||
@@ -88,6 +90,7 @@ export default {
|
||||
"dash.health.title": "Estado",
|
||||
"dash.health.reachable": "accesible",
|
||||
"dash.health.unreachable": "inaccesible",
|
||||
"dash.health.getKey": "Obtener una clave del router",
|
||||
|
||||
// models
|
||||
"models.title": "Modelos",
|
||||
@@ -186,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "todos",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Zona de pruebas",
|
||||
"pg.subtitle": "Envía un prompt por el pipeline en vivo e inspecciona la decisión de enrutamiento.",
|
||||
"pg.system": "Prompt del sistema",
|
||||
"pg.systemPlaceholder": "mensaje de sistema opcional",
|
||||
@@ -225,6 +228,18 @@ export default {
|
||||
"settings.router.timeout": "Tiempo de espera (ms)",
|
||||
"settings.router.verifyTls": "Verificar certificado TLS",
|
||||
"settings.router.taxonomy": "ID de taxonomía",
|
||||
"settings.router.getKey": "Obtén o recarga una clave del router en",
|
||||
"settings.router.secret": "Clave del router (valor)",
|
||||
"settings.router.test": "Probar router",
|
||||
"router.probe.ok": "Router OK ({ms} ms)",
|
||||
"router.probe.noKey": "La clave del router no está configurada",
|
||||
"router.probe.auth": "Clave rechazada — inválida o revocada",
|
||||
"router.probe.payment": "Pago requerido — plan vencido o cuota agotada",
|
||||
"router.probe.quota": "Límite de tasa excedido",
|
||||
"router.probe.timeout": "Tiempo de espera del router agotado",
|
||||
"router.probe.unreachable": "Router inaccesible",
|
||||
"router.probe.error": "Error del router",
|
||||
"router.usage": "Consumo",
|
||||
"settings.route": "Entrada de enrutamiento",
|
||||
"settings.route.strategy": "Estrategia de texto",
|
||||
"settings.route.maxChars": "Máx. caracteres",
|
||||
@@ -265,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "TTL de entrada",
|
||||
"settings.cache.embed": "Modelo de embeddings",
|
||||
"settings.cache.note": "Los ajustes del caché se aplican tras reiniciar.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importar modelo",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Borrar registros",
|
||||
"dash.clearConfirm": "¿Borrar todas las estadísticas y registros guardados? Esta acción no se puede deshacer.",
|
||||
"dash.cleared": "Registros borrados",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Introduce un id o un nombre de modelo",
|
||||
"models.form.apiKey": "Clave API",
|
||||
"models.form.apiKeyPlaceholder": "pega tu clave API",
|
||||
"models.form.apiKeyNotNeeded": "no necesaria (servidor local)",
|
||||
"models.form.apiKeyHint": "Se guarda de forma segura en la pasarela; no se vuelve a mostrar.",
|
||||
"models.form.keyStored": "Hay una clave guardada — déjalo vacío para conservarla o pega una nueva.",
|
||||
"models.form.keyFromEnv": "Se está usando una clave de una variable de entorno.",
|
||||
"models.form.paramsTitle": "Parámetros de generación y razonamiento (O(1), MTP, temperatura)",
|
||||
"models.form.temperature": "Temperatura",
|
||||
"models.form.temperatureHint": "Temperatura de generación (0.0 = determinista, 1.0+ = creativa).",
|
||||
"models.form.topP": "Top-P (muestreo nucleus)",
|
||||
"models.form.topPHint": "Umbral de muestreo (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens máx.",
|
||||
"models.form.maxTokensHint": "Número máximo de tokens generados.",
|
||||
"models.form.thinkBudget": "Modo razonamiento / presupuesto de pensamiento",
|
||||
"models.form.thinkBudgetHint": "Presupuesto de tokens de razonamiento (0 = desactivado, N > 0 = máximo de tokens de razonamiento).",
|
||||
"models.form.systemPrompt": "Prompt de sistema por defecto",
|
||||
"models.form.systemPromptHint": "Se antepone automáticamente si la petición no trae prompt de sistema.",
|
||||
"models.form.systemPromptPlaceholder": "Eres un asistente útil…",
|
||||
"models.form.testKey": "Probar y cargar modelos",
|
||||
"models.form.testing": "comprobando…",
|
||||
"models.form.foundModels": "{n} modelos cargados",
|
||||
"models.form.modelPlaceholder": "escribe para buscar…",
|
||||
"models.form.modelHint": "Pulsa «Probar y cargar modelos» para traer la lista con tu clave y luego elige uno.",
|
||||
"models.managed": "gestionado",
|
||||
"models.managedHint": "Lo proporciona el servidor CMF gestionado (Ajustes → Modelos locales). Configúralo allí.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Usar el router (desactivado = solo modelos locales)",
|
||||
"settings.cmf": "Modelos locales (CMF)",
|
||||
"settings.cmf.localOnly": "Usar solo modelos locales",
|
||||
"settings.cmf.manage": "Ejecutar un servidor de modelo local",
|
||||
"settings.cmf.autoInstall": "Instalar cortiq automáticamente (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Actualizar cortiq automáticamente",
|
||||
"settings.cmf.model": "Ruta del modelo .cmf local",
|
||||
"settings.cmf.port": "Puerto del servidor local",
|
||||
"settings.cmf.modelId": "Id del modelo en el pool",
|
||||
"settings.cmf.note": "Sirve un .cmf local mediante cortiq (github.com/infosave2007/cmf). Funciona incluso cuando el router no está disponible.",
|
||||
"settings.cmf.threads": "Hilos de trabajo",
|
||||
"settings.cmf.gpu": "Usar GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Los hilos paralelizan los matvec de decodificación (~3× más rápido; 0 = valor por defecto del runtime). Descargar en GPU ayuda sobre todo al prefill de prompts largos — la decodificación está limitada por el ancho de banda de memoria, así que en Apple Silicon es prácticamente neutro.",
|
||||
"settings.cmf.pick": "selecciona un modelo",
|
||||
"settings.cmf.models": "Modelos locales",
|
||||
"settings.cmf.modelsHelp": "Cada fila es un modelo que la pasarela ejecuta (su propio cortiq serve en su propio puerto) y registra en el pool. Añade tantos como archivos .cmf y puertos libres tengas.",
|
||||
"settings.cmf.rowHint": "id · archivo .cmf · puerto · hilos · GPU. Los cambios se aplican al reiniciar; borrar una fila detiene ese servidor.",
|
||||
"settings.cmf.addServer": "+ Añadir modelo local",
|
||||
"settings.cmf.checkPort": "Comprobar puerto",
|
||||
"settings.cmf.portFree": "puerto libre",
|
||||
"settings.cmf.portBusy": "puerto ocupado",
|
||||
"settings.cmf.useSuggested": "usar {port}",
|
||||
"settings.cmf.serving": "sirviendo {model} en :{port}",
|
||||
"settings.cmf.intro": "Ejecuta modelos en local con el motor cortiq — funciona incluso sin conexión o cuando el router está caído.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "La herramienta de línea de comandos `cortiq` que carga y ejecuta modelos .cmf. Instálala una vez antes de arrancar un servidor local.",
|
||||
"settings.cmf.install": "Instalar cortiq",
|
||||
"settings.cmf.reinstall": "Reinstalar / actualizar",
|
||||
"settings.cmf.installing": "instalando… (compila, unos minutos)",
|
||||
"settings.cmf.installedOk": "cortiq instalado",
|
||||
"settings.cmf.notInstalled": "no instalado",
|
||||
"settings.cmf.addModel": "Añadir un modelo",
|
||||
"settings.cmf.addModelHelp": "Los modelos se descargan y convierten en la pestaña Importar (HuggingFace → .cmf). Después apunta el servidor local al archivo .cmf de abajo.",
|
||||
"settings.cmf.importNote": "¿Aún no tienes modelo? Importa uno:",
|
||||
"settings.cmf.importLink": "Abrir Importar",
|
||||
"settings.cmf.modelHelp": "Ruta del archivo .cmf que se va a servir (p. ej. models/qwen.cmf), generado en la pestaña Importar. ¿Varios modelos para distintas tareas? Impórtalos y cambia esta ruta.",
|
||||
"settings.cmf.portHelp": "Puerto TCP en el que escucha el servidor de modelo local (8081 por defecto). Cámbialo solo si choca con otro servicio.",
|
||||
"settings.cmf.threadsHelp": "Cuántos núcleos de CPU usar para la generación. Más = más rápido (~3× frente a un núcleo); 0 deja que decida el runtime.",
|
||||
"settings.cmf.gpuHelp": "Descargar el cálculo matricial en la GPU (Metal/wgpu). En Apple Silicon acelera sobre todo los prompts largos — la decodificación está limitada por el ancho de banda de memoria.",
|
||||
"settings.cmf.localOnlyHelp": "Enviar cada petición al modelo local, ignorando el router y cualquier proveedor en la nube.",
|
||||
"settings.cmf.manageHelp": "La pasarela arranca `cortiq serve` para el .cmf de abajo al iniciarse y enruta hacia él. Requiere cortiq instalado (paso 1). Se aplica al reiniciar.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq falta al arrancar, instalarlo automáticamente desde crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Al arrancar, actualizar cortiq a la versión más reciente si existe una más nueva.",
|
||||
"settings.router.enabledHelp": "Dejar que el router de cortiq clasifique cada petición (tipo de tarea + complejidad) y elija un modelo. Desactivado = usar solo la tabla de enrutado local.",
|
||||
"settings.router.verifyTlsHelp": "Verificar el certificado TLS del router. Desactívalo solo con un certificado autofirmado o ligado a una IP.",
|
||||
"settings.telemetry.metricsHelp": "Exponer métricas Prometheus en GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Añadir un campo `cortiq` (tarea, modelo, coste) al cuerpo de cada respuesta. Las cabeceras de respuesta se envían siempre.",
|
||||
"settings.stats.enabledHelp": "Registrar las peticiones recientes para alimentar el panel y las gráficas.",
|
||||
"settings.cache.enabledHelp": "Caché semántica: devolver una respuesta guardada cuando un prompt nuevo se parece mucho a uno anterior, evitando la llamada al modelo. Requiere un modelo de embeddings en el pool.",
|
||||
"settings.log.bodiesHelp": "Registrar el cuerpo completo de los mensajes de petición y respuesta. Desactivado por defecto por privacidad.",
|
||||
"settings.router.saveKey": "Guardar clave",
|
||||
"settings.router.keySaved": "Clave del router guardada",
|
||||
"settings.router.keyEmpty": "Pega antes la clave",
|
||||
"settings.router.secretHelp": "Pega la clave del correo de api.allaigate.com y pulsa Guardar clave. NO es un valor de «Claves API» — esas son para los clientes que llaman a tu pasarela.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clave guardada (pega otra para reemplazarla)",
|
||||
"settings.route.strategyHelp": "Qué texto se envía al router para clasificar: last_user = solo el último mensaje del usuario; last_user_plus_system = último mensaje del usuario + prompt de sistema; concat_all = toda la conversación.",
|
||||
"settings.route.maxCharsHelp": "Máximo de caracteres de ese texto enviados al router (lo que exceda se recorta). Más = clasificación más precisa, pero llamadas al router más lentas y caras.",
|
||||
"settings.route.cacheTtlHelp": "Cuánto tiempo se cachea una decisión de enrutado para una petición idéntica antes de volver a preguntar al router (p. ej. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Perfil de enrutado por defecto cuando la petición no indica ninguno: cost-saver (el más barato), balanced o quality-first (el mejor modelo). Influye en qué modelo elige el router.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Busca en HuggingFace, convierte a un .cmf local y sírvelo.",
|
||||
"import.searchPlaceholder": "Buscar modelos en HuggingFace (p. ej. qwen3, llama)…",
|
||||
"import.searching": "Buscando…",
|
||||
"import.noResults": "No se encontraron modelos.",
|
||||
"import.configure": "Configurar la conversión",
|
||||
"import.outName": "Nombre de salida",
|
||||
"import.outNameHint": "Nombre del archivo .cmf (letras, dígitos, - y _).",
|
||||
"import.quant": "Cuantización",
|
||||
"import.convert": "Convertir y cargar",
|
||||
"import.converting": "Convirtiendo",
|
||||
"import.started": "Conversión iniciada",
|
||||
"import.failed": "La conversión falló — revisa el registro.",
|
||||
"import.register": "Registrar como modelo local",
|
||||
"import.registered": "Registrado como «{id}». Ejecuta cortiq-server para servirlo.",
|
||||
"import.jobs": "Descargas y conversiones",
|
||||
"import.cancel": "Cancelar",
|
||||
"import.cancelling": "Cancelando — limpiando el archivo incompleto…",
|
||||
"import.delete": "Eliminar",
|
||||
"import.deleteConfirm": "¿Eliminar este modelo y su archivo .cmf convertido del disco?",
|
||||
"import.deleted": "Eliminado junto con sus archivos",
|
||||
"import.state.running": "en curso",
|
||||
"import.state.done": "listo",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelado",
|
||||
"import.featuredTitle": "Modelos .cmf listos para usar",
|
||||
"import.featuredSub": "Modelos .cmf ya convertidos de la cuenta infosave en HuggingFace — la lista se lee en vivo, los modelos nuevos aparecen solos.",
|
||||
"import.readyCmfBadge": ".cmf listo",
|
||||
"import.directDownload": "Descargar",
|
||||
"import.q.q8_2f": "8 bits de dos campos (𝒲×θ) — recomendado: mejor relación calidad/tamaño.",
|
||||
"import.q.q8_row": "8 bits por fila — simple y robusto.",
|
||||
"import.q.q4": "4 bits por bloque — el más pequeño, menor calidad.",
|
||||
"import.q.q4_tiled": "4 bits en mosaico (Q4_TILED) — 1,66× más rápido en GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — presupuesto de bits por experto.",
|
||||
"import.q.q1t": "Ternario de 1,58 bits (Q1T) — BitNet b1.58 con valores atípicos dispersos.",
|
||||
"import.q.q1s": "1 bit disperso (Q1S) — base de 1 bit con valores atípicos holográficos.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit con difusión de error.",
|
||||
"import.q.q1": "1 bit (Q1) — formato puramente binario.",
|
||||
"import.q.f16": "16 bits — sin cuantización.",
|
||||
"import.q.f32": "32 bits — precisión completa (archivo grande).",
|
||||
"import.adv.title": "Ajustes avanzados",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Núcleo lineal",
|
||||
"import.adv.linearHint": "Cómo se trasladan o pliegan las capas de atención lineal.",
|
||||
"import.adv.nphase": "vmf_phase fases/cabeza",
|
||||
"import.adv.vbitShape": "Forma del presupuesto VBIT",
|
||||
"import.adv.meanBits": "Bits medios VBIT",
|
||||
"import.adv.shard": "Tamaño máx. de shard (GB)",
|
||||
"import.adv.noShard": "sin fragmentar",
|
||||
"import.adv.skipMtp": "omitir la cabeza MTP",
|
||||
"import.adv.o1": "Atención O(1)",
|
||||
"import.adv.o1Hint": "Atención en streaming con memoria constante: la caché KV se libera tras el prefill, así que los contextos largos no hacen crecer la memoria. Los pesos quedan idénticos byte a byte — solo se guarda una indicación para el momento de carga; puedes volver atrás cuando quieras (servir con --o1 off). Validado en Qwen3-0.6B: todas las capas ≈ ×1,18 de perplejidad.",
|
||||
"import.adv.o1Off": "desactivada — atención exacta (por defecto)",
|
||||
"import.adv.o1All": "all — convertir todas las capas softmax",
|
||||
"import.adv.o1Deep": "deep N — solo las N capas más profundas",
|
||||
"import.adv.o1Custom": "personalizado — lista explícita de capas",
|
||||
"import.adv.o1DeepN": "N capas profundas",
|
||||
"import.adv.o1Layers": "Capas (separadas por comas)",
|
||||
"import.adv.o1mHint": "presupuesto de landmarks (32 por defecto)",
|
||||
"import.adv.o1windowHint": "ancho de la ventana exacta (128 por defecto)",
|
||||
"import.adv.o1sinkHint": "claves sink exactas permanentes (4 por defecto)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuración inicial",
|
||||
"onboarding.welcome": "Bienvenido a Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Elige tu idioma — puedes cambiarlo cuando quieras en la cabecera.",
|
||||
"onboarding.next": "Siguiente",
|
||||
"onboarding.back": "Atrás",
|
||||
"onboarding.skip": "Omitir por ahora",
|
||||
"onboarding.skipSetup": "Omitir",
|
||||
"onboarding.finish": "Finalizar configuración",
|
||||
"onboarding.featuredTitle": "Modelos listos para usar (un clic)",
|
||||
"onboarding.featuredSub": "Son modelos .cmf ya convertidos de infosave en HuggingFace — basta con descargarlos, sin conversión. Elige uno o varios para empezar al instante.",
|
||||
"onboarding.download": "Descargar",
|
||||
"onboarding.hfTitle": "Cualquier modelo de HuggingFace — conversión automática",
|
||||
"onboarding.hfSub": "Busca cualquier modelo, elige una cuantización y lo convertimos a .cmf automáticamente. Sin editar la configuración a mano.",
|
||||
"onboarding.noJobs": "Todavía no hay descargas.",
|
||||
"onboarding.doneTitle": "Todo listo",
|
||||
"onboarding.doneSub": "Los modelos descargados se registrarán automáticamente. Puedes añadir más en Importar o Modelos.",
|
||||
"onboarding.doneHint": "«Finalizar configuración» conecta cada descarga completada como modelo local, arranca su servidor .cmf y apunta el enrutado hacia él. Todo esto se puede cambiar luego en Modelos.",
|
||||
"onboarding.willRegister": "Se conectarán como modelos locales",
|
||||
"onboarding.nothingToRegister": "Aún no hay descargas terminadas — no hay nada que conectar. Vuelve un paso atrás y elige un modelo.",
|
||||
"onboarding.registeredN": "{n} modelo(s) conectado(s): {ids}",
|
||||
"onboarding.runtimeMissing": "El runtime cortiq no está instalado — instálalo en Ajustes → Modelos locales para poder servirlos.",
|
||||
"onboarding.downloaded": "descargado",
|
||||
"onboarding.update": "Actualizar",
|
||||
"onboarding.deleteFile": "Eliminar el archivo .cmf del disco",
|
||||
"onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.",
|
||||
"onboarding.unregistered": "También se dio de baja: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Imagen",
|
||||
"pg.tab.video": "Vídeo",
|
||||
"pg.generate": "Generar",
|
||||
"pg.generating": "Generando…",
|
||||
"pg.genFailed": "La generación falló — revisa el registro.",
|
||||
"pg.result": "Resultado",
|
||||
"pg.genMetrics": "Métricas de generación",
|
||||
"pg.imagePromptPlaceholder": "un zorro rojo en la nieve al atardecer, fotorrealista, pelaje detallado",
|
||||
"pg.videoPromptPlaceholder": "un corgi con gorro de chef voltea una tortita, chisporroteo y un ladrido alegre",
|
||||
"pg.imageEmpty": "La imagen generada aparecerá aquí.",
|
||||
"pg.videoEmpty": "El clip generado (con audio) aparecerá aquí.",
|
||||
"pg.width": "Ancho",
|
||||
"pg.height": "Alto",
|
||||
"pg.steps": "Pasos",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Seed aleatorio",
|
||||
"pg.cfgHint": "Escala de guidance; ≤ 1 desactiva el CFG y reduce el trabajo a la mitad.",
|
||||
"pg.frames": "Fotogramas",
|
||||
"pg.framesHint": "A 24 fps; se ajusta a la rejilla del modelo.",
|
||||
"pg.quality": "Calidad JPEG",
|
||||
"pg.videoSizeHint": "Múltiplos de 32; el lado corto entrenado es 768.",
|
||||
"pg.videoStepsHint": "La LoRA Turbo está entrenada para 4 pasos.",
|
||||
"pg.firstFrame": "Primer fotograma (opcional)",
|
||||
"pg.lastFrame": "Último fotograma (opcional)",
|
||||
"pg.keyframeHint": "Convierte la ejecución en keyframe-to-video.",
|
||||
"pg.resolution": "Resolución",
|
||||
"pg.genTime": "Tiempo de generación",
|
||||
"pg.timePerStep": "Por paso",
|
||||
"pg.timePerFrame": "Por fotograma",
|
||||
"pg.fileSize": "Tamaño del archivo",
|
||||
"pg.duration": "Duración",
|
||||
"pg.play": "Reproducir",
|
||||
"pg.noMediaModels": "Aún no hay un modelo {kind} local. Descarga uno (p. ej. de la colección infosave) en Importar:",
|
||||
"pg.tokSpeed": "Velocidad",
|
||||
"pg.ttft": "Primer token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Registros",
|
||||
"logs.subtitle": "Registro reciente de la pasarela — lo que mostraría la terminal.",
|
||||
"logs.filterPlaceholder": "filtrar…",
|
||||
"logs.pause": "Pausa",
|
||||
"logs.resume": "Reanudar",
|
||||
"logs.empty": "Nada coincide.",
|
||||
"notice.openAdmin": "La consola está abierta sin token y accesible desde otras máquinas. Configura un token si esta máquina no es privada.",
|
||||
"notice.update": "Actualización disponible: v{v}.",
|
||||
"dash.sys.title": "Sistema",
|
||||
"dash.sys.cores": "núcleos",
|
||||
"dash.sys.disk": "Disco (modelos)",
|
||||
"dash.sys.free": "libre",
|
||||
"dash.sys.gateway": "pasarela",
|
||||
"fit.disk": "no hay espacio suficiente",
|
||||
"fit.diskTitle": "La descarga supera el espacio libre del disco de modelos.",
|
||||
"fit.ram": "puede no caber en RAM",
|
||||
"fit.ramTitle": "El modelo se acerca o supera la RAM de esta máquina — puede que no funcione.",
|
||||
"pg.eta": "restante",
|
||||
"pg.min": "min",
|
||||
"pg.queued": "En cola — posición {n}. Arranca en cuanto termine la generación actual.",
|
||||
"pg.gallery": "Generaciones recientes",
|
||||
"pg.galDeleteConfirm": "¿Eliminar esta generación y sus archivos?",
|
||||
"pg.clearChat": "Limpiar",
|
||||
"keys.snip.title": "Conecta tu aplicación",
|
||||
"keys.snip.hint": "Pega uno de estos ejemplos en tu proyecto — solo hay que sustituir el marcador de la clave.",
|
||||
"keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
|
||||
"keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
|
||||
"keys.snip.keyPh": "tu clave API",
|
||||
"settings.restart": "Reiniciar todo",
|
||||
"settings.restartConfirm": "¿Reiniciar la pasarela y todos los servidores de modelos locales? Las solicitudes en curso se interrumpirán.",
|
||||
"settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
|
||||
"settings.shutdown": "Detener todo y apagar",
|
||||
"settings.shutdownConfirm": "¿Detener todos los servidores de modelos locales y apagar la pasarela? La API y este panel dejarán de responder hasta que vuelva a iniciar la pasarela.",
|
||||
"settings.shutdownDone": "La pasarela está apagada.",
|
||||
"settings.shutdownHint": "Todos los servidores de modelos locales están detenidos y el proceso ha terminado. Inicie de nuevo la pasarela en el servidor para recuperar el panel.",
|
||||
"bench.title": "Velocidad de modelos locales",
|
||||
"bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
|
||||
"bench.run": "Medir",
|
||||
"bench.running": "midiendo…",
|
||||
"bench.failed": "falló",
|
||||
"bench.none": "Aún no hay modelos de chat .cmf locales.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Charla",
|
||||
"task.code": "Código",
|
||||
"task.creative-writing": "Escritura creativa",
|
||||
"task.extraction": "Extracción de datos",
|
||||
"task.math": "Matemáticas",
|
||||
"task.qa": "Preguntas y respuestas",
|
||||
"task.summarization": "Resumen",
|
||||
"task.translation": "Traducción",
|
||||
"routing.tasks.title": "Enrutado por tarea",
|
||||
"routing.tasks.hint": "Los modelos de una regla van primero para su tarea; la cadena por complejidad sigue como respaldo. Vacío = solo por complejidad.",
|
||||
"routing.tasks.inherit": "por complejidad (sin regla)",
|
||||
"routing.tasks.share": "{p}% de las peticiones (7 d)",
|
||||
"routing.tasks.fromRouter": "etiquetas en vivo del router",
|
||||
"routing.tasks.builtin": "router inaccesible — conjunto integrado",
|
||||
"routing.tasks.addCustom": "Añadir tarea",
|
||||
"routing.tasks.addPlaceholder": "etiqueta propia…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Pruebas de calidad y velocidad a través del pipeline en vivo: respuestas verificadas por palabras clave, detección de bucles, tok/s y tiempo del primer token por modelo.",
|
||||
"bm.models": "Modelos a probar",
|
||||
"bm.quickMode": "Prueba rápida",
|
||||
"bm.quickHint": "El subconjunto rápido con respuestas cortas — alrededor de un minuto por modelo local.",
|
||||
"bm.suiteInfo": "Suite integrada: {q} pruebas rápidas / {f} completas, más las tuyas.",
|
||||
"bm.run": "Ejecutar",
|
||||
"bm.pickModel": "Elige al menos un modelo.",
|
||||
"bm.testOf": "prueba {i} de {n}",
|
||||
"bm.results": "Resultados",
|
||||
"bm.quick": "rápida",
|
||||
"bm.test": "Prueba",
|
||||
"bm.cellHint": "Haz clic en una celda para ver la respuesta completa.",
|
||||
"bm.history": "Ejecuciones anteriores (guardadas en este navegador)",
|
||||
"bm.custom": "Pruebas propias",
|
||||
"bm.customHint": "Prompt + palabras clave separadas por comas que la respuesta debe contener (| para sinónimos). Se guardan en este navegador.",
|
||||
"bm.promptPh": "prompt…",
|
||||
"bm.checkPh": "palabras clave, separadas por comas",
|
||||
"bm.addTest": "Añadir prueba",
|
||||
"bm.emptyAnswer": "respuesta vacía",
|
||||
"bm.loopDetected": "bucle detectado",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Enrutado inteligente",
|
||||
"bm.mode.models": "Modelos individuales",
|
||||
"bm.mode.routingDesc": "La función clave del producto a prueba: cada sonda pasa por cortiq-auto — ves qué tarea detectó el router, qué modelo eligió, si coincide con lo esperado y cuánto costó.",
|
||||
"bm.mode.modelsDesc": "La misma suite fijada a los modelos elegidos — compara su calidad, velocidad y coste directamente, sin pasar por el router.",
|
||||
"bm.answersOk": "respuestas correctas",
|
||||
"bm.classAcc": "clasificación de tareas",
|
||||
"bm.modelDist": "Las peticiones fueron a",
|
||||
"bm.expectedLbl": "esperado:",
|
||||
"bm.answer": "Respuesta",
|
||||
"bm.liveRunning": "en curso…",
|
||||
"bm.spent": "gastado",
|
||||
"bm.verdict": "🏁 Listo: respuestas {p}, gastado {s}",
|
||||
"bm.clearHistory": "Borrar todo",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — pega en la terminal (la pasarela habla el protocolo Anthropic de forma nativa):",
|
||||
"keys.h.codex": "Codex — añade el proveedor a ~/.codex/config.toml y arranca:",
|
||||
"keys.h.opencode": "OpenCode — escribe ~/.config/opencode/opencode.json (¡sobrescribe!) y arranca:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — añade el modelo a ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline (VS Code) — en los ajustes de la extensión elige:",
|
||||
"keys.h.cline2": "Ajustes → API Provider → OpenAI Compatible, luego pega los valores de arriba.",
|
||||
"keys.h.note": "cortiq-auto = enrutado inteligente: el router elige el modelo en cada petición. Para fijar uno, indica su nombre exacto. En Windows ejecuta los scripts en Git Bash / WSL o define las mismas variables en el entorno del sistema.",
|
||||
"keys.h.protoOff": "El protocolo Anthropic (/v1/messages) está desactivado — Claude Code no podrá conectarse.",
|
||||
"keys.h.enableProto": "Activar",
|
||||
"keys.h.st.installed": "Instalado en la máquina de la pasarela",
|
||||
"keys.h.st.configured": "Ya configurado para esta pasarela",
|
||||
"keys.h.st.notFound": "No encontrado en la máquina de la pasarela",
|
||||
"bm.degraded": "router degradado",
|
||||
"bm.routerDegraded": "El router no respondió a tiempo — la petición usó la cadena de reserva; la clasificación no se cuenta.",
|
||||
"bm.cutByCap": "presupuesto gastado en razonar — sin respuesta final",
|
||||
"shell.uiStale": "La pasarela se actualizó — esta página usa una interfaz antigua.",
|
||||
"shell.reload": "Recargar",
|
||||
"keys.col.requests": "Peticiones",
|
||||
"keys.col.tokens": "Tokens ent/sal",
|
||||
"keys.col.costAll": "$ total",
|
||||
"routing.tasks.tier": "Complejidad",
|
||||
"routing.tasks.tierAuto": "Auto (router)",
|
||||
"routing.tasks.tierHint": "Qué modelo atiende esta tarea en qué complejidad; «cualquiera» lidera en todos los niveles. El orden = orden de respaldo. Un veredicto high sigue escalando sobre las entradas «cualquiera».",
|
||||
"routing.tasks.tierAny": "cualquier complejidad",
|
||||
"routing.tasks.addModel": "modelo",
|
||||
"routing.bands.enable": "Mis límites de complejidad",
|
||||
"routing.bands.hint": "Arrastra los tiradores: dónde termina low y dónde empieza high — se empujan entre sí y las cifras se actualizan en vivo. La puntuación continua del router se re-mapea con TUS límites; sin marcar = las bandas del propio router.",
|
||||
"tier.low": "fácil",
|
||||
"tier.medium": "medio",
|
||||
"tier.high": "difícil",
|
||||
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
|
||||
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
|
||||
"settings.cache.entries": "entradas",
|
||||
"settings.cache.hits": "aciertos",
|
||||
"settings.cache.saved": "ahorrado",
|
||||
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
|
||||
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
|
||||
"keys.budgetOf": "presupuesto {b}/mes",
|
||||
"dash.kpi.orBalance": "Saldo OpenRouter",
|
||||
"keys.h.install": "Instalar:",
|
||||
"keys.snip.keyInputPh": "pega tu clave API",
|
||||
"keys.snip.apply": "Insertar clave",
|
||||
};
|
||||
|
||||
Vendored
+388
-3
@@ -30,9 +30,10 @@ export default {
|
||||
"nav.routing": "Routage",
|
||||
"nav.protocols": "Protocoles",
|
||||
"nav.keys": "Clés API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Bac à sable",
|
||||
"nav.settings": "Paramètres",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clé routeur & facturation",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "Routeur",
|
||||
@@ -40,6 +41,7 @@ export default {
|
||||
"status.online": "en ligne",
|
||||
"status.offline": "hors ligne",
|
||||
"status.checking": "vérification…",
|
||||
"status.noKey": "pas de clé",
|
||||
"theme.toggle": "Thème",
|
||||
"logout": "Déconnexion",
|
||||
|
||||
@@ -74,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "par tâche",
|
||||
"dash.groupby.account": "par compte",
|
||||
"dash.recent.title": "Requêtes récentes",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Playground.",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Bac à sable.",
|
||||
"dash.col.time": "Heure",
|
||||
"dash.col.model": "Modèle",
|
||||
"dash.col.tier": "Niveau",
|
||||
@@ -88,6 +90,7 @@ export default {
|
||||
"dash.health.title": "Santé",
|
||||
"dash.health.reachable": "accessible",
|
||||
"dash.health.unreachable": "inaccessible",
|
||||
"dash.health.getKey": "Obtenir une clé routeur",
|
||||
|
||||
// models
|
||||
"models.title": "Modèles",
|
||||
@@ -186,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "tous",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Bac à sable",
|
||||
"pg.subtitle": "Envoyez une invite via le pipeline en direct et inspectez la décision de routage.",
|
||||
"pg.system": "Invite système",
|
||||
"pg.systemPlaceholder": "message système optionnel",
|
||||
@@ -225,6 +228,18 @@ export default {
|
||||
"settings.router.timeout": "Délai d'expiration (ms)",
|
||||
"settings.router.verifyTls": "Vérifier le certificat TLS",
|
||||
"settings.router.taxonomy": "ID de taxonomie",
|
||||
"settings.router.getKey": "Obtenir ou recharger une clé routeur sur",
|
||||
"settings.router.secret": "Clé routeur (valeur)",
|
||||
"settings.router.test": "Tester le routeur",
|
||||
"router.probe.ok": "Routeur OK ({ms} ms)",
|
||||
"router.probe.noKey": "La clé routeur n'est pas définie",
|
||||
"router.probe.auth": "Clé rejetée — invalide ou révoquée",
|
||||
"router.probe.payment": "Paiement requis — forfait expiré ou quota épuisé",
|
||||
"router.probe.quota": "Limite de débit dépassée",
|
||||
"router.probe.timeout": "Délai du routeur dépassé",
|
||||
"router.probe.unreachable": "Routeur inaccessible",
|
||||
"router.probe.error": "Erreur du routeur",
|
||||
"router.usage": "Consommation",
|
||||
"settings.route": "Entrée de routage",
|
||||
"settings.route.strategy": "Stratégie de texte",
|
||||
"settings.route.maxChars": "Caractères max",
|
||||
@@ -265,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "TTL d'entrée",
|
||||
"settings.cache.embed": "Modèle d'embeddings",
|
||||
"settings.cache.note": "Les paramètres du cache prennent effet après un redémarrage.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importer un modèle",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Effacer les journaux",
|
||||
"dash.clearConfirm": "Effacer toutes les statistiques et les journaux enregistrés ? Action irréversible.",
|
||||
"dash.cleared": "Journaux effacés",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Saisissez un identifiant ou un nom de modèle",
|
||||
"models.form.apiKey": "Clé API",
|
||||
"models.form.apiKeyPlaceholder": "collez votre clé API",
|
||||
"models.form.apiKeyNotNeeded": "non requise (serveur local)",
|
||||
"models.form.apiKeyHint": "Stockée de façon sécurisée sur la passerelle ; elle n'est plus affichée ensuite.",
|
||||
"models.form.keyStored": "Une clé est enregistrée — laissez vide pour la conserver, ou collez-en une nouvelle.",
|
||||
"models.form.keyFromEnv": "Une clé issue d'une variable d'environnement est utilisée.",
|
||||
"models.form.paramsTitle": "Paramètres de génération et de raisonnement (O(1), MTP, température)",
|
||||
"models.form.temperature": "Température",
|
||||
"models.form.temperatureHint": "Température de génération (0.0 = déterministe, 1.0+ = créatif).",
|
||||
"models.form.topP": "Top-P (échantillonnage nucleus)",
|
||||
"models.form.topPHint": "Seuil d'échantillonnage (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens max.",
|
||||
"models.form.maxTokensHint": "Nombre maximal de tokens générés.",
|
||||
"models.form.thinkBudget": "Mode raisonnement / budget de réflexion",
|
||||
"models.form.thinkBudgetHint": "Budget de tokens de raisonnement (0 = désactivé, N > 0 = maximum de tokens de raisonnement).",
|
||||
"models.form.systemPrompt": "Prompt système par défaut",
|
||||
"models.form.systemPromptHint": "Ajouté automatiquement si la requête ne contient pas de prompt système.",
|
||||
"models.form.systemPromptPlaceholder": "Vous êtes un assistant utile…",
|
||||
"models.form.testKey": "Tester et charger les modèles",
|
||||
"models.form.testing": "vérification…",
|
||||
"models.form.foundModels": "{n} modèles chargés",
|
||||
"models.form.modelPlaceholder": "tapez pour rechercher…",
|
||||
"models.form.modelHint": "Cliquez sur « Tester et charger les modèles » pour récupérer la liste liée à votre clé, puis choisissez-en un.",
|
||||
"models.managed": "géré",
|
||||
"models.managedHint": "Fourni par le serveur CMF géré (Paramètres → Modèles locaux). Configurez-le là-bas.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Utiliser le routeur (désactivé = modèles locaux uniquement)",
|
||||
"settings.cmf": "Modèles locaux (CMF)",
|
||||
"settings.cmf.localOnly": "Utiliser uniquement les modèles locaux",
|
||||
"settings.cmf.manage": "Lancer un serveur de modèle local",
|
||||
"settings.cmf.autoInstall": "Installer cortiq automatiquement (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Mettre cortiq à jour automatiquement",
|
||||
"settings.cmf.model": "Chemin du modèle .cmf local",
|
||||
"settings.cmf.port": "Port du serveur local",
|
||||
"settings.cmf.modelId": "Identifiant du modèle dans le pool",
|
||||
"settings.cmf.note": "Sert un .cmf local via cortiq (github.com/infosave2007/cmf). Fonctionne même quand le routeur est indisponible.",
|
||||
"settings.cmf.threads": "Threads de travail",
|
||||
"settings.cmf.gpu": "Utiliser le GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Les threads parallélisent les matvecs de décodage (~3× plus rapide ; 0 = valeur par défaut du runtime). Le déport GPU aide surtout au prefill des longs prompts — le décodage est limité par la bande passante mémoire, donc sur Apple Silicon l'effet est à peu près neutre.",
|
||||
"settings.cmf.pick": "choisir un modèle",
|
||||
"settings.cmf.models": "Modèles locaux",
|
||||
"settings.cmf.modelsHelp": "Chaque ligne est un modèle que la passerelle exécute (son propre cortiq serve sur son propre port) et enregistre dans le pool. Ajoutez-en autant que vous avez de fichiers .cmf et de ports libres.",
|
||||
"settings.cmf.rowHint": "id · fichier .cmf · port · threads · GPU. Les changements prennent effet au redémarrage ; supprimer une ligne arrête ce serveur.",
|
||||
"settings.cmf.addServer": "+ Ajouter un modèle local",
|
||||
"settings.cmf.checkPort": "Vérifier le port",
|
||||
"settings.cmf.portFree": "port libre",
|
||||
"settings.cmf.portBusy": "port occupé",
|
||||
"settings.cmf.useSuggested": "utiliser {port}",
|
||||
"settings.cmf.serving": "{model} servi sur :{port}",
|
||||
"settings.cmf.intro": "Exécutez des modèles en local avec le moteur cortiq — fonctionne même hors ligne ou quand le routeur est en panne.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "L'outil en ligne de commande `cortiq` qui charge et exécute les modèles .cmf. Installez-le une fois avant de démarrer un serveur local.",
|
||||
"settings.cmf.install": "Installer cortiq",
|
||||
"settings.cmf.reinstall": "Réinstaller / mettre à jour",
|
||||
"settings.cmf.installing": "installation… (compilation, quelques minutes)",
|
||||
"settings.cmf.installedOk": "cortiq installé",
|
||||
"settings.cmf.notInstalled": "non installé",
|
||||
"settings.cmf.addModel": "Ajouter un modèle",
|
||||
"settings.cmf.addModelHelp": "Les modèles se téléchargent et se convertissent dans l'onglet Importer (HuggingFace → .cmf). Faites ensuite pointer le serveur local vers le fichier .cmf ci-dessous.",
|
||||
"settings.cmf.importNote": "Pas encore de modèle ? Importez-en un :",
|
||||
"settings.cmf.importLink": "Ouvrir Importer",
|
||||
"settings.cmf.modelHelp": "Chemin du fichier .cmf à servir (par ex. models/qwen.cmf), produit par l'onglet Importer. Plusieurs modèles pour des tâches différentes ? Importez-les et changez ce chemin.",
|
||||
"settings.cmf.portHelp": "Port TCP sur lequel écoute le serveur de modèle local (8081 par défaut). À changer seulement en cas de conflit avec un autre service.",
|
||||
"settings.cmf.threadsHelp": "Nombre de cœurs CPU utilisés pour la génération. Plus = plus rapide (~3× par rapport à un seul cœur) ; 0 laisse le runtime décider.",
|
||||
"settings.cmf.gpuHelp": "Déporter les calculs matriciels sur le GPU (Metal/wgpu). Sur Apple Silicon, cela accélère surtout les longs prompts — le décodage est limité par la bande passante mémoire.",
|
||||
"settings.cmf.localOnlyHelp": "Router chaque requête vers le modèle local, en ignorant le routeur et tous les fournisseurs cloud.",
|
||||
"settings.cmf.manageHelp": "Au démarrage, la passerelle lance `cortiq serve` pour le .cmf ci-dessous et y route les requêtes. Nécessite cortiq installé (étape 1). Prend effet au redémarrage.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq est absent au démarrage, l'installer automatiquement depuis crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Au démarrage, mettre cortiq à niveau vers la dernière version si elle est plus récente.",
|
||||
"settings.router.enabledHelp": "Laisser le routeur cortiq classer chaque requête (type de tâche + complexité) et choisir un modèle. Désactivé = utiliser uniquement la table de routage locale.",
|
||||
"settings.router.verifyTlsHelp": "Vérifier le certificat TLS du routeur. À désactiver uniquement pour un certificat auto-signé ou lié à une IP.",
|
||||
"settings.telemetry.metricsHelp": "Exposer les métriques Prometheus sur GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Ajouter un champ `cortiq` (tâche, modèle, coût) au corps de chaque réponse. Les en-têtes de réponse sont toujours envoyés.",
|
||||
"settings.stats.enabledHelp": "Enregistrer les requêtes récentes pour alimenter le tableau de bord et les graphiques.",
|
||||
"settings.cache.enabledHelp": "Cache sémantique : renvoyer une réponse enregistrée quand un nouveau prompt ressemble beaucoup à un précédent, en évitant l'appel au modèle. Nécessite un modèle d'embedding dans le pool.",
|
||||
"settings.log.bodiesHelp": "Journaliser le corps complet des messages de requête et de réponse. Désactivé par défaut pour la confidentialité.",
|
||||
"settings.router.saveKey": "Enregistrer la clé",
|
||||
"settings.router.keySaved": "Clé du routeur enregistrée",
|
||||
"settings.router.keyEmpty": "Collez d'abord la clé",
|
||||
"settings.router.secretHelp": "Collez la clé reçue par e-mail depuis api.allaigate.com, puis cliquez sur Enregistrer la clé. Ce n'est PAS une valeur de « Clés API » — celles-ci servent aux clients qui appellent votre passerelle.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clé enregistrée (recollez-en une pour la remplacer)",
|
||||
"settings.route.strategyHelp": "Texte envoyé au routeur pour la classification : last_user = uniquement le dernier message utilisateur ; last_user_plus_system = dernier message utilisateur + prompt système ; concat_all = toute la conversation.",
|
||||
"settings.route.maxCharsHelp": "Nombre maximal de caractères de ce texte envoyés au routeur (au-delà, il est tronqué). Plus = classification plus précise, mais appels au routeur plus lents et plus coûteux.",
|
||||
"settings.route.cacheTtlHelp": "Durée de mise en cache d'une décision de routage pour une requête identique avant de réinterroger le routeur (par ex. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Profil de routage par défaut quand la requête n'en précise aucun : cost-saver (le moins cher), balanced ou quality-first (meilleur modèle). Influence le modèle choisi par le routeur.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Cherchez sur HuggingFace, convertissez en .cmf local et servez-le.",
|
||||
"import.searchPlaceholder": "Rechercher des modèles HuggingFace (par ex. qwen3, llama)…",
|
||||
"import.searching": "Recherche…",
|
||||
"import.noResults": "Aucun modèle trouvé.",
|
||||
"import.configure": "Configurer la conversion",
|
||||
"import.outName": "Nom de sortie",
|
||||
"import.outNameHint": "Nom du fichier .cmf (lettres, chiffres, - et _).",
|
||||
"import.quant": "Quantification",
|
||||
"import.convert": "Convertir et charger",
|
||||
"import.converting": "Conversion",
|
||||
"import.started": "Conversion lancée",
|
||||
"import.failed": "Échec de la conversion — voir le journal.",
|
||||
"import.register": "Enregistrer comme modèle local",
|
||||
"import.registered": "Enregistré sous « {id} ». Lancez cortiq-server pour le servir.",
|
||||
"import.jobs": "Téléchargements et conversions",
|
||||
"import.cancel": "Annuler",
|
||||
"import.cancelling": "Annulation — nettoyage du fichier incomplet…",
|
||||
"import.delete": "Supprimer",
|
||||
"import.deleteConfirm": "Supprimer ce modèle et son fichier .cmf converti du disque ?",
|
||||
"import.deleted": "Supprimé avec ses fichiers",
|
||||
"import.state.running": "en cours",
|
||||
"import.state.done": "terminé",
|
||||
"import.state.error": "erreur",
|
||||
"import.state.cancelled": "annulé",
|
||||
"import.featuredTitle": "Modèles .cmf prêts à l'emploi",
|
||||
"import.featuredSub": "Modèles .cmf déjà convertis du compte infosave sur HuggingFace — la liste est lue en direct, les nouveaux modèles apparaissent d'eux-mêmes.",
|
||||
"import.readyCmfBadge": ".cmf prêt",
|
||||
"import.directDownload": "Télécharger",
|
||||
"import.q.q8_2f": "8 bits à deux champs (𝒲×θ) — recommandé : meilleur rapport qualité/taille.",
|
||||
"import.q.q8_row": "8 bits par ligne — simple et robuste.",
|
||||
"import.q.q4": "4 bits par bloc — le plus compact, qualité moindre.",
|
||||
"import.q.q4_tiled": "4 bits en tuiles (Q4_TILED) — 1,66× plus rapide sur GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — budget de bits par expert.",
|
||||
"import.q.q1t": "Ternaire 1,58 bit (Q1T) — BitNet b1.58 avec valeurs aberrantes éparses.",
|
||||
"import.q.q1s": "1 bit épars (Q1S) — base 1 bit avec valeurs aberrantes holographiques.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit avec diffusion d'erreur.",
|
||||
"import.q.q1": "1 bit (Q1) — format purement binaire.",
|
||||
"import.q.f16": "16 bits — sans quantification.",
|
||||
"import.q.f32": "32 bits — pleine précision (fichier volumineux).",
|
||||
"import.adv.title": "Paramètres avancés",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Cœur linéaire",
|
||||
"import.adv.linearHint": "Comment les couches d'attention linéaire sont reportées ou fusionnées.",
|
||||
"import.adv.nphase": "vmf_phase phases/tête",
|
||||
"import.adv.vbitShape": "Forme du budget VBIT",
|
||||
"import.adv.meanBits": "Bits moyens VBIT",
|
||||
"import.adv.shard": "Taille max. des shards (Go)",
|
||||
"import.adv.noShard": "pas de découpage",
|
||||
"import.adv.skipMtp": "ignorer la tête MTP",
|
||||
"import.adv.o1": "Attention O(1)",
|
||||
"import.adv.o1Hint": "Attention en flux à mémoire constante : le cache KV est libéré après le prefill, la mémoire ne croît donc pas avec de longs contextes. Les poids restent identiques octet pour octet — seule une indication de chargement est enregistrée ; vous pouvez revenir en arrière à tout moment (servir avec --o1 off). Validé sur Qwen3-0.6B : toutes couches ≈ ×1,18 de perplexité.",
|
||||
"import.adv.o1Off": "désactivé — attention exacte (par défaut)",
|
||||
"import.adv.o1All": "all — convertir chaque couche softmax",
|
||||
"import.adv.o1Deep": "deep N — seulement les N couches les plus profondes",
|
||||
"import.adv.o1Custom": "personnalisé — liste explicite de couches",
|
||||
"import.adv.o1DeepN": "N couches profondes",
|
||||
"import.adv.o1Layers": "Couches (séparées par des virgules)",
|
||||
"import.adv.o1mHint": "budget de landmarks (32 par défaut)",
|
||||
"import.adv.o1windowHint": "largeur de la fenêtre exacte (128 par défaut)",
|
||||
"import.adv.o1sinkHint": "clés sink exactes permanentes (4 par défaut)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuration initiale",
|
||||
"onboarding.welcome": "Bienvenue dans Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choisissez votre langue — vous pourrez la changer à tout moment dans l'en-tête.",
|
||||
"onboarding.next": "Suivant",
|
||||
"onboarding.back": "Retour",
|
||||
"onboarding.skip": "Passer pour l'instant",
|
||||
"onboarding.skipSetup": "Passer",
|
||||
"onboarding.finish": "Terminer la configuration",
|
||||
"onboarding.featuredTitle": "Modèles prêts à l'emploi (en un clic)",
|
||||
"onboarding.featuredSub": "Ce sont des modèles .cmf déjà convertis, publiés par infosave sur HuggingFace — il suffit de les télécharger, aucune conversion nécessaire. Choisissez-en un ou plusieurs pour démarrer immédiatement.",
|
||||
"onboarding.download": "Télécharger",
|
||||
"onboarding.hfTitle": "N'importe quel modèle HuggingFace — conversion automatique",
|
||||
"onboarding.hfSub": "Cherchez un modèle, choisissez une quantification, et nous le convertissons automatiquement en .cmf. Aucune modification manuelle de la configuration.",
|
||||
"onboarding.noJobs": "Aucun téléchargement pour l'instant.",
|
||||
"onboarding.doneTitle": "Tout est prêt",
|
||||
"onboarding.doneSub": "Les modèles téléchargés seront enregistrés automatiquement. Vous pourrez en ajouter d'autres dans Importer ou Modèles.",
|
||||
"onboarding.doneHint": "« Terminer la configuration » connecte chaque téléchargement achevé comme modèle local, démarre son serveur .cmf et y dirige le routage. Tout cela reste modifiable ensuite dans Modèles.",
|
||||
"onboarding.willRegister": "Seront connectés comme modèles locaux",
|
||||
"onboarding.nothingToRegister": "Aucun téléchargement terminé — rien à connecter. Revenez à l'étape précédente pour choisir un modèle.",
|
||||
"onboarding.registeredN": "{n} modèle(s) connecté(s) : {ids}",
|
||||
"onboarding.runtimeMissing": "Le runtime cortiq n'est pas installé — installez-le dans Paramètres → Modèles locaux pour les servir.",
|
||||
"onboarding.downloaded": "téléchargé",
|
||||
"onboarding.update": "Mettre à jour",
|
||||
"onboarding.deleteFile": "Supprimer le fichier .cmf du disque",
|
||||
"onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.",
|
||||
"onboarding.unregistered": "Également désenregistré : {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Image",
|
||||
"pg.tab.video": "Vidéo",
|
||||
"pg.generate": "Générer",
|
||||
"pg.generating": "Génération…",
|
||||
"pg.genFailed": "Échec de la génération — voir le journal.",
|
||||
"pg.result": "Résultat",
|
||||
"pg.genMetrics": "Métriques de génération",
|
||||
"pg.imagePromptPlaceholder": "un renard roux dans la neige au coucher du soleil, photoréaliste, fourrure détaillée",
|
||||
"pg.videoPromptPlaceholder": "un corgi en toque de chef fait sauter une crêpe, grésillements et aboiement joyeux",
|
||||
"pg.imageEmpty": "L'image générée apparaîtra ici.",
|
||||
"pg.videoEmpty": "Le clip généré (avec le son) apparaîtra ici.",
|
||||
"pg.width": "Largeur",
|
||||
"pg.height": "Hauteur",
|
||||
"pg.steps": "Étapes",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Seed aléatoire",
|
||||
"pg.cfgHint": "Force du guidance ; ≤ 1 désactive le CFG et divise le travail par deux.",
|
||||
"pg.frames": "Images",
|
||||
"pg.framesHint": "À 24 fps ; arrondi à la grille du modèle.",
|
||||
"pg.quality": "Qualité JPEG",
|
||||
"pg.videoSizeHint": "Multiples de 32 ; le petit côté entraîné est 768.",
|
||||
"pg.videoStepsHint": "La LoRA Turbo est entraînée pour 4 étapes.",
|
||||
"pg.firstFrame": "Première image (optionnel)",
|
||||
"pg.lastFrame": "Dernière image (optionnel)",
|
||||
"pg.keyframeHint": "Transforme la génération en keyframe-to-video.",
|
||||
"pg.resolution": "Résolution",
|
||||
"pg.genTime": "Temps de génération",
|
||||
"pg.timePerStep": "Par étape",
|
||||
"pg.timePerFrame": "Par image",
|
||||
"pg.fileSize": "Taille du fichier",
|
||||
"pg.duration": "Durée",
|
||||
"pg.play": "Lire",
|
||||
"pg.noMediaModels": "Pas encore de modèle {kind} local. Téléchargez-en un (par ex. de la collection infosave) dans Importer :",
|
||||
"pg.tokSpeed": "Vitesse",
|
||||
"pg.ttft": "Premier token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Journaux",
|
||||
"logs.subtitle": "Journal récent de la passerelle — ce que montrerait le terminal.",
|
||||
"logs.filterPlaceholder": "filtrer…",
|
||||
"logs.pause": "Pause",
|
||||
"logs.resume": "Reprendre",
|
||||
"logs.empty": "Rien ne correspond.",
|
||||
"notice.openAdmin": "La console est ouverte sans jeton et accessible depuis d'autres machines. Définissez un jeton si cette machine n'est pas privée.",
|
||||
"notice.update": "Mise à jour disponible : v{v}.",
|
||||
"dash.sys.title": "Système",
|
||||
"dash.sys.cores": "cœurs",
|
||||
"dash.sys.disk": "Disque (modèles)",
|
||||
"dash.sys.free": "libre",
|
||||
"dash.sys.gateway": "passerelle",
|
||||
"fit.disk": "espace disque insuffisant",
|
||||
"fit.diskTitle": "Le téléchargement dépasse l'espace libre du disque des modèles.",
|
||||
"fit.ram": "risque de ne pas tenir en RAM",
|
||||
"fit.ramTitle": "Le modèle approche ou dépasse la RAM de cette machine — il pourrait ne pas tourner.",
|
||||
"pg.eta": "restant",
|
||||
"pg.min": "min",
|
||||
"pg.queued": "En file d'attente — position {n}. Démarre dès la fin de la génération en cours.",
|
||||
"pg.gallery": "Générations récentes",
|
||||
"pg.galDeleteConfirm": "Supprimer cette génération et ses fichiers ?",
|
||||
"pg.clearChat": "Effacer",
|
||||
"keys.snip.title": "Connecter votre application",
|
||||
"keys.snip.hint": "Collez un de ces exemples dans votre projet — seul le espace réservé de la clé est à remplacer.",
|
||||
"keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
|
||||
"keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
|
||||
"keys.snip.keyPh": "votre clé API",
|
||||
"settings.restart": "Tout redémarrer",
|
||||
"settings.restartConfirm": "Redémarrer la passerelle et tous les serveurs de modèles locaux ? Les requêtes en cours seront interrompues.",
|
||||
"settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
|
||||
"settings.shutdown": "Tout arrêter et éteindre",
|
||||
"settings.shutdownConfirm": "Arrêter tous les serveurs de modèles locaux et éteindre la passerelle ? L'API et ce panneau ne répondront plus tant que vous n'aurez pas relancé la passerelle.",
|
||||
"settings.shutdownDone": "La passerelle est éteinte.",
|
||||
"settings.shutdownHint": "Tous les serveurs de modèles locaux sont arrêtés et le processus est terminé. Relancez la passerelle sur le serveur pour retrouver le panneau.",
|
||||
"bench.title": "Vitesse des modèles locaux",
|
||||
"bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
|
||||
"bench.run": "Mesurer",
|
||||
"bench.running": "mesure…",
|
||||
"bench.failed": "échec",
|
||||
"bench.none": "Pas encore de modèles de chat .cmf locaux.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Bavardage",
|
||||
"task.code": "Code",
|
||||
"task.creative-writing": "Écriture créative",
|
||||
"task.extraction": "Extraction de données",
|
||||
"task.math": "Mathématiques",
|
||||
"task.qa": "Questions-réponses",
|
||||
"task.summarization": "Résumé",
|
||||
"task.translation": "Traduction",
|
||||
"routing.tasks.title": "Routage par tâche",
|
||||
"routing.tasks.hint": "Les modèles d'une règle passent en premier pour leur tâche ; la chaîne par complexité reste en secours. Vide = complexité seule.",
|
||||
"routing.tasks.inherit": "par complexité (aucune règle)",
|
||||
"routing.tasks.share": "{p}% des requêtes (7 j)",
|
||||
"routing.tasks.fromRouter": "étiquettes reçues du routeur",
|
||||
"routing.tasks.builtin": "routeur injoignable — jeu d'étiquettes intégré",
|
||||
"routing.tasks.addCustom": "Ajouter une tâche",
|
||||
"routing.tasks.addPlaceholder": "étiquette personnalisée…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Sondes de qualité et de vitesse via le pipeline en direct : réponses vérifiées par mots-clés, boucles détectées, tok/s et temps du premier token par modèle.",
|
||||
"bm.models": "Modèles à tester",
|
||||
"bm.quickMode": "Test rapide",
|
||||
"bm.quickHint": "Le sous-ensemble rapide aux réponses courtes — environ une minute par modèle local.",
|
||||
"bm.suiteInfo": "Suite intégrée : {q} tests rapides / {f} complets, plus les vôtres.",
|
||||
"bm.run": "Lancer",
|
||||
"bm.pickModel": "Choisissez au moins un modèle.",
|
||||
"bm.testOf": "test {i} sur {n}",
|
||||
"bm.results": "Résultats",
|
||||
"bm.quick": "rapide",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Cliquez sur une cellule pour voir la réponse complète.",
|
||||
"bm.history": "Exécutions précédentes (stockées dans ce navigateur)",
|
||||
"bm.custom": "Tests personnalisés",
|
||||
"bm.customHint": "Prompt + mots-clés séparés par des virgules que la réponse doit contenir (| pour les synonymes). Stockés dans ce navigateur.",
|
||||
"bm.promptPh": "prompt…",
|
||||
"bm.checkPh": "mots-clés, séparés par des virgules",
|
||||
"bm.addTest": "Ajouter un test",
|
||||
"bm.emptyAnswer": "réponse vide",
|
||||
"bm.loopDetected": "boucle détectée",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Routage intelligent",
|
||||
"bm.mode.models": "Modèles individuels",
|
||||
"bm.mode.routingDesc": "La fonctionnalité clé du produit à l'épreuve : chaque sonde passe par cortiq-auto — vous voyez quelle tâche le routeur a détectée, quel modèle il a choisi, si cela correspond aux attentes et combien cela a coûté.",
|
||||
"bm.mode.modelsDesc": "La même suite épinglée aux modèles choisis — comparez leur qualité, vitesse et coût en direct, sans passer par le routeur.",
|
||||
"bm.answersOk": "réponses correctes",
|
||||
"bm.classAcc": "classification des tâches",
|
||||
"bm.modelDist": "Requêtes envoyées vers",
|
||||
"bm.expectedLbl": "attendu :",
|
||||
"bm.answer": "Réponse",
|
||||
"bm.liveRunning": "en cours…",
|
||||
"bm.spent": "dépensé",
|
||||
"bm.verdict": "🏁 Terminé : réponses {p}, dépensé {s}",
|
||||
"bm.clearHistory": "Tout effacer",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — collez dans le terminal (la passerelle parle nativement le protocole Anthropic) :",
|
||||
"keys.h.codex": "Codex — ajoute le fournisseur à ~/.codex/config.toml et démarre :",
|
||||
"keys.h.opencode": "OpenCode — écrit ~/.config/opencode/opencode.json (écrase !) et démarre :",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — ajoute le modèle à ~/.continue/config.yaml :",
|
||||
"keys.h.cline1": "Cline (VS Code) — dans les réglages de l'extension, choisissez :",
|
||||
"keys.h.cline2": "Réglages → API Provider → OpenAI Compatible, puis collez les valeurs ci-dessus.",
|
||||
"keys.h.note": "cortiq-auto = routage intelligent : le routeur choisit le modèle à chaque requête. Pour en fixer un, indiquez son nom exact. Sous Windows, lancez les scripts dans Git Bash / WSL ou définissez les mêmes variables via l'environnement système.",
|
||||
"keys.h.protoOff": "Le protocole Anthropic (/v1/messages) est désactivé — Claude Code ne pourra pas se connecter.",
|
||||
"keys.h.enableProto": "Activer",
|
||||
"keys.h.st.installed": "Installé sur la machine de la passerelle",
|
||||
"keys.h.st.configured": "Déjà configuré pour cette passerelle",
|
||||
"keys.h.st.notFound": "Introuvable sur la machine de la passerelle",
|
||||
"bm.degraded": "routeur dégradé",
|
||||
"bm.routerDegraded": "Le routeur n'a pas répondu à temps — la requête a suivi la chaîne de secours ; la classification n'est pas comptée.",
|
||||
"bm.cutByCap": "budget épuisé en raisonnement — pas de réponse",
|
||||
"shell.uiStale": "La passerelle a été mise à jour — cette page utilise une ancienne interface.",
|
||||
"shell.reload": "Recharger",
|
||||
"keys.col.requests": "Requêtes",
|
||||
"keys.col.tokens": "Tokens in/out",
|
||||
"keys.col.costAll": "$ au total",
|
||||
"routing.tasks.tier": "Complexité",
|
||||
"routing.tasks.tierAuto": "Auto (routeur)",
|
||||
"routing.tasks.tierHint": "Quel modèle sert cette tâche à quelle complexité ; « toute » mène à tous les niveaux. L'ordre = ordre de repli. Un verdict high escalade toujours au-dessus des entrées « toute ».",
|
||||
"routing.tasks.tierAny": "toute complexité",
|
||||
"routing.tasks.addModel": "modèle",
|
||||
"routing.bands.enable": "Mes limites de complexité",
|
||||
"routing.bands.hint": "Faites glisser les poignées : où finit low et où commence high — elles se poussent l'une l'autre, les chiffres se mettent à jour en direct. Le score continu du routeur est re-découpé avec VOS limites ; décoché = les bandes du routeur.",
|
||||
"tier.low": "facile",
|
||||
"tier.medium": "moyen",
|
||||
"tier.high": "difficile",
|
||||
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
|
||||
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
|
||||
"settings.cache.entries": "entrées",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "économisé",
|
||||
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
|
||||
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
|
||||
"keys.budgetOf": "budget {b}/mois",
|
||||
"dash.kpi.orBalance": "Solde OpenRouter",
|
||||
"keys.h.install": "Installer :",
|
||||
"keys.snip.keyInputPh": "collez votre clé API",
|
||||
"keys.snip.apply": "Insérer la clé",
|
||||
};
|
||||
|
||||
Vendored
+382
-5
@@ -25,14 +25,16 @@ export default {
|
||||
"common.all": "все",
|
||||
|
||||
// nav
|
||||
"nav.dashboard": "Дашборд",
|
||||
"nav.dashboard": "Рабочий стол",
|
||||
"nav.models": "Модели",
|
||||
"nav.import": "Импорт модели",
|
||||
"nav.routing": "Роутинг",
|
||||
"nav.protocols": "Протоколы",
|
||||
"nav.keys": "API-ключи",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Песочница",
|
||||
"nav.settings": "Настройки",
|
||||
"nav.foot": "Шлюз · v{v}",
|
||||
"nav.billing": "Ключ роутера и оплата",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "Роутер",
|
||||
@@ -40,6 +42,7 @@ export default {
|
||||
"status.online": "в сети",
|
||||
"status.offline": "недоступен",
|
||||
"status.checking": "проверка…",
|
||||
"status.noKey": "нет ключа",
|
||||
"theme.toggle": "Тема",
|
||||
"logout": "Выйти",
|
||||
|
||||
@@ -53,8 +56,11 @@ export default {
|
||||
"login.failed": "Неверный токен администратора.",
|
||||
|
||||
// dashboard
|
||||
"dash.title": "Дашборд",
|
||||
"dash.title": "Рабочий стол",
|
||||
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
|
||||
"dash.clear": "Очистить логи",
|
||||
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
|
||||
"dash.cleared": "Логи очищены",
|
||||
"dash.range": "Период",
|
||||
"dash.range.1h": "1ч",
|
||||
"dash.range.24h": "24ч",
|
||||
@@ -74,7 +80,7 @@ export default {
|
||||
"dash.groupby.label": "по задаче",
|
||||
"dash.groupby.account": "по аккаунту",
|
||||
"dash.recent.title": "Последние запросы",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Playground.",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Песочницу.",
|
||||
"dash.col.time": "Время",
|
||||
"dash.col.model": "Модель",
|
||||
"dash.col.tier": "Уровень",
|
||||
@@ -88,6 +94,7 @@ export default {
|
||||
"dash.health.title": "Состояние",
|
||||
"dash.health.reachable": "доступен",
|
||||
"dash.health.unreachable": "недоступен",
|
||||
"dash.health.getKey": "Получить ключ роутера",
|
||||
|
||||
// models
|
||||
"models.title": "Модели",
|
||||
@@ -124,6 +131,32 @@ export default {
|
||||
"models.form.secretMissing": "не задан",
|
||||
"models.form.secretPlaceholder": "задать или заменить ключ (только запись)",
|
||||
"models.deleteConfirm": "Удалить модель \"{id}\"?",
|
||||
"models.form.idOrModel": "Укажите id или имя модели",
|
||||
"models.form.apiKey": "API-ключ",
|
||||
"models.form.apiKeyPlaceholder": "вставьте API-ключ",
|
||||
"models.form.apiKeyNotNeeded": "не нужен (локальный сервер)",
|
||||
"models.form.apiKeyHint": "Хранится безопасно на шлюзе; больше не показывается.",
|
||||
"models.form.keyStored": "Ключ сохранён — оставьте пустым, чтобы не менять, или вставьте новый.",
|
||||
"models.form.keyFromEnv": "Используется ключ из переменной окружения.",
|
||||
"models.form.paramsTitle": "Параметры генерации, O(1) и режим размышления",
|
||||
"models.form.temperature": "Температура",
|
||||
"models.form.temperatureHint": "Температура генерации (0.0 — точные факты, 1.0+ — творчество).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Порог сэмплирования токенов (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Макс. токенов (Max Tokens)",
|
||||
"models.form.maxTokensHint": "Максимальное количество генерируемых токенов.",
|
||||
"models.form.thinkBudget": "Режим размышления / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Бюджет токенов мыслей (0 = выключен, N > 0 = лимит рассуждений).",
|
||||
"models.form.systemPrompt": "Системный промпт по умолчанию",
|
||||
"models.form.systemPromptHint": "Автоматически подставляется, если запрос не содержит системное сообщение.",
|
||||
"models.form.systemPromptPlaceholder": "Вы — полезный ассистент...",
|
||||
"models.form.testKey": "Проверить и загрузить модели",
|
||||
"models.form.testing": "проверка…",
|
||||
"models.form.foundModels": "загружено моделей: {n}",
|
||||
"models.form.modelPlaceholder": "введите для поиска…",
|
||||
"models.form.modelHint": "Нажмите «Проверить и загрузить модели», чтобы подтянуть список по ключу, затем выберите.",
|
||||
"models.managed": "управляемая",
|
||||
"models.managedHint": "Предоставляется CMF-сервером (Настройки → Локальные модели). Настраивается там.",
|
||||
|
||||
// routing
|
||||
"routing.title": "Роутинг",
|
||||
@@ -186,7 +219,7 @@ export default {
|
||||
"keys.allowAll": "все",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Песочница",
|
||||
"pg.subtitle": "Отправьте запрос через активный пайплайн и изучите решение роутера.",
|
||||
"pg.system": "Системный промпт",
|
||||
"pg.systemPlaceholder": "необязательное системное сообщение",
|
||||
@@ -224,12 +257,83 @@ export default {
|
||||
"settings.router.keyEnv": "Имя env-переменной с ключом роутера",
|
||||
"settings.router.timeout": "Таймаут (мс)",
|
||||
"settings.router.verifyTls": "Проверять TLS-сертификат",
|
||||
"settings.router.enabled": "Использовать роутер (выкл = только локальные модели)",
|
||||
"settings.cmf": "Локальные модели (CMF)",
|
||||
"settings.cmf.localOnly": "Использовать только локальные модели",
|
||||
"settings.cmf.manage": "Запускать локальный сервер моделей",
|
||||
"settings.cmf.autoInstall": "Автоустановка cortiq (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Автообновление cortiq",
|
||||
"settings.cmf.model": "Путь к локальной .cmf-модели",
|
||||
"settings.cmf.port": "Порт локального сервера",
|
||||
"settings.cmf.modelId": "ID модели в пуле",
|
||||
"settings.cmf.note": "Обслуживает локальный .cmf через cortiq (github.com/infosave2007/cmf). Работает даже когда роутер недоступен.",
|
||||
"settings.cmf.threads": "Рабочие потоки",
|
||||
"settings.cmf.gpu": "Использовать GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Потоки распараллеливают декод-матвеки (~×3; 0 = по умолчанию рантайма). GPU-оффлоад помогает в основном на prefill длинных промптов — декод упирается в пропускную способность памяти, поэтому на Apple Silicon почти нейтрален.",
|
||||
"settings.cmf.pick": "выберите модель",
|
||||
"settings.cmf.models": "Локальные модели",
|
||||
"settings.cmf.modelsHelp": "Каждая строка — отдельная модель: шлюз поднимает свой cortiq serve на своём порту и регистрирует её в пуле. Добавляйте столько, сколько есть .cmf-файлов и свободных портов.",
|
||||
"settings.cmf.rowHint": "id · файл .cmf · порт · потоки · GPU. Изменения применяются после перезапуска; удаление строки останавливает сервер.",
|
||||
"settings.cmf.addServer": "+ Добавить модель",
|
||||
"settings.cmf.checkPort": "Проверить порт",
|
||||
"settings.cmf.portFree": "порт свободен",
|
||||
"settings.cmf.portBusy": "порт занят",
|
||||
"settings.cmf.useSuggested": "занять {port}",
|
||||
"settings.cmf.serving": "работает: {model} на :{port}",
|
||||
"settings.cmf.intro": "Запускайте модели локально движком cortiq — работает даже офлайн или когда роутер недоступен.",
|
||||
"settings.cmf.runtime": "Среда cortiq",
|
||||
"settings.cmf.runtimeHelp": "Инструмент командной строки `cortiq`, который загружает и запускает .cmf-модели. Установите один раз перед запуском локального сервера.",
|
||||
"settings.cmf.install": "Установить cortiq",
|
||||
"settings.cmf.reinstall": "Переустановить / обновить",
|
||||
"settings.cmf.installing": "устанавливается… (компилируется, пара минут)",
|
||||
"settings.cmf.installedOk": "cortiq установлен",
|
||||
"settings.cmf.notInstalled": "не установлен",
|
||||
"settings.cmf.addModel": "Добавить модель",
|
||||
"settings.cmf.addModelHelp": "Модели скачиваются и конвертируются во вкладке Импорт (HuggingFace → .cmf). Затем укажите путь к .cmf ниже.",
|
||||
"settings.cmf.importNote": "Ещё нет модели? Импортируйте:",
|
||||
"settings.cmf.importLink": "Открыть Импорт",
|
||||
"settings.cmf.modelHelp": "Путь к .cmf-файлу для запуска (напр. models/qwen.cmf), создаётся во вкладке Импорт. Несколько моделей под разные задачи? Импортируйте их и меняйте этот путь.",
|
||||
"settings.cmf.portHelp": "TCP-порт локального сервера моделей (по умолчанию 8081). Меняйте, только если конфликтует с другим сервисом.",
|
||||
"settings.cmf.threadsHelp": "Сколько ядер CPU использовать для генерации. Больше = быстрее (~×3 против 1 ядра); 0 — на усмотрение рантайма.",
|
||||
"settings.cmf.gpuHelp": "Выгрузить матричные операции на GPU (Metal/wgpu). На Apple Silicon ускоряет в основном длинные промпты — декод упирается в пропускную способность памяти.",
|
||||
"settings.cmf.localOnlyHelp": "Направлять все запросы в локальную модель, игнорируя роутер и облачные провайдеры.",
|
||||
"settings.cmf.manageHelp": "Шлюз запускает `cortiq serve` для .cmf ниже при старте и маршрутизирует в него. Нужен установленный cortiq (шаг 1). Применяется после перезапуска.",
|
||||
"settings.cmf.autoInstallHelp": "Если cortiq не найден при старте — установить автоматически с crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "При старте обновлять cortiq до новейшей версии, когда есть новее.",
|
||||
"settings.router.enabledHelp": "Использовать cortiq-router для классификации запроса (тип задачи + сложность) и выбора модели. Выкл = только локальная таблица маршрутизации.",
|
||||
"settings.router.verifyTlsHelp": "Проверять TLS-сертификат роутера. Выключайте только для самоподписанного / привязанного к IP сертификата.",
|
||||
"settings.telemetry.metricsHelp": "Отдавать метрики Prometheus по GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Добавлять поле `cortiq` (задача, модель, стоимость) в тело каждого ответа. Заголовки ответа шлются всегда.",
|
||||
"settings.stats.enabledHelp": "Записывать недавние запросы для дашборда и графиков.",
|
||||
"settings.cache.enabledHelp": "Семантический кэш: возвращать сохранённый ответ, когда новый промпт очень похож на прошлый, пропуская вызов модели. Нужна embedding-модель в пуле.",
|
||||
"settings.log.bodiesHelp": "Логировать полные тела запросов/ответов. По умолчанию выкл ради приватности.",
|
||||
"settings.router.taxonomy": "ID таксономии",
|
||||
"settings.router.getKey": "Получить или пополнить ключ роутера:",
|
||||
"settings.router.secret": "Ключ роутера (значение)",
|
||||
"settings.router.saveKey": "Сохранить ключ",
|
||||
"settings.router.keySaved": "Ключ роутера сохранён",
|
||||
"settings.router.keyEmpty": "Сначала вставьте ключ",
|
||||
"settings.router.secretHelp": "Вставьте ключ из письма с api.allaigate.com и нажмите «Сохранить ключ». Это НЕ ключ из раздела «Ключи» — те выдаются клиентам для доступа к вашему шлюзу.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — ключ сохранён (вставьте снова, чтобы заменить)",
|
||||
"settings.router.test": "Проверить роутер",
|
||||
"router.probe.ok": "Роутер работает ({ms} мс)",
|
||||
"router.probe.noKey": "Ключ роутера не задан",
|
||||
"router.probe.auth": "Ключ отклонён — недействителен или отозван",
|
||||
"router.probe.payment": "Требуется оплата — тариф истёк или квота исчерпана",
|
||||
"router.probe.quota": "Превышен лимит запросов",
|
||||
"router.probe.timeout": "Роутер не ответил вовремя",
|
||||
"router.probe.unreachable": "Роутер недоступен",
|
||||
"router.probe.error": "Ошибка роутера",
|
||||
"router.usage": "Использование",
|
||||
"settings.route": "Входные данные роутинга",
|
||||
"settings.route.strategy": "Стратегия текста",
|
||||
"settings.route.maxChars": "Макс. символов",
|
||||
"settings.route.cacheTtl": "TTL кэша",
|
||||
"settings.route.profile": "Профиль по умолчанию",
|
||||
"settings.route.strategyHelp": "Какой текст отправляется роутеру для классификации: last_user — только последнее сообщение пользователя; last_user_plus_system — последнее сообщение + системный промпт; concat_all — вся переписка.",
|
||||
"settings.route.maxCharsHelp": "Максимум символов этого текста, отправляемого роутеру (длиннее — обрезается). Больше = точнее классификация, но медленнее/дороже вызовы роутера.",
|
||||
"settings.route.cacheTtlHelp": "Сколько кэшируется решение роутинга для одинакового запроса, прежде чем роутер спрашивается снова (напр. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Профиль роутинга по умолчанию, если запрос его не задаёт: cost-saver (дешевле), balanced (баланс) или quality-first (качество). Влияет на выбор модели роутером.",
|
||||
"settings.breaker": "Автоматический выключатель",
|
||||
"settings.breaker.threshold": "Порог ошибок",
|
||||
"settings.breaker.cooldown": "Время восстановления",
|
||||
@@ -265,4 +369,277 @@ export default {
|
||||
"settings.cache.ttl": "TTL записи",
|
||||
"settings.cache.embed": "Модель эмбеддингов",
|
||||
"settings.cache.note": "Настройки кэша применяются после перезапуска.",
|
||||
|
||||
// импорт (HuggingFace → CMF)
|
||||
"import.subtitle": "Найдите модель на HuggingFace, сконвертируйте в локальный .cmf и подключите.",
|
||||
"import.searchPlaceholder": "Поиск моделей на HuggingFace (напр. qwen3, llama)…",
|
||||
"import.searching": "Ищу…",
|
||||
"import.noResults": "Модели не найдены.",
|
||||
"import.configure": "Настройка конвертации",
|
||||
"import.outName": "Имя файла",
|
||||
"import.outNameHint": "Имя .cmf (буквы, цифры, - и _).",
|
||||
"import.quant": "Квантизация",
|
||||
"import.convert": "Сконвертировать и загрузить",
|
||||
"import.converting": "Конвертация",
|
||||
"import.started": "Конвертация запущена",
|
||||
"import.failed": "Конвертация не удалась — смотрите лог.",
|
||||
"import.register": "Подключить как локальную модель",
|
||||
"import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
|
||||
"import.jobs": "Загрузки / Конвертации",
|
||||
"import.cancel": "Отменить",
|
||||
"import.cancelling": "Отмена — удаляю недописанный файл…",
|
||||
"import.delete": "Удалить",
|
||||
"import.deleteConfirm": "Удалить эту модель и сконвертированный .cmf-файл с диска?",
|
||||
"import.deleted": "Удалено вместе с файлами",
|
||||
"import.state.running": "идёт",
|
||||
"import.state.done": "готово",
|
||||
"import.state.error": "ошибка",
|
||||
"import.state.cancelled": "отменено",
|
||||
"import.featuredTitle": "Рекомендуемые готовые .cmf модели",
|
||||
"import.featuredSub": "Готовые .cmf модели из аккаунта infosave на HuggingFace — список читается вживую, новые модели появляются здесь сами.",
|
||||
"import.readyCmfBadge": "Готовая .cmf",
|
||||
"import.directDownload": "Скачать",
|
||||
"import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
|
||||
"import.q.q8_row": "8-бит построчно — просто, надёжно.",
|
||||
"import.q.q4": "4-бит блочно — компактнее, ниже качество.",
|
||||
"import.q.q4_tiled": "4-бит тайловый (Q4_TILED) — х1.66 быстрее на Metal GPU.",
|
||||
"import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
|
||||
"import.q.q1t": "1.58-бит тернарный (Q1T) — BitNet b1.58 с оутлайерами.",
|
||||
"import.q.q1s": "1-бит спарс (Q1S) — 1-бит база с голографическими оутлайерами.",
|
||||
"import.q.q1p": "1-бит PTQ (Q1P) — 1-бит с переносом ошибки (error diffusion).",
|
||||
"import.q.q1": "1-бит (Q1) — чистый бинарный формат.",
|
||||
"import.q.f16": "16-бит — без квантования.",
|
||||
"import.q.f32": "32-бит — полная точность (большой файл).",
|
||||
|
||||
"import.adv.title": "Расширенные настройки",
|
||||
"import.adv.auto": "авто",
|
||||
"import.adv.linearCore": "Линейное ядро",
|
||||
"import.adv.linearHint": "Как переносятся/сворачиваются линейные слои внимания.",
|
||||
"import.adv.nphase": "vmf_phase фаз/голову",
|
||||
"import.adv.vbitShape": "Форма бюджета VBIT",
|
||||
"import.adv.meanBits": "VBIT средних бит",
|
||||
"import.adv.shard": "Шард макс (ГБ)",
|
||||
"import.adv.noShard": "без шардинга",
|
||||
"import.adv.skipMtp": "пропустить MTP-голову",
|
||||
"import.adv.o1": "O(1)-внимание",
|
||||
"import.adv.o1Hint": "Потоковое внимание с постоянной памятью: KV-кэш освобождается после prefill, длинный контекст не раздувает память. Веса не меняются — записывается лишь подсказка на загрузку; вернуться можно всегда (запуск с --o1 off). Проверено на Qwen3-0.6B: все слои ≈ ×1.18 перплексии.",
|
||||
"import.adv.o1Off": "выкл — точное внимание (по умолчанию)",
|
||||
"import.adv.o1All": "all — конвертировать все softmax-слои",
|
||||
"import.adv.o1Deep": "deep N — только N самых глубоких слоёв",
|
||||
"import.adv.o1Custom": "вручную — явный список слоёв",
|
||||
"import.adv.o1DeepN": "N глубоких слоёв",
|
||||
"import.adv.o1Layers": "Слои (через запятую)",
|
||||
"import.adv.o1mHint": "число опорных точек (по умолчанию 32)",
|
||||
"import.adv.o1windowHint": "ширина точного окна (по умолчанию 128)",
|
||||
"import.adv.o1sinkHint": "постоянные точные sink-ключи (по умолчанию 4)",
|
||||
|
||||
// онбординг (первый запуск)
|
||||
"onboarding.title": "Первоначальная настройка",
|
||||
"onboarding.welcome": "Добро пожаловать в Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Выберите язык — его можно сменить в шапке в любой момент.",
|
||||
"onboarding.next": "Далее",
|
||||
"onboarding.back": "Назад",
|
||||
"onboarding.skip": "Пропустить",
|
||||
"onboarding.skipSetup": "Пропустить",
|
||||
"onboarding.finish": "Завершить",
|
||||
"onboarding.featuredTitle": "Готовые модели в один клик",
|
||||
"onboarding.featuredSub": "Это уже сконвертированные .cmf модели от infosave на HuggingFace — просто Скачать, без конвертации. Выберите одну или несколько для мгновенного старта.",
|
||||
"onboarding.download": "Скачать",
|
||||
"onboarding.hfTitle": "Любая модель с HuggingFace — автоконвертация",
|
||||
"onboarding.hfSub": "Найдите любую модель, выберите квантизацию — мы автоматически сконвертируем её в .cmf. Никаких ручных правок конфига.",
|
||||
"onboarding.noJobs": "Пока нет загрузок.",
|
||||
"onboarding.doneTitle": "Готово",
|
||||
"onboarding.doneSub": "Скачанные модели зарегистрируются автоматически. Добавить ещё можно во вкладке Импорт или Модели.",
|
||||
"onboarding.doneHint": "Кнопка «Завершить» подключит все скачанные модели как локальные, поднимет для них .cmf сервер и настроит роутинг. Всё это потом можно изменить во вкладке Модели.",
|
||||
"onboarding.willRegister": "Будут подключены как локальные модели",
|
||||
"onboarding.nothingToRegister": "Завершённых загрузок пока нет — подключать нечего. Вернитесь на шаг назад и выберите модель.",
|
||||
"onboarding.registeredN": "Подключено моделей: {n} — {ids}",
|
||||
"onboarding.runtimeMissing": "Среда cortiq не установлена — установите её в Настройки → Локальные модели, чтобы запускать эти модели.",
|
||||
"onboarding.downloaded": "скачана",
|
||||
"onboarding.update": "Обновить",
|
||||
"onboarding.deleteFile": "Удалить файл .cmf с диска",
|
||||
"onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.",
|
||||
"onboarding.unregistered": "Также отключено из моделей: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Чат",
|
||||
"pg.tab.image": "Изображение",
|
||||
"pg.tab.video": "Видео",
|
||||
"pg.generate": "Сгенерировать",
|
||||
"pg.generating": "Генерация…",
|
||||
"pg.genFailed": "Генерация не удалась — смотрите лог.",
|
||||
"pg.result": "Результат",
|
||||
"pg.genMetrics": "Метрики генерации",
|
||||
"pg.imagePromptPlaceholder": "рыжая лиса в снегу на закате, фотореалистично, детальный мех",
|
||||
"pg.videoPromptPlaceholder": "корги в поварском колпаке подбрасывает блин, шипение сковороды и весёлый лай",
|
||||
"pg.imageEmpty": "Сгенерированное изображение появится здесь.",
|
||||
"pg.videoEmpty": "Сгенерированный ролик (со звуком) появится здесь.",
|
||||
"pg.width": "Ширина",
|
||||
"pg.height": "Высота",
|
||||
"pg.steps": "Шаги",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Случайный seed",
|
||||
"pg.cfgHint": "Сила guidance; ≤ 1 отключает CFG и вдвое ускоряет генерацию.",
|
||||
"pg.frames": "Кадры",
|
||||
"pg.framesHint": "При 24 fps; округляется до сетки модели.",
|
||||
"pg.quality": "Качество JPEG",
|
||||
"pg.videoSizeHint": "Кратно 32; обученная короткая сторона — 768.",
|
||||
"pg.videoStepsHint": "Turbo LoRA обучена на 4 шага.",
|
||||
"pg.firstFrame": "Первый кадр (необязательно)",
|
||||
"pg.lastFrame": "Последний кадр (необязательно)",
|
||||
"pg.keyframeHint": "Превращает запуск в keyframe-to-video.",
|
||||
"pg.resolution": "Разрешение",
|
||||
"pg.genTime": "Время генерации",
|
||||
"pg.timePerStep": "На шаг",
|
||||
"pg.timePerFrame": "На кадр",
|
||||
"pg.fileSize": "Размер файла",
|
||||
"pg.duration": "Длительность",
|
||||
"pg.play": "Играть",
|
||||
"pg.noMediaModels": "Локальной модели ({kind}) ещё нет. Скачайте её (например, из коллекции infosave) во вкладке Импорт:",
|
||||
"pg.tokSpeed": "Скорость",
|
||||
"pg.ttft": "Первый токен",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Логи",
|
||||
"logs.subtitle": "Последние записи журнала шлюза — то, что показал бы терминал.",
|
||||
"logs.filterPlaceholder": "фильтр…",
|
||||
"logs.pause": "Пауза",
|
||||
"logs.resume": "Продолжить",
|
||||
"logs.empty": "Ничего не найдено.",
|
||||
"notice.openAdmin": "Панель управления открыта без токена и доступна с других машин. Задайте токен, если эта машина не приватная.",
|
||||
"notice.update": "Доступно обновление: v{v}.",
|
||||
"dash.sys.title": "Система",
|
||||
"dash.sys.cores": "ядер",
|
||||
"dash.sys.disk": "Диск (модели)",
|
||||
"dash.sys.free": "свободно",
|
||||
"dash.sys.gateway": "шлюз",
|
||||
"fit.disk": "не хватит места",
|
||||
"fit.diskTitle": "Загрузка больше свободного места на диске с моделями.",
|
||||
"fit.ram": "может не влезть в RAM",
|
||||
"fit.ramTitle": "Модель сравнима с объёмом RAM этой машины или больше — может не запуститься.",
|
||||
"pg.eta": "осталось",
|
||||
"pg.min": "мин",
|
||||
"pg.queued": "В очереди — позиция {n}. Стартует, как только закончится текущая генерация.",
|
||||
"pg.gallery": "Последние генерации",
|
||||
"pg.galDeleteConfirm": "Удалить эту генерацию и её файлы?",
|
||||
"pg.clearChat": "Очистить",
|
||||
"keys.snip.title": "Подключите ваше приложение",
|
||||
"keys.snip.hint": "Вставьте один из примеров в свой проект — заменить нужно только плейсхолдер ключа.",
|
||||
"keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
|
||||
"keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
|
||||
"keys.snip.keyPh": "ваш API-ключ",
|
||||
"settings.restart": "Перезапустить всё",
|
||||
"settings.restartConfirm": "Перезапустить шлюз и все серверы локальных моделей? Текущие запросы оборвутся.",
|
||||
"settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
|
||||
"settings.shutdown": "Остановить всё и выключить",
|
||||
"settings.shutdownConfirm": "Остановить все серверы локальных моделей и выключить шлюз? API и эта панель перестанут отвечать, пока вы не запустите шлюз заново.",
|
||||
"settings.shutdownDone": "Шлюз выключен.",
|
||||
"settings.shutdownHint": "Все серверы локальных моделей остановлены, процесс завершён. Запустите шлюз на сервере заново, чтобы вернуть панель.",
|
||||
"bench.title": "Скорость локальных моделей",
|
||||
"bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
|
||||
"bench.run": "Замерить",
|
||||
"bench.running": "замер…",
|
||||
"bench.failed": "ошибка",
|
||||
"bench.none": "Локальных чат-моделей .cmf пока нет.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Светская беседа",
|
||||
"task.code": "Код",
|
||||
"task.creative-writing": "Творческие тексты",
|
||||
"task.extraction": "Извлечение данных",
|
||||
"task.math": "Математика",
|
||||
"task.qa": "Вопросы и ответы",
|
||||
"task.summarization": "Суммаризация",
|
||||
"task.translation": "Перевод",
|
||||
"routing.tasks.title": "Распределение по задачам",
|
||||
"routing.tasks.hint": "Модели правила идут первыми для своей задачи; цепочка по сложности остаётся резервом. Пусто = только по сложности.",
|
||||
"routing.tasks.inherit": "по сложности (без правила)",
|
||||
"routing.tasks.share": "{p}% запросов (7д)",
|
||||
"routing.tasks.fromRouter": "метки получены с роутера",
|
||||
"routing.tasks.builtin": "роутер недоступен — встроенный набор меток",
|
||||
"routing.tasks.addCustom": "Добавить задачу",
|
||||
"routing.tasks.addPlaceholder": "своя метка…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Бенчмарк",
|
||||
"bm.title": "Бенчмарк",
|
||||
"bm.subtitle": "Проверки качества и скорости через живой пайплайн: ответы сверяются по ключевым словам, ловится зацикливание, замеряются tok/s и время первого токена.",
|
||||
"bm.models": "Какие модели тестировать",
|
||||
"bm.quickMode": "Быстрый тест",
|
||||
"bm.quickHint": "Короткое подмножество с краткими ответами — около минуты на локальную модель.",
|
||||
"bm.suiteInfo": "Встроенный набор: {q} быстрых / {f} полных тестов, плюс ваши собственные.",
|
||||
"bm.run": "Запустить",
|
||||
"bm.pickModel": "Выберите хотя бы одну модель.",
|
||||
"bm.testOf": "тест {i} из {n}",
|
||||
"bm.results": "Результаты",
|
||||
"bm.quick": "быстрый",
|
||||
"bm.test": "Тест",
|
||||
"bm.cellHint": "Клик по ячейке — полный ответ модели.",
|
||||
"bm.history": "Прошлые прогоны (хранятся в этом браузере)",
|
||||
"bm.custom": "Свои тесты",
|
||||
"bm.customHint": "Промпт + ключевые слова через запятую, которые должны быть в ответе (| — синонимы). Хранятся в этом браузере.",
|
||||
"bm.promptPh": "промпт…",
|
||||
"bm.checkPh": "ключевые слова через запятую",
|
||||
"bm.addTest": "Добавить тест",
|
||||
"bm.emptyAnswer": "пустой ответ",
|
||||
"bm.loopDetected": "зацикливание",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Умный роутинг",
|
||||
"bm.mode.models": "Отдельные модели",
|
||||
"bm.mode.routingDesc": "Проверка главной фишки продукта: каждый тест идёт через cortiq-auto — видно, какую задачу распознал роутер, какую модель выбрал, совпало ли это с ожиданием и сколько это стоило.",
|
||||
"bm.mode.modelsDesc": "Тот же набор тестов, но закреплённый за выбранными моделями — сравнение их качества, скорости и стоимости напрямую, мимо роутера.",
|
||||
"bm.answersOk": "ответы верны",
|
||||
"bm.classAcc": "классификация задач",
|
||||
"bm.modelDist": "Запросы ушли в",
|
||||
"bm.expectedLbl": "ожидалось:",
|
||||
"bm.answer": "Ответ",
|
||||
"bm.liveRunning": "идёт…",
|
||||
"bm.spent": "потрачено",
|
||||
"bm.verdict": "🏁 Готово: ответы {p}, потрачено {s}",
|
||||
"bm.clearHistory": "Очистить всё",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — вставьте в терминал (шлюз говорит на протоколе Anthropic нативно):",
|
||||
"keys.h.codex": "Codex — добавит провайдера в ~/.codex/config.toml и запустит:",
|
||||
"keys.h.opencode": "OpenCode — запишет ~/.config/opencode/opencode.json (перезапишет!) и запустит:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — добавит модель в ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline (VS Code) — в настройках расширения укажите:",
|
||||
"keys.h.cline2": "Настройки → API Provider → OpenAI Compatible, затем вставьте значения выше.",
|
||||
"keys.h.note": "cortiq-auto — умный роутинг: роутер сам выбирает модель под каждый запрос. Чтобы закрепить конкретную, укажите её точное имя. На Windows запускайте скрипты в Git Bash / WSL или задайте те же переменные через настройки окружения системы.",
|
||||
"keys.h.protoOff": "Протокол Anthropic (/v1/messages) выключен — Claude Code не подключится.",
|
||||
"keys.h.enableProto": "Включить",
|
||||
"keys.h.st.installed": "Установлен на машине шлюза",
|
||||
"keys.h.st.configured": "Уже настроен на этот шлюз",
|
||||
"keys.h.st.notFound": "Не найден на машине шлюза",
|
||||
"bm.degraded": "роутер деградировал",
|
||||
"bm.routerDegraded": "Роутер не успел ответить — запрос ушёл по резервной цепочке; классификация не засчитывается.",
|
||||
"bm.cutByCap": "бюджет ушёл на размышления — ответ не успел",
|
||||
"shell.uiStale": "Шлюз обновился — на этой странице старый интерфейс.",
|
||||
"shell.reload": "Перезагрузить",
|
||||
"keys.col.requests": "Запросы",
|
||||
"keys.col.tokens": "Токены вх/исх",
|
||||
"keys.col.costAll": "$ за всё время",
|
||||
"routing.tasks.tier": "Сложность",
|
||||
"routing.tasks.tierAuto": "Авто (по роутеру)",
|
||||
"routing.tasks.tierHint": "Какая модель обслуживает задачу на какой сложности; «любая» ведёт на всех уровнях. Порядок = порядок фолбэка. Вердикт high по-прежнему эскалирует поверх «любых».",
|
||||
"routing.tasks.tierAny": "любая сложность",
|
||||
"routing.tasks.addModel": "модель",
|
||||
"routing.bands.enable": "Мои границы сложности",
|
||||
"routing.bands.hint": "Тяните ручки: где кончается low и где начинается high — ручки толкают друг друга, цифры обновляются на лету. Непрерывный score роутера пересчитывается ВАШИМИ границами; галочка снята — действуют пороги самого роутера.",
|
||||
"tier.low": "лёгкий",
|
||||
"tier.medium": "средний",
|
||||
"tier.high": "сложный",
|
||||
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
|
||||
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
|
||||
"settings.cache.entries": "записей",
|
||||
"settings.cache.hits": "попаданий",
|
||||
"settings.cache.saved": "сэкономлено",
|
||||
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
|
||||
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
|
||||
"keys.budgetOf": "бюджет {b}/мес",
|
||||
"dash.kpi.orBalance": "Баланс OpenRouter",
|
||||
"keys.h.install": "Установить:",
|
||||
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
|
||||
"keys.snip.apply": "Вставить ключ",
|
||||
};
|
||||
|
||||
Vendored
+385
@@ -33,6 +33,7 @@ export default {
|
||||
"nav.playground": "Oyun alanı",
|
||||
"nav.settings": "Ayarlar",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Router anahtarı ve ödeme",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "Yönlendirici",
|
||||
@@ -40,6 +41,7 @@ export default {
|
||||
"status.online": "çevrimiçi",
|
||||
"status.offline": "çevrimdışı",
|
||||
"status.checking": "kontrol ediliyor…",
|
||||
"status.noKey": "anahtar yok",
|
||||
"theme.toggle": "Tema",
|
||||
"logout": "Çıkış yap",
|
||||
|
||||
@@ -88,6 +90,7 @@ export default {
|
||||
"dash.health.title": "Sağlık",
|
||||
"dash.health.reachable": "erişilebilir",
|
||||
"dash.health.unreachable": "erişilemiyor",
|
||||
"dash.health.getKey": "Router anahtarı al",
|
||||
|
||||
// models
|
||||
"models.title": "Modeller",
|
||||
@@ -225,6 +228,18 @@ export default {
|
||||
"settings.router.timeout": "Zaman aşımı (ms)",
|
||||
"settings.router.verifyTls": "TLS sertifikasını doğrula",
|
||||
"settings.router.taxonomy": "Taksonomi id",
|
||||
"settings.router.getKey": "Router anahtarı almak veya bakiye yüklemek için:",
|
||||
"settings.router.secret": "Router anahtarı (değer)",
|
||||
"settings.router.test": "Router'ı test et",
|
||||
"router.probe.ok": "Router çalışıyor ({ms} ms)",
|
||||
"router.probe.noKey": "Router anahtarı ayarlanmadı",
|
||||
"router.probe.auth": "Anahtar reddedildi — geçersiz veya iptal edilmiş",
|
||||
"router.probe.payment": "Ödeme gerekli — plan süresi doldu veya kota tükendi",
|
||||
"router.probe.quota": "Hız sınırı aşıldı",
|
||||
"router.probe.timeout": "Router zaman aşımına uğradı",
|
||||
"router.probe.unreachable": "Router erişilemiyor",
|
||||
"router.probe.error": "Router hatası",
|
||||
"router.usage": "Kullanım",
|
||||
"settings.route": "Yönlendirme girişi",
|
||||
"settings.route.strategy": "Metin stratejisi",
|
||||
"settings.route.maxChars": "Maks. karakter",
|
||||
@@ -265,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "Giriş TTL",
|
||||
"settings.cache.embed": "Gömme modeli",
|
||||
"settings.cache.note": "Anlamsal önbellek ayarları yeniden başlatmadan sonra geçerli olur.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Model içe aktar",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Günlükleri temizle",
|
||||
"dash.clearConfirm": "Kaydedilmiş tüm istatistikler ve günlükler silinsin mi? Bu işlem geri alınamaz.",
|
||||
"dash.cleared": "Günlükler temizlendi",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Bir id ya da model adı girin",
|
||||
"models.form.apiKey": "API anahtarı",
|
||||
"models.form.apiKeyPlaceholder": "API anahtarınızı yapıştırın",
|
||||
"models.form.apiKeyNotNeeded": "gerekmiyor (yerel sunucu)",
|
||||
"models.form.apiKeyHint": "Ağ geçidinde güvenle saklanır; bir daha gösterilmez.",
|
||||
"models.form.keyStored": "Kayıtlı bir anahtar var — korumak için boş bırakın ya da yenisini yapıştırın.",
|
||||
"models.form.keyFromEnv": "Ortam değişkenindeki anahtar kullanılıyor.",
|
||||
"models.form.paramsTitle": "Üretim ve düşünme parametreleri (O(1), MTP, sıcaklık)",
|
||||
"models.form.temperature": "Sıcaklık",
|
||||
"models.form.temperatureHint": "Üretim sıcaklığı (0.0 = belirlenimci, 1.0+ = yaratıcı).",
|
||||
"models.form.topP": "Top-P (nucleus örnekleme)",
|
||||
"models.form.topPHint": "Örnekleme eşiği (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Maks. token",
|
||||
"models.form.maxTokensHint": "Üretilecek en fazla token sayısı.",
|
||||
"models.form.thinkBudget": "Düşünme modu / düşünme bütçesi",
|
||||
"models.form.thinkBudgetHint": "Akıl yürütme token bütçesi (0 = kapalı, N > 0 = en fazla akıl yürütme tokeni).",
|
||||
"models.form.systemPrompt": "Varsayılan sistem istemi",
|
||||
"models.form.systemPromptHint": "İstekte sistem istemi yoksa otomatik olarak başa eklenir.",
|
||||
"models.form.systemPromptPlaceholder": "Yardımsever bir asistansın…",
|
||||
"models.form.testKey": "Test et ve modelleri yükle",
|
||||
"models.form.testing": "kontrol ediliyor…",
|
||||
"models.form.foundModels": "{n} model yüklendi",
|
||||
"models.form.modelPlaceholder": "aramak için yazın…",
|
||||
"models.form.modelHint": "Anahtarınızla listeyi çekmek için “Test et ve modelleri yükle”ye tıklayın, sonra birini seçin.",
|
||||
"models.managed": "yönetilen",
|
||||
"models.managedHint": "Yönetilen CMF sunucusu tarafından sağlanır (Ayarlar → Yerel modeller). Yapılandırmayı orada yapın.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Yönlendiriciyi kullan (kapalı = yalnızca yerel modeller)",
|
||||
"settings.cmf": "Yerel modeller (CMF)",
|
||||
"settings.cmf.localOnly": "Yalnızca yerel modelleri kullan",
|
||||
"settings.cmf.manage": "Yerel model sunucusu çalıştır",
|
||||
"settings.cmf.autoInstall": "cortiq'i otomatik kur (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq'i otomatik güncelle",
|
||||
"settings.cmf.model": "Yerel .cmf model yolu",
|
||||
"settings.cmf.port": "Yerel sunucu portu",
|
||||
"settings.cmf.modelId": "Havuzdaki model id'si",
|
||||
"settings.cmf.note": "Yerel bir .cmf dosyasını cortiq ile sunar (github.com/infosave2007/cmf). Yönlendirici erişilemez olsa bile çalışır.",
|
||||
"settings.cmf.threads": "İş parçacıkları",
|
||||
"settings.cmf.gpu": "GPU kullan (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "İş parçacıkları çözme aşamasındaki matris-vektör çarpımlarını paralelleştirir (~3× hızlı; 0 = çalışma zamanı varsayılanı). GPU'ya aktarım daha çok uzun istemlerin prefill aşamasına yarar — çözme bellek bant genişliğine bağlı olduğundan Apple Silicon'da etkisi yaklaşık nötrdür.",
|
||||
"settings.cmf.pick": "bir model seçin",
|
||||
"settings.cmf.models": "Yerel modeller",
|
||||
"settings.cmf.modelsHelp": "Her satır, ağ geçidinin çalıştırdığı (kendi portunda kendi cortiq serve süreci olan) ve havuza kaydettiği bir modeldir. Kaç .cmf dosyanız ve boş portunuz varsa o kadar ekleyebilirsiniz.",
|
||||
"settings.cmf.rowHint": "id · .cmf dosyası · port · iş parçacığı · GPU. Değişiklikler yeniden başlatınca geçerli olur; satırı silmek o sunucuyu durdurur.",
|
||||
"settings.cmf.addServer": "+ Yerel model ekle",
|
||||
"settings.cmf.checkPort": "Portu kontrol et",
|
||||
"settings.cmf.portFree": "port boş",
|
||||
"settings.cmf.portBusy": "port kullanımda",
|
||||
"settings.cmf.useSuggested": "{port} kullan",
|
||||
"settings.cmf.serving": "{model} :{port} üzerinde sunuluyor",
|
||||
"settings.cmf.intro": "Modelleri cortiq motoruyla yerelde çalıştırın — çevrimdışıyken ya da yönlendirici çöktüğünde bile çalışır.",
|
||||
"settings.cmf.runtime": "cortiq çalışma zamanı",
|
||||
"settings.cmf.runtimeHelp": ".cmf modellerini yükleyip çalıştıran `cortiq` komut satırı aracı. Yerel sunucuyu başlatmadan önce bir kez kurun.",
|
||||
"settings.cmf.install": "cortiq'i kur",
|
||||
"settings.cmf.reinstall": "Yeniden kur / güncelle",
|
||||
"settings.cmf.installing": "kuruluyor… (derleniyor, birkaç dakika)",
|
||||
"settings.cmf.installedOk": "cortiq kuruldu",
|
||||
"settings.cmf.notInstalled": "kurulu değil",
|
||||
"settings.cmf.addModel": "Model ekle",
|
||||
"settings.cmf.addModelHelp": "Modeller “Model içe aktar” sekmesinde indirilip dönüştürülür (HuggingFace → .cmf). Sonra aşağıdaki yerel sunucuyu o .cmf dosyasına yönlendirin.",
|
||||
"settings.cmf.importNote": "Henüz model yok mu? Bir tane içe aktarın:",
|
||||
"settings.cmf.importLink": "İçe aktarmayı aç",
|
||||
"settings.cmf.modelHelp": "Sunulacak .cmf dosyasının yolu (ör. models/qwen.cmf), içe aktarma sekmesinde üretilir. Farklı görevler için birden çok model mi? Hepsini içe aktarıp bu yolu değiştirin.",
|
||||
"settings.cmf.portHelp": "Yerel model sunucusunun dinlediği TCP portu (varsayılan 8081). Yalnızca başka bir servisle çakışırsa değiştirin.",
|
||||
"settings.cmf.threadsHelp": "Üretim için kaç CPU çekirdeği kullanılacağı. Daha fazlası = daha hızlı (tek çekirdeğe göre ~3×); 0 seçimi çalışma zamanına bırakır.",
|
||||
"settings.cmf.gpuHelp": "Matris hesaplarını GPU'ya aktarır (Metal/wgpu). Apple Silicon'da çoğunlukla uzun istemleri hızlandırır — çözme bellek bant genişliğine bağlıdır.",
|
||||
"settings.cmf.localOnlyHelp": "Her isteği yerel modele yönlendirir; yönlendiriciyi ve tüm bulut sağlayıcılarını yok sayar.",
|
||||
"settings.cmf.manageHelp": "Ağ geçidi açılışta aşağıdaki .cmf için `cortiq serve` başlatır ve istekleri oraya yönlendirir. cortiq'in kurulu olması gerekir (adım 1). Yeniden başlatınca geçerli olur.",
|
||||
"settings.cmf.autoInstallHelp": "Açılışta cortiq yoksa crates.io üzerinden otomatik kurar.",
|
||||
"settings.cmf.autoUpdateHelp": "Açılışta daha yeni bir sürüm varsa cortiq'i en son sürüme yükseltir.",
|
||||
"settings.router.enabledHelp": "Her isteği cortiq yönlendiricisi sınıflandırsın (görev türü + karmaşıklık) ve model seçsin. Kapalı = yalnızca yerel yönlendirme tablosunu kullan.",
|
||||
"settings.router.verifyTlsHelp": "Yönlendiricinin TLS sertifikasını doğrula. Yalnızca kendinden imzalı ya da IP'ye bağlı sertifikada kapatın.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus metriklerini GET /metrics adresinde yayınla.",
|
||||
"settings.cortiq.echoHelp": "Her yanıt gövdesine `cortiq` alanı ekle (görev, model, maliyet). Yanıt başlıkları her zaman gönderilir.",
|
||||
"settings.stats.enabledHelp": "Panoyu ve grafikleri beslemek için son istekleri kaydet.",
|
||||
"settings.cache.enabledHelp": "Anlamsal önbellek: yeni bir istem geçmiştekine çok benziyorsa kaydedilmiş yanıtı döndürür ve model çağrısını atlar. Havuzda bir gömme (embedding) modeli gerekir.",
|
||||
"settings.log.bodiesHelp": "İstek/yanıt mesaj gövdelerinin tamamını günlüğe yaz. Gizlilik için varsayılan olarak kapalı.",
|
||||
"settings.router.saveKey": "Anahtarı kaydet",
|
||||
"settings.router.keySaved": "Yönlendirici anahtarı kaydedildi",
|
||||
"settings.router.keyEmpty": "Önce anahtarı yapıştırın",
|
||||
"settings.router.secretHelp": "api.allaigate.com e-postanızdaki anahtarı yapıştırıp Anahtarı kaydet'e basın. Bu bir “API anahtarları” değeri DEĞİLDİR — onlar ağ geçidinizi çağıran istemciler içindir.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — anahtar kayıtlı (değiştirmek için yenisini yapıştırın)",
|
||||
"settings.route.strategyHelp": "Sınıflandırma için yönlendiriciye gönderilen metin: last_user = yalnızca son kullanıcı mesajı; last_user_plus_system = son kullanıcı mesajı + sistem istemi; concat_all = tüm konuşma.",
|
||||
"settings.route.maxCharsHelp": "Bu metnin yönlendiriciye gönderilen en fazla karakter sayısı (fazlası kırpılır). Daha fazlası = daha isabetli sınıflandırma, ama daha yavaş ve pahalı yönlendirici çağrıları.",
|
||||
"settings.route.cacheTtlHelp": "Aynı istek için yönlendirme kararının, yönlendiriciye yeniden sorulmadan önce ne kadar önbellekte tutulacağı (ör. 60s, 5m).",
|
||||
"settings.route.profileHelp": "İstek belirtmediğinde kullanılacak varsayılan yönlendirme profili: cost-saver (en ucuz), balanced ya da quality-first (en iyi model). Yönlendiricinin hangi modeli seçeceğini etkiler.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "HuggingFace'te arayın, yerel bir .cmf dosyasına dönüştürün ve sunun.",
|
||||
"import.searchPlaceholder": "HuggingFace modellerinde ara (ör. qwen3, llama)…",
|
||||
"import.searching": "Aranıyor…",
|
||||
"import.noResults": "Model bulunamadı.",
|
||||
"import.configure": "Dönüştürmeyi yapılandır",
|
||||
"import.outName": "Çıktı adı",
|
||||
"import.outNameHint": ".cmf dosya adı (harf, rakam, - ve _).",
|
||||
"import.quant": "Niceleme",
|
||||
"import.convert": "Dönüştür ve yükle",
|
||||
"import.converting": "Dönüştürülüyor",
|
||||
"import.started": "Dönüştürme başladı",
|
||||
"import.failed": "Dönüştürme başarısız — günlüğe bakın.",
|
||||
"import.register": "Yerel model olarak kaydet",
|
||||
"import.registered": "“{id}” olarak kaydedildi. Sunmak için cortiq-server'ı çalıştırın.",
|
||||
"import.jobs": "İndirmeler ve dönüştürmeler",
|
||||
"import.cancel": "İptal",
|
||||
"import.cancelling": "İptal ediliyor — yarım kalan dosya temizleniyor…",
|
||||
"import.delete": "Sil",
|
||||
"import.deleteConfirm": "Bu model ve dönüştürülmüş .cmf dosyası diskten silinsin mi?",
|
||||
"import.deleted": "Dosyalarıyla birlikte silindi",
|
||||
"import.state.running": "çalışıyor",
|
||||
"import.state.done": "tamam",
|
||||
"import.state.error": "hata",
|
||||
"import.state.cancelled": "iptal edildi",
|
||||
"import.featuredTitle": "Öne çıkan hazır .cmf modelleri",
|
||||
"import.featuredSub": "HuggingFace'teki infosave hesabından hazır .cmf modelleri — liste canlı okunur, yeni modeller kendiliğinden görünür.",
|
||||
"import.readyCmfBadge": "Hazır .cmf",
|
||||
"import.directDownload": "İndir",
|
||||
"import.q.q8_2f": "8 bit çift alan (𝒲×θ) — önerilen: en iyi kalite/boyut dengesi.",
|
||||
"import.q.q8_row": "Satır başına 8 bit — basit ve sağlam.",
|
||||
"import.q.q4": "4 bit blok — en küçük, kalite daha düşük.",
|
||||
"import.q.q4_tiled": "4 bit döşeli (Q4_TILED) — Metal GPU'da 1,66× daha hızlı.",
|
||||
"import.q.vbit": "Değişken 3–8 bit — uzman başına bit bütçesi.",
|
||||
"import.q.q1t": "1,58 bit üçlü (Q1T) — seyrek aykırı değerlerle BitNet b1.58.",
|
||||
"import.q.q1s": "1 bit seyrek (Q1S) — holografik aykırı değerlerle 1 bit taban.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — hata yayılımlı 1 bit.",
|
||||
"import.q.q1": "1 bit (Q1) — saf ikili biçim.",
|
||||
"import.q.f16": "16 bit — niceleme yok.",
|
||||
"import.q.f32": "32 bit — tam hassasiyet (büyük dosya).",
|
||||
"import.adv.title": "Gelişmiş ayarlar",
|
||||
"import.adv.auto": "otomatik",
|
||||
"import.adv.linearCore": "Doğrusal çekirdek",
|
||||
"import.adv.linearHint": "Doğrusal dikkat katmanlarının nasıl taşındığı/katlandığı.",
|
||||
"import.adv.nphase": "vmf_phase faz/başlık",
|
||||
"import.adv.vbitShape": "VBIT bütçe biçimi",
|
||||
"import.adv.meanBits": "VBIT ortalama bit",
|
||||
"import.adv.shard": "Parça üst sınırı (GB)",
|
||||
"import.adv.noShard": "parçalama yok",
|
||||
"import.adv.skipMtp": "MTP başlığını atla",
|
||||
"import.adv.o1": "O(1) dikkat",
|
||||
"import.adv.o1Hint": "Sabit bellekli akış dikkati: KV önbelleği prefill sonrası serbest bırakılır, böylece uzun bağlamlar belleği büyütmez. Ağırlıklar bayt bayt aynı kalır — yalnızca yükleme anına dair bir ipucu kaydedilir; istediğiniz zaman geri dönebilirsiniz (--o1 off ile sunun). Qwen3-0.6B üzerinde doğrulandı: tüm katmanlar ≈ ×1,18 şaşkınlık (perplexity).",
|
||||
"import.adv.o1Off": "kapalı — tam dikkat (varsayılan)",
|
||||
"import.adv.o1All": "all — her softmax katmanını dönüştür",
|
||||
"import.adv.o1Deep": "deep N — yalnızca en derin N katman",
|
||||
"import.adv.o1Custom": "özel — açık katman listesi",
|
||||
"import.adv.o1DeepN": "N derin katman",
|
||||
"import.adv.o1Layers": "Katmanlar (virgülle ayrılmış)",
|
||||
"import.adv.o1mHint": "landmark bütçesi (varsayılan 32)",
|
||||
"import.adv.o1windowHint": "tam pencere genişliği (varsayılan 128)",
|
||||
"import.adv.o1sinkHint": "kalıcı tam sink anahtarları (varsayılan 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "İlk kurulum",
|
||||
"onboarding.welcome": "Cortiq Gateway'e hoş geldiniz",
|
||||
"onboarding.welcomeSub": "Dilinizi seçin — üst çubuktan istediğiniz zaman değiştirebilirsiniz.",
|
||||
"onboarding.next": "İleri",
|
||||
"onboarding.back": "Geri",
|
||||
"onboarding.skip": "Şimdilik atla",
|
||||
"onboarding.skipSetup": "Atla",
|
||||
"onboarding.finish": "Kurulumu tamamla",
|
||||
"onboarding.featuredTitle": "Hazır modeller (tek tık)",
|
||||
"onboarding.featuredSub": "Bunlar HuggingFace'te infosave tarafından yayınlanan, önceden dönüştürülmüş .cmf modelleridir — sadece indirin, dönüştürme gerekmez. Hemen başlamak için bir ya da birkaçını seçin.",
|
||||
"onboarding.download": "İndir",
|
||||
"onboarding.hfTitle": "Herhangi bir HuggingFace modeli — otomatik dönüştürme",
|
||||
"onboarding.hfSub": "İstediğiniz modeli arayın, bir niceleme seçin; biz onu otomatik olarak .cmf biçimine dönüştürelim. Elle yapılandırma düzenlemesi yok.",
|
||||
"onboarding.noJobs": "Henüz indirme yok.",
|
||||
"onboarding.doneTitle": "Her şey hazır",
|
||||
"onboarding.doneSub": "İndirdiğiniz modeller otomatik olarak kaydedilir. Daha fazlasını İçe aktar ya da Modeller sekmesinden ekleyebilirsiniz.",
|
||||
"onboarding.doneHint": "“Kurulumu tamamla”, biten her indirmeyi yerel model olarak bağlar, .cmf sunucusunu başlatır ve yönlendirmeyi ona çevirir. Tümünü sonradan Modeller sekmesinden değiştirebilirsiniz.",
|
||||
"onboarding.willRegister": "Yerel model olarak bağlanacak",
|
||||
"onboarding.nothingToRegister": "Henüz tamamlanmış indirme yok — bağlanacak bir şey bulunmuyor. Bir adım geri dönüp model seçin.",
|
||||
"onboarding.registeredN": "{n} model bağlandı: {ids}",
|
||||
"onboarding.runtimeMissing": "cortiq çalışma zamanı kurulu değil — bunları sunabilmek için Ayarlar → Yerel modeller bölümünden kurun.",
|
||||
"onboarding.downloaded": "indirildi",
|
||||
"onboarding.update": "Güncelle",
|
||||
"onboarding.deleteFile": ".cmf dosyasını diskten sil",
|
||||
"onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.",
|
||||
"onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Sohbet",
|
||||
"pg.tab.image": "Görsel",
|
||||
"pg.tab.video": "Video",
|
||||
"pg.generate": "Oluştur",
|
||||
"pg.generating": "Oluşturuluyor…",
|
||||
"pg.genFailed": "Oluşturma başarısız — günlüğe bakın.",
|
||||
"pg.result": "Sonuç",
|
||||
"pg.genMetrics": "Oluşturma metrikleri",
|
||||
"pg.imagePromptPlaceholder": "gün batımında karda oturan kızıl tilki, fotogerçekçi, ayrıntılı tüyler",
|
||||
"pg.videoPromptPlaceholder": "aşçı şapkalı bir corgi krep çeviriyor, cızırtı sesleri ve neşeli bir havlama",
|
||||
"pg.imageEmpty": "Oluşturulan görsel burada görünecek.",
|
||||
"pg.videoEmpty": "Oluşturulan klip (sesiyle) burada görünecek.",
|
||||
"pg.width": "Genişlik",
|
||||
"pg.height": "Yükseklik",
|
||||
"pg.steps": "Adım",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Rastgele seed",
|
||||
"pg.cfgHint": "Guidance gücü; ≤ 1 CFG'yi kapatır ve işi yarıya indirir.",
|
||||
"pg.frames": "Kare",
|
||||
"pg.framesHint": "24 fps'te; modelin ızgarasına yuvarlanır.",
|
||||
"pg.quality": "JPEG kalitesi",
|
||||
"pg.videoSizeHint": "32'nin katları; eğitilen kısa kenar 768.",
|
||||
"pg.videoStepsHint": "Turbo LoRA 4 adım için eğitildi.",
|
||||
"pg.firstFrame": "İlk kare (isteğe bağlı)",
|
||||
"pg.lastFrame": "Son kare (isteğe bağlı)",
|
||||
"pg.keyframeHint": "Çalışmayı keyframe-to-video'ya çevirir.",
|
||||
"pg.resolution": "Çözünürlük",
|
||||
"pg.genTime": "Oluşturma süresi",
|
||||
"pg.timePerStep": "Adım başına",
|
||||
"pg.timePerFrame": "Kare başına",
|
||||
"pg.fileSize": "Dosya boyutu",
|
||||
"pg.duration": "Süre",
|
||||
"pg.play": "Oynat",
|
||||
"pg.noMediaModels": "Henüz yerel bir {kind} modeli yok. İçe aktarma sekmesinden bir tane indirin (ör. infosave koleksiyonundan):",
|
||||
"pg.tokSpeed": "Hız",
|
||||
"pg.ttft": "İlk token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Günlükler",
|
||||
"logs.subtitle": "Ağ geçidinin son günlüğü — terminalde görecekleriniz.",
|
||||
"logs.filterPlaceholder": "filtrele…",
|
||||
"logs.pause": "Duraklat",
|
||||
"logs.resume": "Devam et",
|
||||
"logs.empty": "Eşleşen yok.",
|
||||
"notice.openAdmin": "Yönetim konsolu belirteçsiz açık ve diğer makinelerden erişilebilir. Bu makine özel değilse bir belirteç ayarlayın.",
|
||||
"notice.update": "Güncelleme mevcut: v{v}.",
|
||||
"dash.sys.title": "Sistem",
|
||||
"dash.sys.cores": "çekirdek",
|
||||
"dash.sys.disk": "Disk (modeller)",
|
||||
"dash.sys.free": "boş",
|
||||
"dash.sys.gateway": "ağ geçidi",
|
||||
"fit.disk": "disk alanı yetmez",
|
||||
"fit.diskTitle": "İndirme, model diskindeki boş alandan büyük.",
|
||||
"fit.ram": "RAM'e sığmayabilir",
|
||||
"fit.ramTitle": "Model bu makinenin RAM'ine yakın ya da ondan büyük — çalışmayabilir.",
|
||||
"pg.eta": "kaldı",
|
||||
"pg.min": "dk",
|
||||
"pg.queued": "Kuyrukta — {n}. sırada. Mevcut üretim biter bitmez başlar.",
|
||||
"pg.gallery": "Son üretimler",
|
||||
"pg.galDeleteConfirm": "Bu üretim ve dosyaları silinsin mi?",
|
||||
"pg.clearChat": "Temizle",
|
||||
"keys.snip.title": "Uygulamanı bağla",
|
||||
"keys.snip.hint": "Örneklerden birini projene yapıştır — yalnızca anahtar yer tutucusunu değiştirmen gerekir.",
|
||||
"keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
|
||||
"keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
|
||||
"keys.snip.keyPh": "API anahtarın",
|
||||
"settings.restart": "Tümünü yeniden başlat",
|
||||
"settings.restartConfirm": "Ağ geçidi ve tüm yerel model sunucuları yeniden başlatılsın mı? Devam eden istekler kesilecek.",
|
||||
"settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
|
||||
"settings.shutdown": "Tümünü durdur ve kapat",
|
||||
"settings.shutdownConfirm": "Tüm yerel model sunucuları durdurulup ağ geçidi kapatılsın mı? Ağ geçidini yeniden başlatana kadar API ve bu panel yanıt vermez.",
|
||||
"settings.shutdownDone": "Ağ geçidi kapalı.",
|
||||
"settings.shutdownHint": "Tüm yerel model sunucuları durduruldu ve süreç sonlandı. Paneli geri getirmek için ağ geçidini sunucuda yeniden başlatın.",
|
||||
"bench.title": "Yerel model hızı",
|
||||
"bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
|
||||
"bench.run": "Ölç",
|
||||
"bench.running": "ölçülüyor…",
|
||||
"bench.failed": "başarısız",
|
||||
"bench.none": "Henüz yerel .cmf sohbet modeli yok.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Sohbet",
|
||||
"task.code": "Kod",
|
||||
"task.creative-writing": "Yaratıcı yazarlık",
|
||||
"task.extraction": "Veri çıkarma",
|
||||
"task.math": "Matematik",
|
||||
"task.qa": "Soru-cevap",
|
||||
"task.summarization": "Özetleme",
|
||||
"task.translation": "Çeviri",
|
||||
"routing.tasks.title": "Göreve göre yönlendirme",
|
||||
"routing.tasks.hint": "Kuralın modelleri kendi görevi için önce gelir; karmaşıklık zinciri yedek olarak kalır. Boş = yalnızca karmaşıklığa göre.",
|
||||
"routing.tasks.inherit": "karmaşıklığa göre (kural yok)",
|
||||
"routing.tasks.share": "isteklerin %{p}'i (7 g)",
|
||||
"routing.tasks.fromRouter": "etiketler yönlendiriciden canlı",
|
||||
"routing.tasks.builtin": "yönlendiriciye ulaşılamıyor — yerleşik etiket seti",
|
||||
"routing.tasks.addCustom": "Görev ekle",
|
||||
"routing.tasks.addPlaceholder": "özel etiket…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Kıyaslama",
|
||||
"bm.title": "Kıyaslama",
|
||||
"bm.subtitle": "Canlı hat üzerinden kalite ve hız probları: yanıtlar anahtar kelimeyle doğrulanır, döngü tespit edilir, model başına tok/s ve ilk token süresi ölçülür.",
|
||||
"bm.models": "Test edilecek modeller",
|
||||
"bm.quickMode": "Hızlı test",
|
||||
"bm.quickHint": "Kısa yanıtlı hızlı alt küme — yerel model başına yaklaşık bir dakika.",
|
||||
"bm.suiteInfo": "Yerleşik küme: {q} hızlı / {f} tam test, artı kendi testleriniz.",
|
||||
"bm.run": "Çalıştır",
|
||||
"bm.pickModel": "En az bir model seçin.",
|
||||
"bm.testOf": "{n} testten {i}.",
|
||||
"bm.results": "Sonuçlar",
|
||||
"bm.quick": "hızlı",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Modelin tam yanıtı için hücreye tıklayın.",
|
||||
"bm.history": "Önceki çalıştırmalar (bu tarayıcıda saklanır)",
|
||||
"bm.custom": "Özel testler",
|
||||
"bm.customHint": "İstem + yanıtın içermesi gereken virgülle ayrılmış anahtar kelimeler (| eşanlamlılar için). Bu tarayıcıda saklanır.",
|
||||
"bm.promptPh": "istem…",
|
||||
"bm.checkPh": "anahtar kelimeler, virgülle",
|
||||
"bm.addTest": "Test ekle",
|
||||
"bm.emptyAnswer": "boş yanıt",
|
||||
"bm.loopDetected": "döngü tespit edildi",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Akıllı yönlendirme",
|
||||
"bm.mode.models": "Tek tek modeller",
|
||||
"bm.mode.routingDesc": "Ürünün temel özelliği testte: her prob cortiq-auto üzerinden gider — yönlendiricinin hangi görevi algıladığını, hangi modeli seçtiğini, bunun beklentiyle örtüşüp örtüşmediğini ve maliyetini görürsünüz.",
|
||||
"bm.mode.modelsDesc": "Aynı küme seçilen modellere sabitlenir — yönlendiriciyi atlayarak kalite, hız ve maliyeti doğrudan karşılaştırın.",
|
||||
"bm.answersOk": "yanıtlar doğru",
|
||||
"bm.classAcc": "görev sınıflandırması",
|
||||
"bm.modelDist": "İstekler şuraya gitti",
|
||||
"bm.expectedLbl": "beklenen:",
|
||||
"bm.answer": "Yanıt",
|
||||
"bm.liveRunning": "sürüyor…",
|
||||
"bm.spent": "harcanan",
|
||||
"bm.verdict": "🏁 Bitti: yanıtlar {p}, harcanan {s}",
|
||||
"bm.clearHistory": "Tümünü temizle",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — terminale yapıştırın (ağ geçidi Anthropic protokolünü doğal olarak konuşur):",
|
||||
"keys.h.codex": "Codex — sağlayıcıyı ~/.codex/config.toml dosyasına ekler ve başlatır:",
|
||||
"keys.h.opencode": "OpenCode — ~/.config/opencode/opencode.json dosyasını yazar (üzerine yazar!) ve başlatır:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — modeli ~/.continue/config.yaml dosyasına ekler:",
|
||||
"keys.h.cline1": "Cline (VS Code) — eklenti ayarlarında şunları seçin:",
|
||||
"keys.h.cline2": "Ayarlar → API Provider → OpenAI Compatible, ardından yukarıdaki değerleri yapıştırın.",
|
||||
"keys.h.note": "cortiq-auto = akıllı yönlendirme: yönlendirici her istek için modeli kendisi seçer. Belirli bir model için tam adını yazın. Windows'ta betikleri Git Bash / WSL içinde çalıştırın veya aynı değişkenleri sistem ortamından ayarlayın.",
|
||||
"keys.h.protoOff": "Anthropic protokolü (/v1/messages) kapalı — Claude Code bağlanamaz.",
|
||||
"keys.h.enableProto": "Etkinleştir",
|
||||
"keys.h.st.installed": "Ağ geçidi makinesinde kurulu",
|
||||
"keys.h.st.configured": "Bu ağ geçidi için zaten yapılandırılmış",
|
||||
"keys.h.st.notFound": "Ağ geçidi makinesinde bulunamadı",
|
||||
"bm.degraded": "yönlendirici yedeğe düştü",
|
||||
"bm.routerDegraded": "Yönlendirici zamanında yanıt vermedi — istek yedek zincire gitti; sınıflandırma sayılmaz.",
|
||||
"bm.cutByCap": "bütçe düşünmeye gitti — yanıt gelemedi",
|
||||
"shell.uiStale": "Ağ geçidi güncellendi — bu sayfa eski arayüzü çalıştırıyor.",
|
||||
"shell.reload": "Yeniden yükle",
|
||||
"keys.col.requests": "İstekler",
|
||||
"keys.col.tokens": "Token giriş/çıkış",
|
||||
"keys.col.costAll": "$ toplam",
|
||||
"routing.tasks.tier": "Karmaşıklık",
|
||||
"routing.tasks.tierAuto": "Otomatik (yönlendirici)",
|
||||
"routing.tasks.tierHint": "Bu görevi hangi karmaşıklıkta hangi model karşılar; \"her\" tüm katmanlarda önde gider. Sıra = yedekleme sırası. High kararı \"her\" girdilerinin üzerine yine tırmanır.",
|
||||
"routing.tasks.tierAny": "her karmaşıklık",
|
||||
"routing.tasks.addModel": "model",
|
||||
"routing.bands.enable": "Kendi karmaşıklık sınırlarım",
|
||||
"routing.bands.hint": "Tutamaçları sürükleyin: low nerede biter, high nerede başlar — tutamaçlar birbirini iter, sayılar anlık güncellenir. Yönlendiricinin sürekli puanı SİZİN sınırlarınızla yeniden bölümlenir; işaret kaldırılırsa yönlendiricinin kendi bantları geçerli.",
|
||||
"tier.low": "kolay",
|
||||
"tier.medium": "orta",
|
||||
"tier.high": "zor",
|
||||
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
|
||||
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
|
||||
"settings.cache.entries": "kayıt",
|
||||
"settings.cache.hits": "isabet",
|
||||
"settings.cache.saved": "tasarruf",
|
||||
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
|
||||
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
|
||||
"keys.budgetOf": "bütçe {b}/ay",
|
||||
"dash.kpi.orBalance": "OpenRouter bakiyesi",
|
||||
"keys.h.install": "Kurulum:",
|
||||
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
|
||||
"keys.snip.apply": "Anahtarı ekle",
|
||||
};
|
||||
|
||||
Vendored
+385
@@ -33,6 +33,7 @@ export default {
|
||||
"nav.playground": "沙盒",
|
||||
"nav.settings": "设置",
|
||||
"nav.foot": "网关 · v{v}",
|
||||
"nav.billing": "路由器密钥与充值",
|
||||
|
||||
// topbar / status
|
||||
"status.router": "路由器",
|
||||
@@ -40,6 +41,7 @@ export default {
|
||||
"status.online": "在线",
|
||||
"status.offline": "离线",
|
||||
"status.checking": "检测中…",
|
||||
"status.noKey": "无密钥",
|
||||
"theme.toggle": "主题",
|
||||
"logout": "退出",
|
||||
|
||||
@@ -88,6 +90,7 @@ export default {
|
||||
"dash.health.title": "健康状态",
|
||||
"dash.health.reachable": "可达",
|
||||
"dash.health.unreachable": "不可达",
|
||||
"dash.health.getKey": "获取路由器密钥",
|
||||
|
||||
// models
|
||||
"models.title": "模型",
|
||||
@@ -225,6 +228,18 @@ export default {
|
||||
"settings.router.timeout": "超时(ms)",
|
||||
"settings.router.verifyTls": "验证 TLS 证书",
|
||||
"settings.router.taxonomy": "分类体系 ID",
|
||||
"settings.router.getKey": "获取或充值路由器密钥:",
|
||||
"settings.router.secret": "路由器密钥(值)",
|
||||
"settings.router.test": "测试路由器",
|
||||
"router.probe.ok": "路由器正常({ms} ms)",
|
||||
"router.probe.noKey": "未设置路由器密钥",
|
||||
"router.probe.auth": "密钥被拒绝——无效或已吊销",
|
||||
"router.probe.payment": "需要付款——套餐到期或配额用尽",
|
||||
"router.probe.quota": "超出速率限制",
|
||||
"router.probe.timeout": "路由器响应超时",
|
||||
"router.probe.unreachable": "路由器不可达",
|
||||
"router.probe.error": "路由器错误",
|
||||
"router.usage": "用量",
|
||||
"settings.route": "路由输入",
|
||||
"settings.route.strategy": "文本策略",
|
||||
"settings.route.maxChars": "最大字符数",
|
||||
@@ -265,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "条目 TTL",
|
||||
"settings.cache.embed": "嵌入模型",
|
||||
"settings.cache.note": "缓存设置将在重启后生效。",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "导入模型",
|
||||
|
||||
// dash
|
||||
"dash.clear": "清空日志",
|
||||
"dash.clearConfirm": "清空所有已记录的统计与日志?此操作无法撤销。",
|
||||
"dash.cleared": "日志已清空",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "请输入 id 或模型名称",
|
||||
"models.form.apiKey": "API 密钥",
|
||||
"models.form.apiKeyPlaceholder": "粘贴你的 API 密钥",
|
||||
"models.form.apiKeyNotNeeded": "无需密钥(本地服务器)",
|
||||
"models.form.apiKeyHint": "安全保存在网关中,之后不会再次显示。",
|
||||
"models.form.keyStored": "已保存一个密钥 — 留空表示保留,或粘贴新的密钥替换。",
|
||||
"models.form.keyFromEnv": "正在使用环境变量中的密钥。",
|
||||
"models.form.paramsTitle": "生成与思考参数(O(1)、MTP、温度)",
|
||||
"models.form.temperature": "温度",
|
||||
"models.form.temperatureHint": "生成温度(0.0 = 确定性,1.0+ = 更有创造性)。",
|
||||
"models.form.topP": "Top-P(核采样)",
|
||||
"models.form.topPHint": "采样阈值(0.0 .. 1.0)。",
|
||||
"models.form.maxTokens": "最大 token 数",
|
||||
"models.form.maxTokensHint": "生成内容的最大 token 数。",
|
||||
"models.form.thinkBudget": "思考模式 / 思考预算",
|
||||
"models.form.thinkBudgetHint": "推理 token 预算(0 = 关闭,N > 0 = 推理 token 上限)。",
|
||||
"models.form.systemPrompt": "默认系统提示词",
|
||||
"models.form.systemPromptHint": "当请求中没有系统提示词时自动添加到前面。",
|
||||
"models.form.systemPromptPlaceholder": "你是一个乐于助人的助手……",
|
||||
"models.form.testKey": "测试并加载模型",
|
||||
"models.form.testing": "检查中……",
|
||||
"models.form.foundModels": "已加载 {n} 个模型",
|
||||
"models.form.modelPlaceholder": "输入以搜索……",
|
||||
"models.form.modelHint": "点击“测试并加载模型”用你的密钥拉取列表,然后选择一个。",
|
||||
"models.managed": "托管",
|
||||
"models.managedHint": "由托管的 CMF 服务器提供(设置 → 本地模型)。请在那里配置。",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "使用路由器(关闭 = 仅本地模型)",
|
||||
"settings.cmf": "本地模型(CMF)",
|
||||
"settings.cmf.localOnly": "仅使用本地模型",
|
||||
"settings.cmf.manage": "运行本地模型服务器",
|
||||
"settings.cmf.autoInstall": "自动安装 cortiq(crates.io)",
|
||||
"settings.cmf.autoUpdate": "自动更新 cortiq",
|
||||
"settings.cmf.model": "本地 .cmf 模型路径",
|
||||
"settings.cmf.port": "本地服务器端口",
|
||||
"settings.cmf.modelId": "模型在池中的 id",
|
||||
"settings.cmf.note": "通过 cortiq 提供本地 .cmf 服务(github.com/infosave2007/cmf)。即使路由器不可用也能工作。",
|
||||
"settings.cmf.threads": "工作线程",
|
||||
"settings.cmf.gpu": "使用 GPU(Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "线程会并行化解码阶段的矩阵向量乘法(约快 3 倍;0 = 运行时默认值)。GPU 卸载主要加速长提示词的 prefill — 解码受内存带宽限制,因此在 Apple Silicon 上基本持平。",
|
||||
"settings.cmf.pick": "选择一个模型",
|
||||
"settings.cmf.models": "本地模型",
|
||||
"settings.cmf.modelsHelp": "每一行都是网关运行的一个模型(各自独立的 cortiq serve,各占一个端口)并注册到模型池。有多少 .cmf 文件和空闲端口就可以加多少行。",
|
||||
"settings.cmf.rowHint": "id · .cmf 文件 · 端口 · 线程 · GPU。改动在重启后生效;删除一行会停止对应的服务器。",
|
||||
"settings.cmf.addServer": "+ 添加本地模型",
|
||||
"settings.cmf.checkPort": "检查端口",
|
||||
"settings.cmf.portFree": "端口空闲",
|
||||
"settings.cmf.portBusy": "端口被占用",
|
||||
"settings.cmf.useSuggested": "使用 {port}",
|
||||
"settings.cmf.serving": "{model} 正在 :{port} 提供服务",
|
||||
"settings.cmf.intro": "用 cortiq 引擎在本地运行模型 — 即使离线或路由器故障也能工作。",
|
||||
"settings.cmf.runtime": "cortiq 运行时",
|
||||
"settings.cmf.runtimeHelp": "用于加载和运行 .cmf 模型的命令行工具 `cortiq`。启动本地服务器前先安装一次。",
|
||||
"settings.cmf.install": "安装 cortiq",
|
||||
"settings.cmf.reinstall": "重新安装 / 更新",
|
||||
"settings.cmf.installing": "安装中……(需要编译,几分钟)",
|
||||
"settings.cmf.installedOk": "cortiq 已安装",
|
||||
"settings.cmf.notInstalled": "未安装",
|
||||
"settings.cmf.addModel": "添加模型",
|
||||
"settings.cmf.addModelHelp": "模型在“导入模型”标签页下载并转换(HuggingFace → .cmf)。然后把下面的本地服务器指向该 .cmf 文件。",
|
||||
"settings.cmf.importNote": "还没有模型?去导入一个:",
|
||||
"settings.cmf.importLink": "打开导入",
|
||||
"settings.cmf.modelHelp": "要提供服务的 .cmf 文件路径(例如 models/qwen.cmf),由导入标签页生成。要为不同任务准备多个模型?导入它们并切换此路径。",
|
||||
"settings.cmf.portHelp": "本地模型服务器监听的 TCP 端口(默认 8081)。仅在与其他服务冲突时修改。",
|
||||
"settings.cmf.threadsHelp": "生成时使用多少个 CPU 核心。越多越快(相比单核约快 3 倍);0 表示交给运行时决定。",
|
||||
"settings.cmf.gpuHelp": "把矩阵运算卸载到 GPU(Metal/wgpu)。在 Apple Silicon 上主要加速长提示词 — 解码受内存带宽限制。",
|
||||
"settings.cmf.localOnlyHelp": "把每个请求都发往本地模型,忽略路由器和所有云端供应商。",
|
||||
"settings.cmf.manageHelp": "网关启动时会为下面的 .cmf 启动 `cortiq serve` 并把请求路由过去。需要已安装 cortiq(第 1 步)。重启后生效。",
|
||||
"settings.cmf.autoInstallHelp": "如果启动时缺少 cortiq,自动从 crates.io 安装。",
|
||||
"settings.cmf.autoUpdateHelp": "启动时如有更新版本,将 cortiq 升级到最新发布版。",
|
||||
"settings.router.enabledHelp": "让 cortiq 路由器对每个请求分类(任务类型 + 复杂度)并挑选模型。关闭 = 仅使用本地路由表。",
|
||||
"settings.router.verifyTlsHelp": "校验路由器的 TLS 证书。仅在使用自签名或绑定 IP 的证书时关闭。",
|
||||
"settings.telemetry.metricsHelp": "在 GET /metrics 暴露 Prometheus 指标。",
|
||||
"settings.cortiq.echoHelp": "在每个响应体中加入 `cortiq` 字段(任务、模型、成本)。响应头始终会发送。",
|
||||
"settings.stats.enabledHelp": "记录最近的请求,为仪表盘和图表提供数据。",
|
||||
"settings.cache.enabledHelp": "语义缓存:当新提示词与过去的某个非常相似时直接返回已保存的回答,跳过模型调用。需要池中有一个嵌入模型。",
|
||||
"settings.log.bodiesHelp": "记录请求/响应的完整消息体。出于隐私考虑默认关闭。",
|
||||
"settings.router.saveKey": "保存密钥",
|
||||
"settings.router.keySaved": "路由器密钥已保存",
|
||||
"settings.router.keyEmpty": "请先粘贴密钥",
|
||||
"settings.router.secretHelp": "粘贴 api.allaigate.com 邮件中的密钥,然后点击“保存密钥”。这不是“API 密钥”页中的值 — 那些是给调用你网关的客户端用的。",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — 密钥已保存(再次粘贴可替换)",
|
||||
"settings.route.strategyHelp": "发送给路由器用于分类的文本:last_user = 仅最后一条用户消息;last_user_plus_system = 最后一条用户消息 + 系统提示词;concat_all = 整段对话。",
|
||||
"settings.route.maxCharsHelp": "发送给路由器的文本最大字符数(超出会截断)。越多分类越准,但路由器调用更慢也更贵。",
|
||||
"settings.route.cacheTtlHelp": "相同请求的路由决策缓存多久后才再次询问路由器(例如 60s、5m)。",
|
||||
"settings.route.profileHelp": "请求未指定时使用的默认路由档位:cost-saver(最省钱)、balanced 或 quality-first(最好的模型)。会影响路由器挑选哪个模型。",
|
||||
|
||||
// import
|
||||
"import.subtitle": "在 HuggingFace 上搜索模型,转换为本地 .cmf 并提供服务。",
|
||||
"import.searchPlaceholder": "搜索 HuggingFace 模型(例如 qwen3、llama)……",
|
||||
"import.searching": "搜索中……",
|
||||
"import.noResults": "未找到模型。",
|
||||
"import.configure": "配置转换",
|
||||
"import.outName": "输出名称",
|
||||
"import.outNameHint": ".cmf 文件名(字母、数字、- 和 _)。",
|
||||
"import.quant": "量化",
|
||||
"import.convert": "转换并加载",
|
||||
"import.converting": "转换中",
|
||||
"import.started": "转换已开始",
|
||||
"import.failed": "转换失败 — 请查看日志。",
|
||||
"import.register": "注册为本地模型",
|
||||
"import.registered": "已注册为 “{id}”。运行 cortiq-server 即可提供服务。",
|
||||
"import.jobs": "下载与转换",
|
||||
"import.cancel": "取消",
|
||||
"import.cancelling": "正在取消 — 清理未完成的文件……",
|
||||
"import.delete": "删除",
|
||||
"import.deleteConfirm": "从磁盘删除该模型及其转换好的 .cmf 文件?",
|
||||
"import.deleted": "已连同文件一起删除",
|
||||
"import.state.running": "进行中",
|
||||
"import.state.done": "完成",
|
||||
"import.state.error": "错误",
|
||||
"import.state.cancelled": "已取消",
|
||||
"import.featuredTitle": "推荐的开箱即用 .cmf 模型",
|
||||
"import.featuredSub": "来自 HuggingFace 上 infosave 账号的预转换 .cmf 模型 — 列表实时读取,新模型会自动出现。",
|
||||
"import.readyCmfBadge": "现成 .cmf",
|
||||
"import.directDownload": "下载",
|
||||
"import.q.q8_2f": "8 位双场(𝒲×θ)— 推荐:质量与体积的最佳平衡。",
|
||||
"import.q.q8_row": "8 位按行 — 简单稳健。",
|
||||
"import.q.q4": "4 位分块 — 体积最小,质量较低。",
|
||||
"import.q.q4_tiled": "4 位分块平铺(Q4_TILED)— 在 Metal GPU 上快 1.66 倍。",
|
||||
"import.q.vbit": "可变 3–8 位 — 按专家分配位宽预算。",
|
||||
"import.q.q1t": "1.58 位三值(Q1T)— BitNet b1.58,带稀疏离群值。",
|
||||
"import.q.q1s": "1 位稀疏(Q1S)— 1 位基底,带全息离群值。",
|
||||
"import.q.q1p": "1 位 PTQ(Q1P)— 1 位,带误差扩散。",
|
||||
"import.q.q1": "1 位(Q1)— 纯二值格式。",
|
||||
"import.q.f16": "16 位 — 不量化。",
|
||||
"import.q.f32": "32 位 — 全精度(文件很大)。",
|
||||
"import.adv.title": "高级设置",
|
||||
"import.adv.auto": "自动",
|
||||
"import.adv.linearCore": "线性核",
|
||||
"import.adv.linearHint": "线性注意力层如何被保留或折叠。",
|
||||
"import.adv.nphase": "vmf_phase 每头相位数",
|
||||
"import.adv.vbitShape": "VBIT 预算形状",
|
||||
"import.adv.meanBits": "VBIT 平均位宽",
|
||||
"import.adv.shard": "分片上限(GB)",
|
||||
"import.adv.noShard": "不分片",
|
||||
"import.adv.skipMtp": "跳过 MTP 头",
|
||||
"import.adv.o1": "O(1) 注意力",
|
||||
"import.adv.o1Hint": "常量内存的流式注意力:prefill 之后释放 KV 缓存,因此长上下文不会持续增加内存。权重保持逐字节一致 — 只记录一个加载时的提示;随时可以切回(启动时加 --o1 off)。已在 Qwen3-0.6B 上验证:全部层约为 ×1.18 困惑度。",
|
||||
"import.adv.o1Off": "关闭 — 精确注意力(默认)",
|
||||
"import.adv.o1All": "all — 转换所有 softmax 层",
|
||||
"import.adv.o1Deep": "deep N — 仅最深的 N 层",
|
||||
"import.adv.o1Custom": "自定义 — 显式指定层列表",
|
||||
"import.adv.o1DeepN": "最深的 N 层",
|
||||
"import.adv.o1Layers": "层(用逗号分隔)",
|
||||
"import.adv.o1mHint": "landmark 预算(默认 32)",
|
||||
"import.adv.o1windowHint": "精确窗口宽度(默认 128)",
|
||||
"import.adv.o1sinkHint": "常驻精确 sink key 数(默认 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "初始设置",
|
||||
"onboarding.welcome": "欢迎使用 Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "选择你的语言 — 随时可以在顶栏更改。",
|
||||
"onboarding.next": "下一步",
|
||||
"onboarding.back": "上一步",
|
||||
"onboarding.skip": "暂时跳过",
|
||||
"onboarding.skipSetup": "跳过",
|
||||
"onboarding.finish": "完成设置",
|
||||
"onboarding.featuredTitle": "开箱即用的模型(一键)",
|
||||
"onboarding.featuredSub": "这些是 HuggingFace 上 infosave 发布的、已经转换好的 .cmf 模型 — 直接下载即可,无需转换。选择一个或多个即可立即开始。",
|
||||
"onboarding.download": "下载",
|
||||
"onboarding.hfTitle": "任意 HuggingFace 模型 — 自动转换",
|
||||
"onboarding.hfSub": "搜索任意模型,选择量化方式,我们会自动把它转换为 .cmf。无需手动修改配置。",
|
||||
"onboarding.noJobs": "还没有下载任务。",
|
||||
"onboarding.doneTitle": "全部就绪",
|
||||
"onboarding.doneSub": "下载好的模型会自动注册。之后可以在“导入模型”或“模型”中添加更多。",
|
||||
"onboarding.doneHint": "“完成设置”会把每个下载完成的模型接入为本地模型,启动它的 .cmf 服务器并把路由指向它。这些之后都可以在“模型”中修改。",
|
||||
"onboarding.willRegister": "将接入为本地模型",
|
||||
"onboarding.nothingToRegister": "还没有下载完成的模型 — 没有可接入的内容。返回上一步选一个模型。",
|
||||
"onboarding.registeredN": "已接入 {n} 个模型:{ids}",
|
||||
"onboarding.runtimeMissing": "cortiq 运行时尚未安装 — 请在 设置 → 本地模型 中安装后才能运行这些模型。",
|
||||
"onboarding.downloaded": "已下载",
|
||||
"onboarding.update": "更新",
|
||||
"onboarding.deleteFile": "从磁盘删除该 .cmf 文件",
|
||||
"onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。",
|
||||
"onboarding.unregistered": "同时已取消注册:{ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "对话",
|
||||
"pg.tab.image": "图像",
|
||||
"pg.tab.video": "视频",
|
||||
"pg.generate": "生成",
|
||||
"pg.generating": "生成中……",
|
||||
"pg.genFailed": "生成失败 — 请查看日志。",
|
||||
"pg.result": "结果",
|
||||
"pg.genMetrics": "生成指标",
|
||||
"pg.imagePromptPlaceholder": "夕阳下雪地里的红狐狸,照片级真实感,毛发细节丰富",
|
||||
"pg.videoPromptPlaceholder": "戴厨师帽的柯基翻煎饼,滋滋声和欢快的犬吠",
|
||||
"pg.imageEmpty": "生成的图像将显示在这里。",
|
||||
"pg.videoEmpty": "生成的短片(含音频)将显示在这里。",
|
||||
"pg.width": "宽度",
|
||||
"pg.height": "高度",
|
||||
"pg.steps": "步数",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "随机 seed",
|
||||
"pg.cfgHint": "引导强度;≤ 1 关闭 CFG,工作量减半。",
|
||||
"pg.frames": "帧数",
|
||||
"pg.framesHint": "24 fps;按模型网格取整。",
|
||||
"pg.quality": "JPEG 质量",
|
||||
"pg.videoSizeHint": "32 的倍数;训练短边为 768。",
|
||||
"pg.videoStepsHint": "Turbo LoRA 按 4 步训练。",
|
||||
"pg.firstFrame": "首帧(可选)",
|
||||
"pg.lastFrame": "末帧(可选)",
|
||||
"pg.keyframeHint": "使运行变为关键帧生视频。",
|
||||
"pg.resolution": "分辨率",
|
||||
"pg.genTime": "生成耗时",
|
||||
"pg.timePerStep": "每步",
|
||||
"pg.timePerFrame": "每帧",
|
||||
"pg.fileSize": "文件大小",
|
||||
"pg.duration": "时长",
|
||||
"pg.play": "播放",
|
||||
"pg.noMediaModels": "还没有本地{kind}模型。请在“导入模型”中下载一个(例如 infosave 合集):",
|
||||
"pg.tokSpeed": "速度",
|
||||
"pg.ttft": "首个 token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "日志",
|
||||
"logs.subtitle": "网关最近的日志 — 终端里会看到的内容。",
|
||||
"logs.filterPlaceholder": "过滤……",
|
||||
"logs.pause": "暂停",
|
||||
"logs.resume": "继续",
|
||||
"logs.empty": "没有匹配内容。",
|
||||
"notice.openAdmin": "管理控制台未设置令牌即可从其他机器访问。若这台机器不是私有的,请设置令牌。",
|
||||
"notice.update": "有可用更新:v{v}。",
|
||||
"dash.sys.title": "系统",
|
||||
"dash.sys.cores": "核",
|
||||
"dash.sys.disk": "磁盘(模型)",
|
||||
"dash.sys.free": "可用",
|
||||
"dash.sys.gateway": "网关",
|
||||
"fit.disk": "磁盘空间不足",
|
||||
"fit.diskTitle": "下载大小超过模型磁盘的可用空间。",
|
||||
"fit.ram": "可能超出内存",
|
||||
"fit.ramTitle": "模型接近或超过本机内存 — 可能无法运行。",
|
||||
"pg.eta": "剩余",
|
||||
"pg.min": "分钟",
|
||||
"pg.queued": "排队中 — 第 {n} 位。当前生成结束后立即开始。",
|
||||
"pg.gallery": "最近的生成",
|
||||
"pg.galDeleteConfirm": "删除这次生成及其文件?",
|
||||
"pg.clearChat": "清空",
|
||||
"keys.snip.title": "接入你的应用",
|
||||
"keys.snip.hint": "把示例粘贴到你的项目里 — 只需替换密钥占位符。",
|
||||
"keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
|
||||
"keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
|
||||
"keys.snip.keyPh": "你的 API 密钥",
|
||||
"settings.restart": "全部重启",
|
||||
"settings.restartConfirm": "重启网关和所有本地模型服务器?正在处理的请求将被中断。",
|
||||
"settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
|
||||
"settings.shutdown": "全部停止并关机",
|
||||
"settings.shutdownConfirm": "停止所有本地模型服务器并关闭网关?在你重新启动网关之前,API 和此面板都将无法响应。",
|
||||
"settings.shutdownDone": "网关已关闭。",
|
||||
"settings.shutdownHint": "所有本地模型服务器已停止,进程已退出。请在服务器上重新启动网关以恢复面板。",
|
||||
"bench.title": "本地模型速度",
|
||||
"bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s,一次一个模型。",
|
||||
"bench.run": "测速",
|
||||
"bench.running": "测速中……",
|
||||
"bench.failed": "失败",
|
||||
"bench.none": "还没有本地 .cmf 聊天模型。",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "闲聊",
|
||||
"task.code": "代码",
|
||||
"task.creative-writing": "创意写作",
|
||||
"task.extraction": "数据提取",
|
||||
"task.math": "数学",
|
||||
"task.qa": "问答",
|
||||
"task.summarization": "摘要",
|
||||
"task.translation": "翻译",
|
||||
"routing.tasks.title": "按任务路由",
|
||||
"routing.tasks.hint": "规则中的模型对该任务优先;下方的复杂度链仍作为回退。留空 = 仅按复杂度。",
|
||||
"routing.tasks.inherit": "按复杂度(无规则)",
|
||||
"routing.tasks.share": "{p}% 的请求(7 天)",
|
||||
"routing.tasks.fromRouter": "标签实时来自路由器",
|
||||
"routing.tasks.builtin": "路由器不可达 — 使用内置标签集",
|
||||
"routing.tasks.addCustom": "添加任务",
|
||||
"routing.tasks.addPlaceholder": "自定义标签……",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "基准测试",
|
||||
"bm.title": "基准测试",
|
||||
"bm.subtitle": "通过实时管线做质量与速度探测:按关键词核对答案、检测循环输出,并按模型测量 tok/s 与首个 token 时间。",
|
||||
"bm.models": "要测试的模型",
|
||||
"bm.quickMode": "快速测试",
|
||||
"bm.quickHint": "答案简短的快速子集 — 每个本地模型约一分钟。",
|
||||
"bm.suiteInfo": "内置套件:{q} 个快速 / {f} 个完整测试,另加你自己的。",
|
||||
"bm.run": "运行",
|
||||
"bm.pickModel": "请至少选择一个模型。",
|
||||
"bm.testOf": "第 {i}/{n} 个测试",
|
||||
"bm.results": "结果",
|
||||
"bm.quick": "快速",
|
||||
"bm.test": "测试",
|
||||
"bm.cellHint": "点击单元格查看模型的完整回答。",
|
||||
"bm.history": "历史运行(保存在本浏览器)",
|
||||
"bm.custom": "自定义测试",
|
||||
"bm.customHint": "提示词 + 逗号分隔的关键词,回答必须包含它们(| 表示同义词)。保存在本浏览器。",
|
||||
"bm.promptPh": "提示词……",
|
||||
"bm.checkPh": "关键词,逗号分隔",
|
||||
"bm.addTest": "添加测试",
|
||||
"bm.emptyAnswer": "回答为空",
|
||||
"bm.loopDetected": "检测到循环",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "智能路由",
|
||||
"bm.mode.models": "单独模型",
|
||||
"bm.mode.routingDesc": "测试产品核心能力:每个探测都经过 cortiq-auto — 可以看到路由器识别的任务、选中的模型、是否符合预期,以及花费了多少。",
|
||||
"bm.mode.modelsDesc": "同一套测试固定在所选模型上 — 绕过路由器,直接对比它们的质量、速度与成本。",
|
||||
"bm.answersOk": "回答正确",
|
||||
"bm.classAcc": "任务分类",
|
||||
"bm.modelDist": "请求流向",
|
||||
"bm.expectedLbl": "预期:",
|
||||
"bm.answer": "回答",
|
||||
"bm.liveRunning": "进行中……",
|
||||
"bm.spent": "花费",
|
||||
"bm.verdict": "🏁 完成:回答 {p},花费 {s}",
|
||||
"bm.clearHistory": "全部清除",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — 粘贴到终端(网关原生支持 Anthropic 协议):",
|
||||
"keys.h.codex": "Codex — 将提供商追加到 ~/.codex/config.toml 并启动:",
|
||||
"keys.h.opencode": "OpenCode — 写入 ~/.config/opencode/opencode.json(会覆盖!)并启动:",
|
||||
"keys.h.continue": "Continue(VS Code / JetBrains)— 将模型追加到 ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline(VS Code)— 在扩展设置中选择:",
|
||||
"keys.h.cline2": "设置 → API Provider → OpenAI Compatible,然后粘贴上面的值。",
|
||||
"keys.h.note": "cortiq-auto = 智能路由:路由器为每个请求选择模型。要固定某个模型,请填写其精确名称。Windows 上请在 Git Bash / WSL 中运行脚本,或通过系统环境变量设置相同的值。",
|
||||
"keys.h.protoOff": "Anthropic 协议(/v1/messages)已关闭 — Claude Code 无法连接。",
|
||||
"keys.h.enableProto": "启用",
|
||||
"keys.h.st.installed": "已安装在网关所在机器上",
|
||||
"keys.h.st.configured": "已配置为使用此网关",
|
||||
"keys.h.st.notFound": "网关所在机器上未找到",
|
||||
"bm.degraded": "路由器降级",
|
||||
"bm.routerDegraded": "路由器未及时响应 — 请求走了备用链路;分类不计入统计。",
|
||||
"bm.cutByCap": "预算全用于思考——未能给出答案",
|
||||
"shell.uiStale": "网关已更新 — 此页面运行的是旧界面。",
|
||||
"shell.reload": "重新加载",
|
||||
"keys.col.requests": "请求数",
|
||||
"keys.col.tokens": "Token 入/出",
|
||||
"keys.col.costAll": "$ 累计",
|
||||
"routing.tasks.tier": "复杂度",
|
||||
"routing.tasks.tierAuto": "自动(路由器)",
|
||||
"routing.tasks.tierHint": "哪个模型在哪个复杂度上处理该任务;「任意」在所有层级领先。顺序即回退顺序。high 判定仍会越过「任意」条目升级。",
|
||||
"routing.tasks.tierAny": "任意复杂度",
|
||||
"routing.tasks.addModel": "模型",
|
||||
"routing.bands.enable": "自定义复杂度边界",
|
||||
"routing.bands.hint": "拖动手柄设定 low 到哪里结束、high 从哪里开始 — 手柄会相互推动,数字实时更新。路由器的连续得分将按你的边界重新划分;取消勾选则使用路由器自身的边界。",
|
||||
"tier.low": "简单",
|
||||
"tier.medium": "中等",
|
||||
"tier.high": "困难",
|
||||
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
|
||||
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
|
||||
"settings.cache.entries": "条目",
|
||||
"settings.cache.hits": "命中",
|
||||
"settings.cache.saved": "已节省",
|
||||
"keys.form.budget": "每月预算,$(0 = 不限)",
|
||||
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
|
||||
"keys.budgetOf": "预算 {b}/月",
|
||||
"dash.kpi.orBalance": "OpenRouter 余额",
|
||||
"keys.h.install": "安装:",
|
||||
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
|
||||
"keys.snip.apply": "插入密钥",
|
||||
};
|
||||
|
||||
Vendored
+167
@@ -0,0 +1,167 @@
|
||||
// Browser-side codecs for cortiq's dependency-free media outputs:
|
||||
// - P6 PPM → canvas (cortiq imagine links no image library)
|
||||
// - MJPEG+PCM AVI → frame list + canvas player synced to the .wav
|
||||
// (cortiq animate never touches ffmpeg; neither do we)
|
||||
|
||||
/// Parse binary P6 PPM into {width, height, rgb: Uint8Array}.
|
||||
export function parsePpm(buf) {
|
||||
const b = new Uint8Array(buf);
|
||||
// header: "P6" <ws> width <ws> height <ws> maxval <single ws> raster
|
||||
let pos = 0;
|
||||
const isWs = (c) => c === 32 || c === 9 || c === 10 || c === 13;
|
||||
function token() {
|
||||
while (pos < b.length && (isWs(b[pos]) || b[pos] === 35 /* # comment */)) {
|
||||
if (b[pos] === 35) while (pos < b.length && b[pos] !== 10) pos++;
|
||||
else pos++;
|
||||
}
|
||||
let start = pos;
|
||||
while (pos < b.length && !isWs(b[pos])) pos++;
|
||||
return String.fromCharCode(...b.slice(start, pos));
|
||||
}
|
||||
if (token() !== "P6") throw new Error("not a P6 PPM");
|
||||
const width = parseInt(token());
|
||||
const height = parseInt(token());
|
||||
const maxval = parseInt(token());
|
||||
pos++; // single whitespace after maxval
|
||||
if (!width || !height || !maxval) throw new Error("bad PPM header");
|
||||
return { width, height, rgb: b.subarray(pos, pos + width * height * 3) };
|
||||
}
|
||||
|
||||
/// Draw a parsed PPM onto a (new or given) canvas. Returns the canvas.
|
||||
export function ppmToCanvas(buf, canvas) {
|
||||
const { width, height, rgb } = parsePpm(buf);
|
||||
const c = canvas || document.createElement("canvas");
|
||||
c.width = width;
|
||||
c.height = height;
|
||||
const ctx = c.getContext("2d");
|
||||
const img = ctx.createImageData(width, height);
|
||||
for (let i = 0, j = 0; i < width * height; i++, j += 3) {
|
||||
img.data[i * 4] = rgb[j];
|
||||
img.data[i * 4 + 1] = rgb[j + 1];
|
||||
img.data[i * 4 + 2] = rgb[j + 2];
|
||||
img.data[i * 4 + 3] = 255;
|
||||
}
|
||||
ctx.putImageData(img, 0, 0);
|
||||
return c;
|
||||
}
|
||||
|
||||
/// Rasterize an uploaded image file to WxH (stretched, as the engine does with
|
||||
/// the first frame) and return it as base64-encoded binary P6 PPM.
|
||||
export async function fileToPpmB64(file, width, height) {
|
||||
const bmp = await createImageBitmap(file);
|
||||
const c = document.createElement("canvas");
|
||||
c.width = width;
|
||||
c.height = height;
|
||||
const ctx = c.getContext("2d");
|
||||
ctx.drawImage(bmp, 0, 0, width, height);
|
||||
const img = ctx.getImageData(0, 0, width, height).data;
|
||||
const header = `P6\n${width} ${height}\n255\n`;
|
||||
const out = new Uint8Array(header.length + width * height * 3);
|
||||
for (let i = 0; i < header.length; i++) out[i] = header.charCodeAt(i);
|
||||
let p = header.length;
|
||||
for (let i = 0; i < width * height; i++) {
|
||||
out[p++] = img[i * 4];
|
||||
out[p++] = img[i * 4 + 1];
|
||||
out[p++] = img[i * 4 + 2];
|
||||
}
|
||||
// chunked btoa — the frame can be a few MB
|
||||
let bin = "";
|
||||
for (let i = 0; i < out.length; i += 0x8000)
|
||||
bin += String.fromCharCode.apply(null, out.subarray(i, i + 0x8000));
|
||||
return btoa(bin);
|
||||
}
|
||||
|
||||
/// Extract JPEG frames from an MJPEG AVI (RIFF): walk chunks from 'movi',
|
||||
/// collect '00dc'/'00dj' payloads as Blobs.
|
||||
export function aviJpegFrames(buf) {
|
||||
const b = new Uint8Array(buf);
|
||||
const dv = new DataView(buf);
|
||||
const fourcc = (o) => String.fromCharCode(b[o], b[o + 1], b[o + 2], b[o + 3]);
|
||||
if (fourcc(0) !== "RIFF" || fourcc(8) !== "AVI ") throw new Error("not an AVI");
|
||||
// find the 'movi' LIST
|
||||
let movi = -1;
|
||||
for (let o = 12; o + 12 <= b.length; ) {
|
||||
const id = fourcc(o);
|
||||
const size = dv.getUint32(o + 4, true);
|
||||
if (id === "LIST" && fourcc(o + 8) === "movi") {
|
||||
movi = o + 12;
|
||||
break;
|
||||
}
|
||||
o += 8 + size + (size & 1);
|
||||
}
|
||||
if (movi < 0) throw new Error("no movi list");
|
||||
const frames = [];
|
||||
for (let o = movi; o + 8 <= b.length; ) {
|
||||
const id = fourcc(o);
|
||||
const size = dv.getUint32(o + 4, true);
|
||||
if (id === "00dc" || id === "00dj") {
|
||||
if (size > 0) frames.push(new Blob([b.subarray(o + 8, o + 8 + size)], { type: "image/jpeg" }));
|
||||
} else if (id === "LIST") {
|
||||
o += 12; // descend (rec lists)
|
||||
continue;
|
||||
} else if (id === "idx1") {
|
||||
break;
|
||||
}
|
||||
o += 8 + size + (size & 1);
|
||||
}
|
||||
return frames;
|
||||
}
|
||||
|
||||
/// Canvas player for extracted MJPEG frames at a fixed fps, synced to an
|
||||
/// optional <audio> element (the .wav). Returns {canvas, play, pause, seek0, playing}.
|
||||
export function mjpegPlayer(frames, fps, audio) {
|
||||
const canvas = document.createElement("canvas");
|
||||
const ctx = canvas.getContext("2d");
|
||||
let bitmaps = null; // decoded lazily on first play
|
||||
let raf = null;
|
||||
let t0 = 0;
|
||||
let startFrame = 0;
|
||||
const state = { playing: false };
|
||||
|
||||
async function decodeAll() {
|
||||
if (bitmaps) return;
|
||||
bitmaps = await Promise.all(frames.map((f) => createImageBitmap(f)));
|
||||
if (bitmaps.length) {
|
||||
canvas.width = bitmaps[0].width;
|
||||
canvas.height = bitmaps[0].height;
|
||||
ctx.drawImage(bitmaps[0], 0, 0);
|
||||
}
|
||||
}
|
||||
function drawAt(i) {
|
||||
if (bitmaps && bitmaps[i]) ctx.drawImage(bitmaps[i], 0, 0);
|
||||
}
|
||||
function tick() {
|
||||
// audio clock when available (keeps A/V in sync), wall clock otherwise
|
||||
const t = audio && !audio.paused ? audio.currentTime : (performance.now() - t0) / 1000;
|
||||
const i = Math.min(frames.length - 1, startFrame + Math.floor(t * fps));
|
||||
drawAt(i);
|
||||
if (i >= frames.length - 1) {
|
||||
pause();
|
||||
return;
|
||||
}
|
||||
raf = requestAnimationFrame(tick);
|
||||
}
|
||||
async function play() {
|
||||
await decodeAll();
|
||||
state.playing = true;
|
||||
startFrame = 0;
|
||||
t0 = performance.now();
|
||||
if (audio) {
|
||||
audio.currentTime = 0;
|
||||
audio.play().catch(() => {});
|
||||
}
|
||||
raf = requestAnimationFrame(tick);
|
||||
}
|
||||
function pause() {
|
||||
state.playing = false;
|
||||
if (raf) cancelAnimationFrame(raf);
|
||||
raf = null;
|
||||
if (audio) audio.pause();
|
||||
}
|
||||
async function seek0() {
|
||||
pause();
|
||||
await decodeAll();
|
||||
drawAt(0);
|
||||
}
|
||||
return { canvas, play, pause, seek0, state, decodeAll };
|
||||
}
|
||||
Vendored
+33
-4
@@ -23,9 +23,15 @@ export function clear(node) {
|
||||
while (node && node.firstChild) node.removeChild(node.firstChild);
|
||||
}
|
||||
|
||||
export function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
export function mount(node, ...children) {
|
||||
clear(node);
|
||||
for (const c of children.flat()) if (c != null && c !== false) node.appendChild(c);
|
||||
for (const c of children.flat())
|
||||
if (c != null && c !== false)
|
||||
node.appendChild(typeof c === "object" ? c : document.createTextNode(String(c)));
|
||||
return node;
|
||||
}
|
||||
|
||||
@@ -44,7 +50,14 @@ export function toast(msg, kind = "info", ms = 3200) {
|
||||
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
|
||||
export function modal(title, body, onSave, saveLabel) {
|
||||
const backdrop = h("div", { class: "modal-backdrop" });
|
||||
const close = () => backdrop.remove();
|
||||
const onKeydown = (e) => {
|
||||
if (e.key === "Escape") close();
|
||||
};
|
||||
const close = () => {
|
||||
backdrop.remove();
|
||||
document.body.classList.remove("modal-open");
|
||||
document.removeEventListener("keydown", onKeydown);
|
||||
};
|
||||
backdrop.addEventListener("click", (e) => {
|
||||
if (e.target === backdrop) close();
|
||||
});
|
||||
@@ -71,6 +84,8 @@ export function modal(title, body, onSave, saveLabel) {
|
||||
)
|
||||
);
|
||||
backdrop.appendChild(card);
|
||||
document.body.classList.add("modal-open");
|
||||
document.addEventListener("keydown", onKeydown);
|
||||
document.body.appendChild(backdrop);
|
||||
return { close };
|
||||
}
|
||||
@@ -80,11 +95,25 @@ export function confirmDialog(message) {
|
||||
}
|
||||
|
||||
// number formatting
|
||||
// Money as a DOM node: 4 decimals, the last two rendered smaller — tiny
|
||||
// per-request prices stay readable without shouting six zeros.
|
||||
export function moneyEl(n) {
|
||||
if (n == null) return document.createTextNode("—");
|
||||
if (n === 0) return document.createTextNode("$0");
|
||||
// sub-0.0001 costs (cheap flash models) would collapse to $0.0000 — grow
|
||||
// precision until two significant digits survive
|
||||
const s = Math.abs(n) >= 0.0001 ? n.toFixed(4) : n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
|
||||
return h("span", { class: "money" }, "$" + s.slice(0, -2), h("small", { class: "money-tail" }, s.slice(-2)));
|
||||
}
|
||||
|
||||
export function money(n) {
|
||||
if (n == null) return "—";
|
||||
if (n === 0) return "$0";
|
||||
if (Math.abs(n) < 0.01) return "$" + n.toFixed(6);
|
||||
return "$" + n.toFixed(4);
|
||||
if (Math.abs(n) > 0 && Math.abs(n) < 0.0001) return "$" + n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
|
||||
// up to 4 decimals, trailing zeros trimmed (but keep money-like 2 minimum)
|
||||
let s = n.toFixed(4).replace(/0+$/, "").replace(/\.$/, "");
|
||||
if (/\.\d$/.test(s)) s += "0";
|
||||
return "$" + s;
|
||||
}
|
||||
export function ms(n) {
|
||||
return n == null ? "—" : `${Math.round(Number(n))} ms`;
|
||||
|
||||
Vendored
+530
@@ -0,0 +1,530 @@
|
||||
// Benchmark — two modes over the live pipeline:
|
||||
// 🧭 Smart routing (default): the product's core promise under test — every
|
||||
// probe goes through cortiq-auto and the router's task/tier decision and
|
||||
// chosen model are judged alongside the answer itself.
|
||||
// 🎯 Individual models: the same suite pinned to selected models — compare
|
||||
// quality and speed head-to-head.
|
||||
// A redesign of the old standalone benchmark.html: same scoring ideas
|
||||
// (keyword checks with | synonyms, think-tag stripping, anti-loop detection,
|
||||
// pass/partial/fail), now embedded, localized and CDN-free.
|
||||
import { h, mount, toast, ms, money, moneyEl } from "../ui.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api, testStream } from "../api.js";
|
||||
|
||||
// ── built-in suite ──────────────────────────────────────────────────────────
|
||||
// Checks are language-neutral where possible (digits, identifiers, emails);
|
||||
// `check` entries may hold |-separated synonyms — one hit per entry counts.
|
||||
// `quick: true` marks the fast subset.
|
||||
// `label`: router task labels accepted as a correct classification (null = not
|
||||
// judged — the prompt has no natural label in the taxonomy).
|
||||
const SUITE = [
|
||||
{ id: "math-mult", cat: "math", label: ["math"], quick: true, max: 96, prompt: "What is 17 * 23? Answer with the number only.", check: ["391"] },
|
||||
{ id: "math-eq", cat: "math", label: ["math"], max: 128, prompt: "Solve 2x + 6 = 14. Answer with the value of x only.", check: ["4"] },
|
||||
{ id: "logic-syll", cat: "logic", label: ["qa", "math"], max: 96, prompt: "If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? Answer yes or no.", check: ["yes|да|oui|ja|sí|si|evet|是"] },
|
||||
{ id: "count-letters", cat: "logic", label: ["qa", "math"], quick: true, max: 64, prompt: "How many letters are in the word 'cat'? Answer with a digit.", check: ["3"] },
|
||||
{ id: "apples", cat: "logic", label: ["math", "qa"], quick: true, max: 96, prompt: "John has 3 apples, buys 2 more, then eats 1. How many apples does he have? Answer with a digit.", check: ["4"] },
|
||||
{ id: "code-fact", cat: "code", label: ["code"], max: 256, prompt: "Write a Python function factorial(n) that returns n!.", check: ["def", "factorial", "return"] },
|
||||
{ id: "qa-capital", cat: "qa", label: ["qa"], quick: true, max: 48, prompt: "What is the capital of France? One word.", check: ["paris|париж"] },
|
||||
{ id: "primes", cat: "qa", label: ["math", "qa"], max: 96, prompt: "List the first five prime numbers, comma-separated.", check: ["2", "3", "5", "7", "11"] },
|
||||
{ id: "instr-ok", cat: "instruction", label: null, quick: true, max: 32, prompt: "Reply with exactly the word OK and nothing else.", check: ["ok"] },
|
||||
{ id: "extract-email", cat: "extraction", label: ["extraction"], quick: true, max: 64, prompt: "Extract the email address from: 'Contact us at info@example.com today.' Reply with the address only.", check: ["info@example.com"] },
|
||||
{ id: "translate", cat: "translation", label: ["translation"], max: 64, prompt: "Translate to English: «Доброе утро». Reply with the translation only.", check: ["good morning"] },
|
||||
{ id: "format-json", cat: "instruction", label: null, max: 96, prompt: 'Return a JSON object with a single key "n" whose value is 7. JSON only.', check: ['"n"', "7"] },
|
||||
{ id: "summarize", cat: "summarization", label: ["summarization"], max: 96, prompt: "Summarize in one short sentence: 'The cat sat on the warm mat by the window. After a while it curled up and fell asleep in the sun.'", check: ["cat|кот|кошк"] },
|
||||
{ id: "creative", cat: "creative-writing", label: ["creative-writing"], max: 512, prompt: "Write a two-line poem about the sea.", check: ["sea|море|ocean"] },
|
||||
{ id: "chitchat", cat: "chitchat", label: ["chitchat"], quick: true, max: 48, prompt: "Hi! How are you today?", check: [] },
|
||||
// hard probes: dense enough that the router's complexity model rates them
|
||||
// high tier — this is where tier-based routing actually earns its keep.
|
||||
// Prompts are GENERATED fresh each run (domain + numbers rotate): the
|
||||
// router deliberately scores familiar text as less novel, so any fixed
|
||||
// wording would decay to medium/low over repeated runs.
|
||||
// label: null — ambiguity between task labels is exactly what makes these high
|
||||
{ id: "hard-science", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "science" },
|
||||
{ id: "hard-ops", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "ops" },
|
||||
];
|
||||
|
||||
// Fresh hard prompts per run. Two findings drive the design (measured against
|
||||
// the live router): ambiguity carries the biggest weight in the complexity
|
||||
// score, so INTERDISCIPLINARY prompts (science + engineering + planning) rate
|
||||
// high reliably, while single-domain ones classify too confidently and drop
|
||||
// to medium; and familiar text loses novelty on every repeat, so domains and
|
||||
// numbers rotate to keep each run lexically fresh.
|
||||
function hardPrompt(kind) {
|
||||
const pick = (a) => a[(Math.random() * a.length) | 0];
|
||||
const n = (lo, hi) => lo + ((Math.random() * (hi - lo)) | 0);
|
||||
const science = [
|
||||
() => `Architect a protein-folding inference pipeline for a wet lab: ensemble MSA generation with taxonomy-aware subsampling, template selection under a ${n(30, 60)} percent identity ceiling, distributed attention across ${n(4, 16)} GPUs with activation checkpointing, confidence calibration against experimental B-factors, and an active-learning loop that decides which of ${n(300, 2000)} candidate mutants to synthesize next week.`,
|
||||
() => `Schedule observations for a network of ${n(8, 30)} robotic telescopes chasing gravitational-wave counterparts: probability skymaps updated every ${n(5, 20)} minutes, slew-time-aware target assignment with decaying rewards, cloud-cover forecasts folded in as stochastic availability, spectroscopy triggered only above a ${n(40, 85)} percent kilonova likelihood, and a plan to validate the scheduler against the one historical multi-messenger event.`,
|
||||
() => `Run a closed-loop materials-discovery campaign for solid-state electrolytes: DFT screening of ${n(2000, 9000)} candidates, Bayesian optimization driving ${n(3, 12)} parallel synthesis robots, CNN-based XRD phase identification, an ionic-conductivity target of ${n(2, 25)} mS/cm, and an explicit decision policy for when simulation is trusted over synthesis.`,
|
||||
() => `Design a surface-code error-correction pipeline for a ${n(400, 3000)}-qubit superconducting processor: choose a code distance for a logical error target of 1e-${n(9, 14)} at physical error rate 1e-3, syndrome extraction avoiding hook errors, a real-time decoder under a ${n(1, 4)} microsecond budget, and the classical co-processor layout with feedback for lattice surgery.`,
|
||||
() => `Design a monitoring-reporting-verification pipeline for soil-carbon credits across ${n(500, 5000)} farms: satellite spectral indices calibrated against ${n(800, 4000)} physical core samples, hierarchical Bayesian estimation with shrinkage, additionality argued against county-level counterfactuals, uncertainty discounts priced into issuance, and an audit sampling plan catching over-reporting with ${n(90, 99)} percent power.`,
|
||||
() => `Plan a cryo-EM structure-determination campaign for a ${n(300, 900)} kDa membrane complex: grid-screening strategy across ${n(4, 12)} detergent conditions, particle picking with a distilled transformer at ${n(2, 9)} million particles, ab-initio reconstruction ambiguity resolved by tilt-pair validation, local-resolution-aware model building, and a decision rule for when to stop collecting and publish.`,
|
||||
];
|
||||
const ops = [
|
||||
() => `Model a dengue outbreak across ${n(20, 90)} districts by coupling an SEIR compartment model to vaccine cold-chain optimization: transmission parameters estimated from noisy case counts with a particle filter, uncertainty propagated into weekly demand forecasts, refrigerated routing under a ${n(120, 400)}-dose-per-crate constraint, and a defended ethical weighting for which ${n(3, 12)} districts get priority when supply covers ${n(20, 60)} percent of demand.`,
|
||||
() => `Co-optimize a regional power grid's day-ahead market with a fleet of ${n(200, 2000)} grid-scale batteries: bid curves learned from ${n(2, 6)} years of price data, unit commitment as mixed-integer programming with ramp constraints, degradation amortized into marginal costs, N-1 contingency screening after every dispatch, and a mechanism-design argument that the bidding rules cannot be gamed by a coalition controlling ${n(10, 30)} percent of storage.`,
|
||||
() => `Couple regional climate downscaling at ${n(2, 12)} km resolution to catastrophe-bond pricing: bias-correct ${n(10, 40)} CMIP ensemble members, generate stochastic hurricane tracks, propagate landfall distributions into a portfolio of ${n(30, 200)} bonds, price the attachment probabilities, and defend the discount rate applied to century-scale tail risk.`,
|
||||
() => `Co-design a hospital emergency department's triage policy with its nurse staffing schedule: arrivals as a time-varying Poisson process fitted to ${n(2, 6)} years of data, acuity drift as a hidden Markov chain, staffing as integer programming over ${n(8, 12)}-hour shifts, a queueing bound on door-to-doctor time at the ${n(90, 99)}th percentile, and a fairness constraint preventing systematic delays for low-acuity elderly patients.`,
|
||||
() => `Optimize a container port's berth allocation jointly with tidal windows and rail departures: vessel ETAs with heavy-tailed uncertainty, berth-crane assignment as a mixed-integer program re-solved hourly, tide-constrained draft limits for the ${n(3, 8)} deepest berths, a queueing model of the truck gate coupled to city traffic, and a robustness argument for surviving a ${n(24, 72)}-hour crane outage.`,
|
||||
() => `Schedule observations for a fleet of ${n(6, 40)} agricultural drones across ${n(200, 2000)} fields: multispectral revisit deadlines driven by crop growth-stage models, battery-swap logistics as a vehicle-routing problem with time windows, rain forecasts as stochastic field availability, disease-outbreak alerts pre-empting the plan, and a proof the replanner never starves a field past its agronomic deadline.`,
|
||||
];
|
||||
return pick(kind === "science" ? science : ops)();
|
||||
}
|
||||
|
||||
const LS_CUSTOM = "allaigate_bench_custom";
|
||||
const LS_HISTORY = "allaigate_bench_history";
|
||||
|
||||
function customTests() {
|
||||
try {
|
||||
return JSON.parse(localStorage.getItem(LS_CUSTOM) || "[]");
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
function saveCustom(list) {
|
||||
localStorage.setItem(LS_CUSTOM, JSON.stringify(list));
|
||||
}
|
||||
function history() {
|
||||
try {
|
||||
return JSON.parse(localStorage.getItem(LS_HISTORY) || "[]");
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
function pushHistory(run) {
|
||||
const l = history();
|
||||
l.unshift(run);
|
||||
localStorage.setItem(LS_HISTORY, JSON.stringify(l.slice(0, 12)));
|
||||
}
|
||||
function saveHistory(l) {
|
||||
localStorage.setItem(LS_HISTORY, JSON.stringify(l));
|
||||
}
|
||||
|
||||
// ── evaluation (ported from the old benchmark, trimmed) ─────────────────────
|
||||
function stripThink(text) {
|
||||
return text.replace(/<(?:\|)?(?:think|thought)(?:\|)?>[\s\S]*?(?:<\/(?:\|)?(?:think|thought)(?:\|)?>|$)/gi, "").trim();
|
||||
}
|
||||
// tail loop: the same 12+ char chunk repeated 3+ times back to back
|
||||
function looksLooped(text) {
|
||||
const tail = text.slice(-360);
|
||||
for (let len = 12; len <= 60; len += 8) {
|
||||
const probe = tail.slice(-len);
|
||||
if (probe && tail.split(probe).length - 1 >= 3) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
function evaluate(raw, test) {
|
||||
let text = stripThink(raw);
|
||||
if (!text && raw) text = raw; // max_tokens cut inside <think> — judge the raw
|
||||
const resp = text.toLowerCase();
|
||||
if (!resp.trim()) return { status: "fail", score: 0, note: t("bm.emptyAnswer") };
|
||||
if (looksLooped(resp)) return { status: "fail", score: 0, note: t("bm.loopDetected") };
|
||||
if (!test.check.length) return { status: "pass", score: 1, note: "✓" };
|
||||
let hit = 0;
|
||||
for (const cond of test.check) {
|
||||
const opts = cond.toLowerCase().split("|").map((s) => s.trim());
|
||||
if (opts.some((o) => o && resp.includes(o))) hit++;
|
||||
}
|
||||
const frac = hit / test.check.length;
|
||||
const status = frac >= 1 ? "pass" : frac >= 0.5 ? "partial" : "fail";
|
||||
return { status, score: frac, note: `${hit}/${test.check.length}` };
|
||||
}
|
||||
|
||||
export async function renderBenchmark() {
|
||||
const modelsData = await api.listModels();
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
const root = h("div");
|
||||
let running = false;
|
||||
let stopFlag = false;
|
||||
|
||||
let mode = "routing"; // routing (cortiq-auto, the product's feature) | models
|
||||
const modelCbs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox", checked: ids.length === 1 ? true : null }) }));
|
||||
const quickCb = h("input", { type: "checkbox", checked: true });
|
||||
const resultsHost = h("div", {});
|
||||
const progressHost = h("div", {});
|
||||
const historyHost = h("div", {});
|
||||
|
||||
const modeDesc = h("p", { class: "muted", style: "margin:8px 0 0" }, t("bm.mode.routingDesc"));
|
||||
const modelsPick = h("div", { class: "flex wrap", style: "gap:10px; margin-top:12px; display:none" },
|
||||
...modelCbs.map(({ id, cb }) => h("label", { class: "check" }, cb, h("span", { class: "mono" }, id))));
|
||||
const modeSeg = h("div", { class: "seg" },
|
||||
h("button", { class: "active" }, "🧭 " + t("bm.mode.routing")),
|
||||
h("button", {}, "🎯 " + t("bm.mode.models")));
|
||||
modeSeg.querySelectorAll("button").forEach((b, i) =>
|
||||
b.addEventListener("click", () => {
|
||||
mode = i === 0 ? "routing" : "models";
|
||||
modeSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === i));
|
||||
modelsPick.style.display = mode === "models" ? "" : "none";
|
||||
modeDesc.textContent = t(mode === "routing" ? "bm.mode.routingDesc" : "bm.mode.modelsDesc");
|
||||
}));
|
||||
|
||||
function suite() {
|
||||
const base = SUITE.concat(customTests());
|
||||
return quickCb.checked ? base.filter((x) => x.quick || x.custom) : base;
|
||||
}
|
||||
|
||||
const runBtn = h("button", { class: "btn primary" }, "▶ " + t("bm.run"));
|
||||
const stopBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
|
||||
stopBtn.addEventListener("click", () => {
|
||||
stopFlag = true;
|
||||
stopBtn.disabled = true;
|
||||
});
|
||||
|
||||
runBtn.addEventListener("click", async () => {
|
||||
const models = mode === "routing" ? ["cortiq-auto"] : modelCbs.filter((m) => m.cb.checked).map((m) => m.id);
|
||||
if (!models.length) {
|
||||
toast(t("bm.pickModel"), "warn");
|
||||
return;
|
||||
}
|
||||
running = true;
|
||||
stopFlag = false;
|
||||
runBtn.disabled = true;
|
||||
stopBtn.style.display = "";
|
||||
stopBtn.disabled = false;
|
||||
const tests = suite().filter((x) => mode === "routing" || x.judge !== "route");
|
||||
const run = { ts: Date.now(), mode, quick: quickCb.checked, models: {}, nTests: tests.length };
|
||||
|
||||
for (const model of models) {
|
||||
const rec = { results: [], score: 0, tokS: [], ttft: [], failed: false };
|
||||
run.models[model] = rec;
|
||||
for (let i = 0; i < tests.length; i++) {
|
||||
if (stopFlag) break;
|
||||
const test = tests[i];
|
||||
const prompt = test.gen ? hardPrompt(test.gen) : test.prompt;
|
||||
mount(
|
||||
progressHost,
|
||||
h("div", { class: "card" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "spinner" }), " ",
|
||||
h("b", { class: "mono" }, mode === "routing" ? t("bm.mode.routing") : model), " · ",
|
||||
t("bm.testOf", { i: i + 1, n: tests.length }), " · ",
|
||||
h("span", { class: "muted" }, test.id)),
|
||||
h("div", { class: "muted small", style: "margin-top:6px; font-style:italic" }, "«" + prompt.slice(0, 110) + (prompt.length > 110 ? "…»" : "»")),
|
||||
h("div", { class: "pbar", style: "margin-top:8px" },
|
||||
h("div", { class: "pbar-fill det", style: `width:${Math.round((i / tests.length) * 100)}%` })))
|
||||
);
|
||||
const started = performance.now();
|
||||
let text = "";
|
||||
try {
|
||||
const info = await testStream(
|
||||
// +256: headroom for reasoning models — a tight cap cuts them
|
||||
// mid-<think> and turns a correct model into a false ❌
|
||||
{ model, messages: [{ role: "user", content: prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
|
||||
(d) => (text += d)
|
||||
);
|
||||
const latency = performance.now() - started;
|
||||
let ev;
|
||||
if (test.judge === "route") {
|
||||
// the test asserts the ROUTER's decision: did it rate this high?
|
||||
// Two honest retries. (1) The SAME prompt after a pause: the
|
||||
// router judges difficulty in the background — the first-ever
|
||||
// occurrence routes by prior, the judgement lands in the cache a
|
||||
// couple of seconds later, so the second occurrence is the real
|
||||
// steady-state answer. (2) A fresh template, in case this one's
|
||||
// wording clustered into a burnt region.
|
||||
if (info.tier !== "high" && test.gen) {
|
||||
const attempts = [
|
||||
{ prompt, delayMs: 4000 },
|
||||
{ prompt: hardPrompt(test.gen), delayMs: 0 },
|
||||
];
|
||||
for (const a of attempts) {
|
||||
if (info.tier === "high") break;
|
||||
if (a.delayMs) await new Promise((r) => setTimeout(r, a.delayMs));
|
||||
try {
|
||||
const again = await testStream(
|
||||
{ model, messages: [{ role: "user", content: a.prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
|
||||
() => {}
|
||||
);
|
||||
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
|
||||
if (again.tier === "high") {
|
||||
info.tier = again.tier;
|
||||
info.score = again.score;
|
||||
info.selected_model = again.selected_model;
|
||||
info.task_label = again.task_label;
|
||||
info.route_source = again.route_source;
|
||||
}
|
||||
} catch { /* keep previous attempt */ }
|
||||
}
|
||||
}
|
||||
ev = info.tier === "high"
|
||||
? { status: "pass", score: 1, note: "high ✓" }
|
||||
: { status: "fail", score: 0, note: (info.tier || "—") + " ≠ high" };
|
||||
} else {
|
||||
ev = evaluate(text, test);
|
||||
}
|
||||
const compl = info.usage?.completion_tokens;
|
||||
if (test.judge !== "route" && ev.status === "fail" && !stripThink(text).trim() && compl >= (test.max || 128) + 250) {
|
||||
ev.note = t("bm.cutByCap"); // the whole budget went into <think>
|
||||
}
|
||||
if (compl && latency > 0) {
|
||||
const gen = info.ttft_ms != null ? Math.max(1, latency - info.ttft_ms) : latency;
|
||||
rec.tokS.push(compl / (gen / 1000));
|
||||
}
|
||||
if (info.ttft_ms != null) rec.ttft.push(info.ttft_ms);
|
||||
const row = { id: test.id, cat: test.cat, ...ev, latency: Math.round(latency), cost: info.cost_usd || 0, answer: stripThink(text).slice(0, 300) };
|
||||
if (mode === "routing") {
|
||||
// router timed out / fell back — there was no classification to
|
||||
// judge, so it must not count against accuracy
|
||||
row.degraded = !info.task_label
|
||||
|| info.task_label === "degraded"
|
||||
|| String(info.route_source || "").includes("degraded");
|
||||
row.task = info.task_label || "—";
|
||||
row.tier = info.tier || "—";
|
||||
row.cscore = info.score || 0; // непрерывный коэффициент сложности
|
||||
// (row.score занят долей правильности ответа из evaluate())
|
||||
row.chosen = info.selected_model || "—";
|
||||
row.source = info.route_source || "—";
|
||||
row.expected = test.label; // null = classification not judged
|
||||
row.taskOk = test.label && !row.degraded ? test.label.includes(info.task_label) : null;
|
||||
}
|
||||
rec.results.push(row);
|
||||
} catch (e) {
|
||||
rec.results.push({ id: test.id, cat: test.cat, status: "fail", score: 0, note: String(e.message || e).slice(0, 80), latency: 0, answer: "" });
|
||||
rec.failed = true;
|
||||
}
|
||||
renderResults(run, true); // live: строка появляется сразу после теста
|
||||
}
|
||||
if (stopFlag) break;
|
||||
}
|
||||
|
||||
mount(progressHost);
|
||||
running = false;
|
||||
runBtn.disabled = false;
|
||||
stopBtn.style.display = "none";
|
||||
if (!stopFlag) {
|
||||
pushHistory(run);
|
||||
// финальный аккорд: короткий вердикт с процентами и деньгами
|
||||
const first = run.models[Object.keys(run.models)[0]];
|
||||
if (first) {
|
||||
const st = stats(first);
|
||||
const judged = first.results.filter((r) => r.taskOk != null);
|
||||
const cls = judged.length ? ` · ${judged.filter((r) => r.taskOk).length}/${judged.length}` : "";
|
||||
const total = Object.values(run.models).reduce((a, rec) => a + stats(rec).cost, 0);
|
||||
toast(t("bm.verdict", { p: Math.round(st.score * 100) + "%" + cls, s: money(total) }), "ok", 6000);
|
||||
}
|
||||
}
|
||||
renderResults(run);
|
||||
renderHistory();
|
||||
});
|
||||
|
||||
function statusIcon(s) {
|
||||
return s === "pass" ? "✅" : s === "partial" ? "◐" : "❌";
|
||||
}
|
||||
|
||||
function stats(rec) {
|
||||
const n = rec.results.length || 1;
|
||||
return {
|
||||
score: rec.results.reduce((a, r) => a + r.score, 0) / n,
|
||||
tokS: rec.tokS.length ? rec.tokS.reduce((a, b) => a + b, 0) / rec.tokS.length : null,
|
||||
ttft: rec.ttft.length ? rec.ttft.reduce((a, b) => a + b, 0) / rec.ttft.length : null,
|
||||
cost: rec.results.reduce((a, r) => a + (r.cost || 0), 0),
|
||||
};
|
||||
}
|
||||
|
||||
function renderResults(run, live) {
|
||||
const models = Object.keys(run.models);
|
||||
if (!models.length) {
|
||||
mount(resultsHost);
|
||||
return;
|
||||
}
|
||||
const routing = run.mode === "routing";
|
||||
const detail = h("div", {});
|
||||
|
||||
let summary;
|
||||
if (routing) {
|
||||
const rec = run.models[models[0]];
|
||||
const st = stats(rec);
|
||||
const judged = rec.results.filter((r) => r.taskOk != null);
|
||||
const okCount = judged.filter((r) => r.taskOk).length;
|
||||
// какие модели реально выбирал роутер
|
||||
const dist = {};
|
||||
for (const r of rec.results) {
|
||||
if (!r.chosen) continue;
|
||||
dist[r.chosen] = dist[r.chosen] || { n: 0, cost: 0 };
|
||||
dist[r.chosen].n++;
|
||||
dist[r.chosen].cost += r.cost || 0;
|
||||
}
|
||||
const pct = Math.round(st.score * 100);
|
||||
summary = h("div", {},
|
||||
h("div", { class: "kpis", style: "grid-template-columns: repeat(auto-fit, minmax(150px, 1fr))" },
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, pct + "%"), h("div", { class: "lbl" }, t("bm.answersOk"))),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, judged.length ? `${okCount}/${judged.length}` : "—"), h("div", { class: "lbl" }, t("bm.classAcc"))),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, st.ttft != null ? ms(Math.round(st.ttft)) : "—"), h("div", { class: "lbl" }, t("pg.ttft"))),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, st.tokS != null ? st.tokS.toFixed(1) : "—"), h("div", { class: "lbl" }, "tok/s")),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, moneyEl(st.cost)), h("div", { class: "lbl" }, t("bm.spent")))),
|
||||
h("div", { class: "flex wrap", style: "gap:6px; margin-top:4px" },
|
||||
h("span", { class: "muted small" }, t("bm.modelDist") + ":"),
|
||||
...Object.entries(dist).map(([m, d]) =>
|
||||
h("span", { class: "badge" }, `${m} × ${d.n}`, d.cost > 0 ? " · " : "", d.cost > 0 ? moneyEl(d.cost) : ""))));
|
||||
} else {
|
||||
const sorted = models.slice().sort((a, b) => stats(run.models[b]).score - stats(run.models[a]).score);
|
||||
summary = h("div", {},
|
||||
...sorted.map((m) => {
|
||||
const st = stats(run.models[m]);
|
||||
const pct = Math.round(st.score * 100);
|
||||
return h("div", { class: "bm-sum" },
|
||||
h("div", { class: "flex" },
|
||||
h("b", { class: "mono grow" }, m),
|
||||
h("span", { class: "badge " + (pct >= 80 ? "ok" : pct >= 50 ? "" : "bad") }, pct + "%"),
|
||||
st.tokS != null ? h("span", { class: "muted small", style: "margin-left:8px" }, st.tokS.toFixed(1) + " tok/s") : null,
|
||||
st.ttft != null ? h("span", { class: "muted small", style: "margin-left:8px" }, "TTFT " + ms(Math.round(st.ttft))) : null,
|
||||
st.cost > 0 ? h("span", { class: "muted small", style: "margin-left:8px" }, t("bm.spent") + " ", moneyEl(st.cost)) : null),
|
||||
h("div", { class: "sysbar" }, h("i", { class: pct < 50 ? "bad" : "", style: `width:${pct}%` })));
|
||||
}));
|
||||
}
|
||||
|
||||
function answerCell(r, m) {
|
||||
const cell = h("td", { class: "bm-cell", title: r.answer || r.note },
|
||||
statusIcon(r.status) + " ",
|
||||
h("span", { class: "muted small" }, r.note + (r.latency ? ` · ${(r.latency / 1000).toFixed(1)}s` : "")));
|
||||
cell.addEventListener("click", () => {
|
||||
mount(detail, h("div", { class: "card" },
|
||||
h("div", { class: "card-title" }, `${r.id} · ${m} · ${statusIcon(r.status)}`),
|
||||
h("pre", { class: "conv-log", style: "max-height:200px" }, r.answer || "—")));
|
||||
});
|
||||
return cell;
|
||||
}
|
||||
|
||||
let header, rows;
|
||||
if (routing) {
|
||||
const rec = run.models[models[0]];
|
||||
header = h("tr", {},
|
||||
h("th", {}, t("bm.test")),
|
||||
h("th", {}, t("pg.task")),
|
||||
h("th", {}, t("pg.tier")),
|
||||
h("th", {}, t("pg.model")),
|
||||
h("th", {}, t("bm.answer")));
|
||||
rows = rec.results.map((r) =>
|
||||
h("tr", {},
|
||||
h("td", {}, h("div", {}, r.id), h("div", { class: "muted small" }, r.cat)),
|
||||
h("td", {},
|
||||
r.degraded
|
||||
? h("span", { class: "muted", title: t("bm.routerDegraded") }, "⚠ " + t("bm.degraded"))
|
||||
: r.taskOk == null
|
||||
? h("span", { class: "muted" }, r.task || "—")
|
||||
: h("span", { class: r.taskOk ? "" : "err" },
|
||||
(r.taskOk ? "✓ " : "✗ ") + r.task,
|
||||
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
|
||||
h("td", {},
|
||||
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
|
||||
h("td", {},
|
||||
h("span", { class: "mono small" }, r.chosen || "—"),
|
||||
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
|
||||
r.source && r.source.includes("task-rule") ? h("div", { class: "badge", style: "margin-top:2px" }, "task-rule") : null),
|
||||
answerCell(r, r.chosen || "cortiq-auto")));
|
||||
} else {
|
||||
header = h("tr", {}, h("th", {}, t("bm.test")), ...models.map((m) => h("th", { class: "mono" }, m)));
|
||||
rows = (run.models[models[0]].results || []).map((_, i) =>
|
||||
h("tr", {},
|
||||
h("td", {},
|
||||
h("div", {}, run.models[models[0]].results[i].id),
|
||||
h("div", { class: "muted small" }, run.models[models[0]].results[i].cat)),
|
||||
...models.map((m) => {
|
||||
const r = run.models[m].results[i];
|
||||
return r ? answerCell(r, m) : h("td", {}, "—");
|
||||
})));
|
||||
}
|
||||
|
||||
mount(resultsHost,
|
||||
h("div", { class: "card" },
|
||||
h("div", { class: "card-head" },
|
||||
h("h3", {}, (routing ? "🧭 " : "🎯 ") + t("bm.results")),
|
||||
h("span", { class: "card-head sub" },
|
||||
(live ? t("bm.liveRunning") + " · " : "") + new Date(run.ts).toLocaleString() + (run.quick ? " · " + t("bm.quick") : ""))),
|
||||
summary,
|
||||
h("div", { class: "table-wrap", style: "margin-top:14px" }, h("table", {}, h("thead", {}, header), h("tbody", {}, ...rows))),
|
||||
h("div", { class: "muted small", style: "margin-top:6px" }, t("bm.cellHint"))),
|
||||
detail);
|
||||
}
|
||||
|
||||
function renderHistory() {
|
||||
const l = history();
|
||||
if (!l.length) {
|
||||
mount(historyHost);
|
||||
return;
|
||||
}
|
||||
const clearAll = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("bm.clearHistory"));
|
||||
clearAll.addEventListener("click", () => {
|
||||
saveHistory([]);
|
||||
renderHistory();
|
||||
});
|
||||
mount(historyHost, h("div", { class: "card" },
|
||||
h("div", { class: "card-head" },
|
||||
h("h3", {}, t("bm.history")),
|
||||
h("div", { class: "right" }, clearAll)),
|
||||
...l.map((run, idx) => {
|
||||
const models = Object.keys(run.models);
|
||||
const scoreOf = (m) => {
|
||||
const rs = run.models[m].results || [];
|
||||
return rs.length ? rs.reduce((a, r) => a + r.score, 0) / rs.length : 0;
|
||||
};
|
||||
const best = models.slice().sort((a, b) => scoreOf(b) - scoreOf(a))[0];
|
||||
const row = h("div", { class: "flex bench-row", style: "cursor:pointer" },
|
||||
h("span", {}, run.mode === "routing" ? "🧭" : "🎯"),
|
||||
h("span", { class: "muted small" }, new Date(run.ts).toLocaleString()),
|
||||
h("span", { class: "grow mono small" }, run.mode === "routing" ? t("bm.mode.routing") : models.join(", ")),
|
||||
run.quick ? h("span", { class: "badge" }, t("bm.quick")) : null,
|
||||
best ? h("b", {}, Math.round(scoreOf(best) * 100) + "%") : null,
|
||||
h("button", { class: "btn sm ghost danger", title: t("common.delete"), onClick: (e) => {
|
||||
e.stopPropagation();
|
||||
l.splice(idx, 1);
|
||||
saveHistory(l);
|
||||
renderHistory();
|
||||
} }, "×"));
|
||||
row.addEventListener("click", () => renderResults(run));
|
||||
return row;
|
||||
})));
|
||||
}
|
||||
|
||||
// ── custom tests (stored in the browser) ──
|
||||
function customCard() {
|
||||
const host = h("div", { class: "card" });
|
||||
function paint() {
|
||||
const list = customTests();
|
||||
const promptIn = h("input", { placeholder: t("bm.promptPh") });
|
||||
const checkIn = h("input", { placeholder: t("bm.checkPh") });
|
||||
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("bm.addTest"));
|
||||
addBtn.addEventListener("click", () => {
|
||||
const prompt = promptIn.value.trim();
|
||||
const check = checkIn.value.split(",").map((s) => s.trim()).filter(Boolean);
|
||||
if (!prompt || !check.length) return;
|
||||
list.push({ id: "custom-" + (list.length + 1), cat: "custom", custom: true, max: 128, prompt, check });
|
||||
saveCustom(list);
|
||||
paint();
|
||||
});
|
||||
mount(host,
|
||||
h("div", { class: "card-title" }, t("bm.custom")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bm.customHint")),
|
||||
...list.map((tst, i) =>
|
||||
h("div", { class: "flex bench-row" },
|
||||
h("span", { class: "grow small" }, tst.prompt.slice(0, 90)),
|
||||
h("span", { class: "muted small mono" }, tst.check.join(", ").slice(0, 40)),
|
||||
h("button", { class: "btn sm ghost danger", onClick: () => { list.splice(i, 1); saveCustom(list); paint(); } }, "×"))),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-top:8px" }, promptIn, checkIn, addBtn));
|
||||
}
|
||||
paint();
|
||||
return host;
|
||||
}
|
||||
|
||||
mount(root,
|
||||
h("div", { class: "page-head" },
|
||||
h("h2", {}, t("bm.title")),
|
||||
h("p", {}, t("bm.subtitle"))),
|
||||
ids.length === 0 ? h("div", { class: "callout" }, t("routing.empty")) : null,
|
||||
h("div", { class: "card" },
|
||||
modeSeg,
|
||||
modeDesc,
|
||||
modelsPick,
|
||||
h("div", { class: "flex", style: "justify-content:space-between; margin-top:14px" },
|
||||
h("label", { class: "check", style: "margin:0", title: t("bm.quickHint") }, quickCb, "⚡ " + t("bm.quickMode")),
|
||||
h("div", { class: "flex", style: "gap:8px" }, stopBtn, runBtn)),
|
||||
h("div", { class: "muted small", style: "margin-top:6px" },
|
||||
t("bm.suiteInfo", { q: SUITE.filter((x) => x.quick).length, f: SUITE.length }))),
|
||||
progressHost,
|
||||
resultsHost,
|
||||
customCard(),
|
||||
historyHost);
|
||||
renderHistory();
|
||||
return root;
|
||||
}
|
||||
Vendored
+142
-13
@@ -1,8 +1,9 @@
|
||||
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
|
||||
import { h, mount, money, num, ms, pct, timeAgo } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { lineChart, barList } from "../charts.js";
|
||||
import { SITE_URL } from "../app.js";
|
||||
|
||||
const RANGES = ["1h", "24h", "7d"];
|
||||
const GROUPS = ["model", "tier", "label", "account"];
|
||||
@@ -26,14 +27,96 @@ export async function renderDashboard() {
|
||||
let groupby = "model";
|
||||
const root = h("div");
|
||||
|
||||
// ── host metrics: CPU sparkline + RAM / disk bars, refreshed every 2 s ──
|
||||
const sysHost = h("div", {});
|
||||
const cpuHistory = []; // last ~60 samples (2 min)
|
||||
function fmtG(n) {
|
||||
return (n / 1e9).toFixed(1) + " GB";
|
||||
}
|
||||
function bar(frac, cls) {
|
||||
return h("div", { class: "sysbar" }, h("i", { class: cls || "", style: `width:${Math.min(100, frac * 100)}%` }));
|
||||
}
|
||||
function sparkline(values, w, hgt) {
|
||||
const c = document.createElement("canvas");
|
||||
c.width = w;
|
||||
c.height = hgt;
|
||||
const ctx = c.getContext("2d");
|
||||
const accent = getComputedStyle(document.documentElement).getPropertyValue("--accent").trim() || "#5b8def";
|
||||
ctx.strokeStyle = accent;
|
||||
ctx.lineWidth = 1.5;
|
||||
ctx.beginPath();
|
||||
const n = Math.max(values.length, 2);
|
||||
values.forEach((v, i) => {
|
||||
const x = (i / (n - 1)) * (w - 2) + 1;
|
||||
const y = hgt - 2 - (Math.min(v, 100) / 100) * (hgt - 4);
|
||||
if (i === 0) ctx.moveTo(x, y);
|
||||
else ctx.lineTo(x, y);
|
||||
});
|
||||
ctx.stroke();
|
||||
return c;
|
||||
}
|
||||
async function refreshSystem() {
|
||||
let sys;
|
||||
try {
|
||||
sys = await api.system();
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!sys.ts) return;
|
||||
cpuHistory.push(sys.cpu_percent || 0);
|
||||
if (cpuHistory.length > 60) cpuHistory.shift();
|
||||
const ramFrac = sys.ram_total ? sys.ram_used / sys.ram_total : 0;
|
||||
const diskFrac = sys.disk_total ? (sys.disk_total - sys.disk_free) / sys.disk_total : 0;
|
||||
mount(
|
||||
sysHost,
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-title" }, t("dash.sys.title")),
|
||||
h(
|
||||
"div",
|
||||
{ class: "sysgrid" },
|
||||
h("div", { class: "sysitem" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "k grow" }, `CPU · ${sys.cores} ${t("dash.sys.cores")}`),
|
||||
h("span", { class: "v mono" }, (sys.cpu_percent || 0).toFixed(0) + "%")),
|
||||
sparkline(cpuHistory, 220, 36),
|
||||
h("div", { class: "muted small" }, `load ${Number(sys.load1 || 0).toFixed(1)}`)),
|
||||
h("div", { class: "sysitem" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "k grow" }, "RAM"),
|
||||
h("span", { class: "v mono" }, `${fmtG(sys.ram_used)} / ${fmtG(sys.ram_total)}`)),
|
||||
bar(ramFrac, ramFrac > 0.9 ? "bad" : ""),
|
||||
h("div", { class: "muted small" }, t("dash.sys.gateway") + " " + (sys.process_rss / 1e6).toFixed(0) + " MB")),
|
||||
h("div", { class: "sysitem" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "k grow" }, t("dash.sys.disk")),
|
||||
h("span", { class: "v mono" }, t("dash.sys.free") + " " + fmtG(sys.disk_free)),
|
||||
),
|
||||
bar(diskFrac, diskFrac > 0.92 ? "bad" : ""),
|
||||
h("div", { class: "muted small" }, `${fmtG(sys.disk_total - sys.disk_free)} / ${fmtG(sys.disk_total)}`))
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
const sysTimer = setInterval(() => {
|
||||
if (!root.isConnected) {
|
||||
clearInterval(sysTimer);
|
||||
return;
|
||||
}
|
||||
refreshSystem();
|
||||
}, 2000);
|
||||
refreshSystem();
|
||||
|
||||
async function load() {
|
||||
const rangeSecs = range;
|
||||
let stats, recent, health;
|
||||
let stats, recent, health, balance;
|
||||
try {
|
||||
[stats, recent, health] = await Promise.all([
|
||||
[stats, recent, health, balance] = await Promise.all([
|
||||
api.stats(rangeSecs, groupby),
|
||||
api.requests(40, 0),
|
||||
api.health().catch(() => null),
|
||||
api.balance().catch(() => null),
|
||||
]);
|
||||
} catch (e) {
|
||||
mount(root, h("div", { class: "callout" }, String(e.message || e)));
|
||||
@@ -52,10 +135,24 @@ export async function renderDashboard() {
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
h("div", { class: "grow" }, h("h2", {}, t("dash.title")), h("p", {}, t("dash.subtitle"))),
|
||||
seg(RANGES, range, (r) => {
|
||||
range = r;
|
||||
load();
|
||||
}, (r) => t("dash.range." + r))
|
||||
h("div", { class: "flex", style: "gap:8px;align-items:center" },
|
||||
seg(RANGES, range, (r) => {
|
||||
range = r;
|
||||
load();
|
||||
}, (r) => t("dash.range." + r)),
|
||||
h("button", {
|
||||
class: "btn sm danger",
|
||||
onClick: async () => {
|
||||
if (!(await confirmDialog(t("dash.clearConfirm")))) return;
|
||||
try {
|
||||
await api.clearStats();
|
||||
toast(t("dash.cleared"), "good");
|
||||
await load();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
},
|
||||
}, t("dash.clear")))
|
||||
)
|
||||
),
|
||||
|
||||
@@ -65,12 +162,19 @@ export async function renderDashboard() {
|
||||
{ class: "kpis" },
|
||||
kpi(num(tot.requests), t("dash.kpi.requests")),
|
||||
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
|
||||
kpi(money(tot.cost_usd), t("dash.kpi.cost")),
|
||||
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
|
||||
balance && balance.openrouter_usd != null
|
||||
? kpi(
|
||||
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
|
||||
t("dash.kpi.orBalance"))
|
||||
: null,
|
||||
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
|
||||
kpi(pct(tot.success_rate), t("dash.kpi.success")),
|
||||
kpi(num(tot.failovers), t("dash.kpi.failovers"))
|
||||
),
|
||||
|
||||
sysHost,
|
||||
|
||||
h(
|
||||
"div",
|
||||
{ class: "grid cols-2" },
|
||||
@@ -173,7 +277,9 @@ function recentTable(rows) {
|
||||
{},
|
||||
h("td", { class: "mono" }, timeAgo(r.ts)),
|
||||
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
|
||||
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
|
||||
h("td", {},
|
||||
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
|
||||
h("td", {}, r.task_label || "—"),
|
||||
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
|
||||
h("td", { class: "mono" }, money(r.cost_usd)),
|
||||
@@ -186,7 +292,19 @@ function recentTable(rows) {
|
||||
);
|
||||
}
|
||||
|
||||
function keySourceLabel(src) {
|
||||
if (src === "store") return t("models.form.secretStored");
|
||||
if (src === "env") return t("models.form.secretEnv");
|
||||
return t("models.form.secretMissing");
|
||||
}
|
||||
|
||||
function healthCard(hh) {
|
||||
const r = hh.router || {};
|
||||
const keySrc = r.key_source || "none";
|
||||
const err = r.last_error;
|
||||
// problems fixed on the site (get / pay for a key) get the top-up link
|
||||
const needsSite =
|
||||
!r.reachable || keySrc === "missing" || (err && ["auth", "payment", "quota"].includes(err.kind));
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
@@ -196,9 +314,20 @@ function healthCard(hh) {
|
||||
{ class: "flex wrap", style: "margin-bottom:12px" },
|
||||
h(
|
||||
"span",
|
||||
{ class: "badge " + (hh.router?.reachable ? "ok" : "bad"), title: hh.router?.url || "" },
|
||||
t("status.router") + ": " + (hh.router?.reachable ? t("dash.health.reachable") : t("dash.health.unreachable"))
|
||||
)
|
||||
{ class: "badge " + (r.reachable ? "ok" : "bad"), title: r.url || "" },
|
||||
t("status.router") + ": " + (r.reachable ? t("dash.health.reachable") : t("dash.health.unreachable"))
|
||||
),
|
||||
keySrc !== "none"
|
||||
? h(
|
||||
"span",
|
||||
{ class: "badge " + keyBadge(keySrc), title: r.key_env || "" },
|
||||
t("models.col.key") + ": " + keySourceLabel(keySrc)
|
||||
)
|
||||
: null,
|
||||
err ? h("span", { class: "badge bad" }, t("router.probe." + (err.kind || "error"))) : null,
|
||||
needsSite
|
||||
? h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("dash.health.getKey") + " ↗")
|
||||
: null
|
||||
),
|
||||
h(
|
||||
"div",
|
||||
|
||||
Vendored
+205
@@ -0,0 +1,205 @@
|
||||
// Ready-to-run .cmf models, read live from the infosave HuggingFace account via
|
||||
// /setup/featured — so a newly published model shows up without a code change.
|
||||
// Shared by the first-run wizard and the Import view: same cards, same download
|
||||
// / progress / delete behaviour in both places.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export async function loadFeatured() {
|
||||
try {
|
||||
const r = await api.setupFeatured();
|
||||
return r.models || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export function quantFromVariant(name) {
|
||||
const s = (name || "").toLowerCase();
|
||||
if (s.includes("q1t")) return "Q1T";
|
||||
if (s.includes("q2tp")) return "Q2TP";
|
||||
if (s.includes("q4tp")) return "Q4TP";
|
||||
if (s.includes("q4t")) return "Q4T";
|
||||
if (s.includes("q1")) return "Q1";
|
||||
if (s.includes("q4")) return "Q4";
|
||||
if (s.includes("q8")) return "Q8";
|
||||
return "";
|
||||
}
|
||||
|
||||
// Same thresholds as the backend's format_bytes, so card and job agree.
|
||||
export function fmtBytes(n) {
|
||||
if (!n && n !== 0) return "";
|
||||
if (n >= 1e9) return (n / 1e9).toFixed(1) + " GB";
|
||||
if (n >= 1e6) return Math.round(n / 1e6) + " MB";
|
||||
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
|
||||
return n + " B";
|
||||
}
|
||||
|
||||
export function jobPct(j) {
|
||||
if (j.progress == null) return null;
|
||||
return Math.max(0, Math.min(100, Math.round(j.progress * 100)));
|
||||
}
|
||||
|
||||
/// "downloading · 42% · 1.4 / 3.2 GB" — whatever of it the job actually knows.
|
||||
/// A settled job shows its result, not the phase it stopped in.
|
||||
export function jobStatusText(j) {
|
||||
const parts = [];
|
||||
if (j.state === "running") {
|
||||
if (j.phase) parts.push(j.phase);
|
||||
const pct = jobPct(j);
|
||||
if (pct != null) parts.push(pct + "%");
|
||||
if (j.total_bytes) parts.push(`${fmtBytes(j.done_bytes || 0)} / ${fmtBytes(j.total_bytes)}`);
|
||||
} else if (j.state === "done" && j.size_bytes) {
|
||||
parts.push(fmtBytes(j.size_bytes));
|
||||
}
|
||||
return parts.join(" · ");
|
||||
}
|
||||
|
||||
export function jobBar(j) {
|
||||
const pct = jobPct(j);
|
||||
if (j.state === "done") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill ok", style: "width:100%" }));
|
||||
if (j.state !== "running") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill err", style: "width:100%" }));
|
||||
// known percentage → determinate fill; unknown (listing files) → animated slider
|
||||
return h("div", { class: "pbar" },
|
||||
pct != null
|
||||
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
|
||||
: h("div", { class: "pbar-fill run" })
|
||||
);
|
||||
}
|
||||
|
||||
/// HF gives `desc` as a bare pipeline tag ("text-generation") for live entries and
|
||||
/// a real sentence for curated ones. A one-word tag belongs in the badge row, not
|
||||
/// on a line of its own under every card.
|
||||
function descBadge(desc) {
|
||||
return desc && !desc.includes(" ") ? h("span", { class: "badge" }, desc) : null;
|
||||
}
|
||||
function descLine(desc) {
|
||||
return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null;
|
||||
}
|
||||
|
||||
/// Disk/RAM sanity for a model of `bytes` size against the live system
|
||||
/// snapshot (ctx.sys): running a model needs it in RAM; downloading needs disk.
|
||||
function fitWarnings(bytes, sys) {
|
||||
const out = [];
|
||||
if (!bytes || !sys) return out;
|
||||
if (sys.disk_free && bytes > sys.disk_free)
|
||||
out.push(h("span", { class: "badge bad", title: t("fit.diskTitle") }, "⚠ " + t("fit.disk")));
|
||||
else if (sys.ram_total && bytes > sys.ram_total * 0.75)
|
||||
out.push(h("span", { class: "badge warn", title: t("fit.ramTitle") }, "⚠ " + t("fit.ram")));
|
||||
return out;
|
||||
}
|
||||
|
||||
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v), sys? }.
|
||||
/// Live progress lands in the per-variant host so a poll can repaint it in place,
|
||||
/// without re-rendering the grid under the user's cursor.
|
||||
export function featuredCard(m, ctx) {
|
||||
const badge = m.badge || "CMF";
|
||||
const label = m.label || m.id;
|
||||
const desc = m.desc || "";
|
||||
const variants = m.variants && m.variants.length ? m.variants : null;
|
||||
const size = m.size || (m.size_bytes ? fmtBytes(m.size_bytes) : "");
|
||||
const isDownloaded = !!m.downloaded;
|
||||
const progressHost = (v) => {
|
||||
const host = h("div", { class: "card-prog" });
|
||||
const jid = (v && v.job) || m.job;
|
||||
if (jid) ctx.hosts.set(jid, host);
|
||||
return host;
|
||||
};
|
||||
// multi-variant (e.g. DeepSeek with parts-q2tp / parts-q4tp) → separate buttons per variant
|
||||
if (variants && variants.length > 1) {
|
||||
return h("div", { class: "hf-card featured-card" + (isDownloaded ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, isDownloaded ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
descBadge(desc),
|
||||
...fitWarnings(m.size_bytes, ctx.sys),
|
||||
isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; flex-direction:column; gap:6px; margin-top:8px" },
|
||||
...variants.map((v) => {
|
||||
const q = quantFromVariant(v.name) || badge;
|
||||
const done = !!v.downloaded;
|
||||
const upd = !!v.update_available;
|
||||
const lab = done
|
||||
? (upd ? `↻ ${t("onboarding.update")} ${q} · ${v.size}` : `✓ ${q} · ${v.size}`)
|
||||
: `⬇ ${q} · ${v.size}`;
|
||||
const btn = h("button", {
|
||||
class: "btn sm " + (done && !upd ? "ghost" : "primary"),
|
||||
disabled: done && !upd ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v),
|
||||
title: v.name + (done ? " — " + t("onboarding.downloaded") : ""),
|
||||
}, lab);
|
||||
const row = h("div", { style: "display:flex; gap:6px; align-items:center; flex-wrap:wrap" }, btn);
|
||||
if (done) {
|
||||
const del = h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑");
|
||||
del.addEventListener("click", () => ctx.onDelete(m, v));
|
||||
row.appendChild(del);
|
||||
}
|
||||
return h("div", {}, row, progressHost(v));
|
||||
})
|
||||
)
|
||||
);
|
||||
}
|
||||
const v0 = variants && variants[0];
|
||||
const done0 = !!(v0 && v0.downloaded) || isDownloaded;
|
||||
const upd0 = !!(v0 && v0.update_available) || !!m.update_available;
|
||||
const q0 = quantFromVariant((v0 && v0.name) || "") || badge;
|
||||
const size0 = (v0 && v0.size) || size;
|
||||
const btn0 = h("button", {
|
||||
class: "btn sm " + (done0 && !upd0 ? "ghost" : "primary"),
|
||||
disabled: done0 && !upd0 ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v0),
|
||||
title: done0 && !upd0 ? t("onboarding.downloaded") : ""
|
||||
}, done0
|
||||
? (upd0 ? `↻ ${t("onboarding.update")} ${q0} · ${size0}` : `✓ ${q0} · ${size0}`)
|
||||
: `⬇ ${q0}${size0 ? " · " + size0 : ""}`);
|
||||
const del0 = done0 ? h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete")) : null;
|
||||
if (del0) del0.addEventListener("click", () => ctx.onDelete(m, v0));
|
||||
return h("div", { class: "hf-card featured-card" + (done0 ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, done0 ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null,
|
||||
descBadge(desc),
|
||||
...(done0 ? [] : fitWarnings((v0 && v0.size_bytes) || m.size_bytes, ctx.sys))
|
||||
),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; gap:6px; margin-top:8px; align-items:center; flex-wrap:wrap" }, btn0, del0),
|
||||
progressHost(v0)
|
||||
);
|
||||
}
|
||||
|
||||
/// Repaint the per-variant progress hosts from the current job list.
|
||||
export function paintFeaturedHosts(hosts, jobs, onCancel) {
|
||||
const byId = new Map(jobs.map((j) => [j.id, j]));
|
||||
for (const [id, host] of hosts) {
|
||||
const j = byId.get(id);
|
||||
if (!j || j.state !== "running") { mount(host); continue; }
|
||||
const cancel = h("button", { class: "btn sm ghost", style: "margin-left:auto", title: t("import.cancel") }, "✕");
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await onCancel(j); } catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
mount(host,
|
||||
jobBar(j),
|
||||
h("div", { style: "display:flex; align-items:center; gap:8px" },
|
||||
h("span", { class: "muted small" }, jobStatusText(j)),
|
||||
cancel
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Start the download for a featured variant. `local_name` is the stem the
|
||||
/// gateway itself derived, sent back verbatim so download / progress / delete
|
||||
/// can never disagree about which file this is.
|
||||
export async function startFeaturedDownload(m, v) {
|
||||
const fallbackBase = m.id.split("/").pop().toLowerCase().replace(/[^a-z0-9_-]/g, "-");
|
||||
const name = (v && v.local_name) || m.local_name || fallbackBase;
|
||||
const variant = v && v.name ? v.name : undefined;
|
||||
await api.startImport({ repo: m.id, quant: "Q1", name, variant });
|
||||
return variant;
|
||||
}
|
||||
Vendored
+459
@@ -0,0 +1,459 @@
|
||||
// Import a HuggingFace model → convert to a local .cmf (our format) with a
|
||||
// live progress view. Search HF, pick quantization + advanced params, watch
|
||||
// the conversion (multiple in parallel, resumable across tab switches), then
|
||||
// register the result as a local model.
|
||||
import { h, mount, toast, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobStatusText,
|
||||
} from "./featured.js";
|
||||
|
||||
const QUANTS = [
|
||||
["Q8_2F", "import.q.q8_2f"],
|
||||
["Q8_ROW", "import.q.q8_row"],
|
||||
["Q4_BLOCK", "import.q.q4"],
|
||||
["Q4_TILED", "import.q.q4_tiled"],
|
||||
["VBIT", "import.q.vbit"],
|
||||
["Q1T", "import.q.q1t"],
|
||||
["Q1S", "import.q.q1s"],
|
||||
["Q1P", "import.q.q1p"],
|
||||
["Q1", "import.q.q1"],
|
||||
["F16", "import.q.f16"],
|
||||
["F32", "import.q.f32"],
|
||||
];
|
||||
|
||||
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control,
|
||||
hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function fmtNum(n) {
|
||||
if (n == null) return "—";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
|
||||
if (n >= 1e3) return (n / 1e3).toFixed(1) + "k";
|
||||
return String(n);
|
||||
}
|
||||
function fmtBytes(n) {
|
||||
if (!n) return "";
|
||||
const g = n / 1e9;
|
||||
return g >= 1 ? g.toFixed(2) + " GB" : (n / 1e6).toFixed(0) + " MB";
|
||||
}
|
||||
// state → localized label (falls back to the raw state)
|
||||
function stateLabel(s) {
|
||||
const k = "import.state." + s;
|
||||
const v = t(k);
|
||||
return v === k ? s : v;
|
||||
}
|
||||
|
||||
export function renderImport() {
|
||||
let selected = null;
|
||||
let lastModels = []; // cache of the last search, so re-highlighting needs no refetch
|
||||
let jobsTimer = null;
|
||||
let featured = []; // ready-made .cmf models from the infosave HF account
|
||||
let featuredHosts = new Map(); // job id → live progress slot inside a featured card
|
||||
let lastJobs = [];
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
|
||||
const searchIn = h("input", {
|
||||
class: "hf-search",
|
||||
placeholder: t("import.searchPlaceholder"),
|
||||
autofocus: true,
|
||||
});
|
||||
const results = h("div", { class: "hf-grid" });
|
||||
const formHost = h("div", {});
|
||||
const jobsHost = h("div", {});
|
||||
|
||||
// ── live HF search (debounced) ──
|
||||
let debounce = null;
|
||||
function renderCards(models) {
|
||||
if (!models.length) {
|
||||
mount(results, h("div", { class: "muted pad" }, t("import.noResults")));
|
||||
return;
|
||||
}
|
||||
mount(results, ...models.map(modelCard));
|
||||
}
|
||||
async function doSearch() {
|
||||
const q = searchIn.value.trim();
|
||||
mount(results, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
try {
|
||||
const r = await api.hfSearch(q, 24);
|
||||
lastModels = r.models || [];
|
||||
renderCards(lastModels);
|
||||
} catch (e) {
|
||||
mount(results, h("div", { class: "err pad" }, "HF: " + e.message));
|
||||
}
|
||||
}
|
||||
searchIn.addEventListener("input", () => {
|
||||
clearTimeout(debounce);
|
||||
debounce = setTimeout(doSearch, 300);
|
||||
});
|
||||
|
||||
function modelCard(m) {
|
||||
const card = h(
|
||||
"div",
|
||||
{ class: "hf-card" + (selected && selected.id === m.id ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, m.id),
|
||||
h(
|
||||
"div",
|
||||
{ class: "hf-meta" },
|
||||
m.pipeline_tag ? h("span", { class: "badge" }, m.pipeline_tag) : null,
|
||||
h("span", { class: "muted" }, "▼ " + fmtNum(m.downloads)),
|
||||
h("span", { class: "muted" }, "♥ " + fmtNum(m.likes)),
|
||||
(m.tags || []).some((x) => x === "gated") ? h("span", { class: "badge bad" }, "gated") : null
|
||||
)
|
||||
);
|
||||
card.addEventListener("click", () => selectModel(m));
|
||||
return card;
|
||||
}
|
||||
|
||||
function selectModel(m) {
|
||||
selected = m;
|
||||
renderCards(lastModels); // re-highlight from cache — no refetch, no flash
|
||||
renderForm();
|
||||
formHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
}
|
||||
|
||||
// ── conversion form ──
|
||||
function renderForm() {
|
||||
if (!selected) {
|
||||
mount(formHost);
|
||||
return;
|
||||
}
|
||||
const nameIn = h("input", { value: (selected.id.split("/").pop() || "model").toLowerCase() });
|
||||
const quantSel = h("select", {}, ...QUANTS.map(([v], i) => opt(v, v, i === 0)));
|
||||
const quantHint = h("div", { class: "hint" }, t("import.q.q8_2f"));
|
||||
quantSel.addEventListener("change", () => {
|
||||
const k = QUANTS.find(([v]) => v === quantSel.value);
|
||||
quantHint.textContent = k ? t(k[1]) : "";
|
||||
});
|
||||
|
||||
// advanced (collapsed)
|
||||
const linearSel = h("select", {}, opt("", t("import.adv.auto")), opt("gated_delta_net", "gated_delta_net"), opt("vmf_phase", "vmf_phase"));
|
||||
const nphaseIn = h("input", { type: "number", min: "1", placeholder: "16" });
|
||||
const vbitSel = h("select", {}, opt("", t("import.adv.auto")), opt("log2", "log2"), opt("cubic", "cubic"));
|
||||
const meanBitsIn = h("input", { type: "number", step: "0.1", placeholder: "5.0" });
|
||||
const shardIn = h("input", { type: "number", step: "0.5", placeholder: t("import.adv.noShard") });
|
||||
const skipMtp = h("input", { type: "checkbox" });
|
||||
// O(1) attention (cortiq ≥ 0.2.0): constant-memory streaming attention hint,
|
||||
// weights byte-identical — the runtime applies it at load.
|
||||
const o1Sel = h("select", {},
|
||||
opt("", t("import.adv.o1Off")),
|
||||
opt("all", t("import.adv.o1All")),
|
||||
opt("deep", t("import.adv.o1Deep")),
|
||||
opt("custom", t("import.adv.o1Custom")));
|
||||
const o1DeepIn = h("input", { type: "number", min: "1", placeholder: "12" });
|
||||
const o1CustomIn = h("input", { placeholder: "0,4,8" });
|
||||
const o1M = h("input", { type: "number", min: "1", placeholder: "32" });
|
||||
const o1Window = h("input", { type: "number", min: "0", placeholder: "128" });
|
||||
const o1Sink = h("input", { type: "number", min: "0", placeholder: "4" });
|
||||
const o1SpecRow = h("div", { class: "row", style: "display:none" },
|
||||
field(t("import.adv.o1DeepN"), o1DeepIn),
|
||||
field(t("import.adv.o1Layers"), o1CustomIn));
|
||||
const o1Knobs = h("div", { class: "row", style: "display:none" },
|
||||
field("--o1-m", o1M, t("import.adv.o1mHint")),
|
||||
field("--o1-window", o1Window, t("import.adv.o1windowHint")),
|
||||
field("--o1-sink", o1Sink, t("import.adv.o1sinkHint")));
|
||||
o1Sel.addEventListener("change", () => {
|
||||
const on = !!o1Sel.value;
|
||||
o1Knobs.style.display = on ? "" : "none";
|
||||
o1SpecRow.style.display = o1Sel.value === "deep" || o1Sel.value === "custom" ? "" : "none";
|
||||
o1DeepIn.parentElement.style.display = o1Sel.value === "deep" ? "" : "none";
|
||||
o1CustomIn.parentElement.style.display = o1Sel.value === "custom" ? "" : "none";
|
||||
});
|
||||
const advBody = h(
|
||||
"div",
|
||||
{ class: "adv-body", style: "display:none" },
|
||||
h("div", { class: "row" },
|
||||
field(t("import.adv.linearCore"), linearSel, t("import.adv.linearHint")),
|
||||
field(t("import.adv.nphase"), nphaseIn)),
|
||||
h("div", { class: "row" },
|
||||
field(t("import.adv.vbitShape"), vbitSel),
|
||||
field(t("import.adv.meanBits"), meanBitsIn)),
|
||||
h("div", { class: "row" },
|
||||
field(t("import.adv.shard"), shardIn),
|
||||
field("MTP", h("label", { class: "check" }, skipMtp, t("import.adv.skipMtp")))),
|
||||
field(t("import.adv.o1"), o1Sel, t("import.adv.o1Hint")),
|
||||
o1SpecRow,
|
||||
o1Knobs,
|
||||
);
|
||||
const advToggle = h("button", { class: "btn ghost sm" }, "⚙ " + t("import.adv.title"));
|
||||
advToggle.addEventListener("click", () => {
|
||||
advBody.style.display = advBody.style.display === "none" ? "block" : "none";
|
||||
});
|
||||
|
||||
const goBtn = h("button", { class: "btn primary" }, "⬇ " + t("import.convert"));
|
||||
goBtn.addEventListener("click", async () => {
|
||||
const params = {
|
||||
repo: selected.id,
|
||||
quant: quantSel.value,
|
||||
name: nameIn.value.trim(),
|
||||
skip_mtp: skipMtp.checked,
|
||||
};
|
||||
if (linearSel.value) params.linear_core = linearSel.value;
|
||||
if (nphaseIn.value) params.nphase = parseInt(nphaseIn.value);
|
||||
if (vbitSel.value) params.vbit_shape = vbitSel.value;
|
||||
if (meanBitsIn.value) params.mean_bits = parseFloat(meanBitsIn.value);
|
||||
if (shardIn.value) params.shard_max_gb = parseFloat(shardIn.value);
|
||||
if (o1Sel.value) {
|
||||
// spec: all | deepN | explicit layer list
|
||||
params.o1 =
|
||||
o1Sel.value === "deep" ? "deep" + (parseInt(o1DeepIn.value) || 12)
|
||||
: o1Sel.value === "custom" ? o1CustomIn.value.trim()
|
||||
: "all";
|
||||
if (!params.o1) delete params.o1;
|
||||
else {
|
||||
if (o1M.value) params.o1_m = parseInt(o1M.value);
|
||||
if (o1Window.value) params.o1_window = parseInt(o1Window.value);
|
||||
if (o1Sink.value) params.o1_sink = parseInt(o1Sink.value);
|
||||
}
|
||||
}
|
||||
goBtn.disabled = true;
|
||||
try {
|
||||
await api.startImport(params);
|
||||
toast(t("import.started"), "ok");
|
||||
await refreshJobs(); // picks up the new job + (re)starts polling
|
||||
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
} catch (e) {
|
||||
toast(e.message, "err");
|
||||
}
|
||||
goBtn.disabled = false;
|
||||
});
|
||||
|
||||
mount(
|
||||
formHost,
|
||||
h("div", { class: "card import-form" },
|
||||
h("div", { class: "card-title" }, t("import.configure") + ": " + selected.id),
|
||||
h("div", { class: "row" },
|
||||
field(t("import.outName"), nameIn, t("import.outNameHint")),
|
||||
field(t("import.quant"), quantSel)),
|
||||
quantHint,
|
||||
h("div", { class: "adv" }, advToggle, advBody),
|
||||
h("div", { class: "actions" }, goBtn),
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// ── jobs panel: all conversions, live, resumable, cancellable ──
|
||||
function ensurePolling() {
|
||||
if (!jobsTimer) {
|
||||
jobsTimer = setInterval(() => {
|
||||
// stop once this view instance was replaced (navigated away)
|
||||
if (!root.isConnected) {
|
||||
stopPolling();
|
||||
return;
|
||||
}
|
||||
refreshJobs();
|
||||
}, 1500);
|
||||
}
|
||||
}
|
||||
function stopPolling() {
|
||||
if (jobsTimer) {
|
||||
clearInterval(jobsTimer);
|
||||
jobsTimer = null;
|
||||
}
|
||||
}
|
||||
// Load + render all jobs. Runs on mount (before `root` is attached — so no
|
||||
// isConnected guard here) and on each poll tick.
|
||||
async function cancelJob(j) {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "warn");
|
||||
await refreshJobs();
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
let jobs = [];
|
||||
try {
|
||||
jobs = (await api.listImports()).jobs || [];
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
jobs = jobs.filter((j) => j.state !== "cancelled"); // hide user-cancelled jobs
|
||||
lastJobs = jobs;
|
||||
renderJobs(jobs);
|
||||
// A settled download changes what the featured cards offer (✓ + delete), so
|
||||
// re-read the list on that transition — not on every poll tick.
|
||||
const settled = jobs.some((j) => jobStates.get(j.id) === "running" && j.state !== "running");
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) await renderFeatured();
|
||||
else paintFeaturedHosts(featuredHosts, jobs, cancelJob);
|
||||
if (jobs.some((j) => j.state === "running")) ensurePolling();
|
||||
else stopPolling();
|
||||
}
|
||||
|
||||
function jobCard(j) {
|
||||
const running = j.state === "running";
|
||||
const pct = j.progress != null ? Math.round(j.progress * 100) : null;
|
||||
// det = solid bar at a known %, run = indeterminate slider, ok/err = terminal.
|
||||
let bar;
|
||||
if (j.state === "done") {
|
||||
bar = h("div", { class: "pbar-fill ok", style: "width:100%" });
|
||||
} else if (running && pct != null) {
|
||||
bar = h("div", { class: "pbar-fill det", style: `width:${pct}%` });
|
||||
} else if (running) {
|
||||
bar = h("div", { class: "pbar-fill run" }); // CSS drives width + animation
|
||||
} else {
|
||||
bar = h("div", { class: "pbar-fill err", style: "width:100%" }); // error / cancelled
|
||||
}
|
||||
const barWrap = h("div", { class: "pbar" }, bar);
|
||||
const badge = h(
|
||||
"span",
|
||||
{ class: "badge " + (j.state === "done" ? "" : running ? "" : "bad") },
|
||||
stateLabel(j.state) +
|
||||
(running && pct != null ? " · " + pct + "%" : "") +
|
||||
(j.size_bytes ? " · " + fmtBytes(j.size_bytes) : "")
|
||||
);
|
||||
const phase = h("div", { class: "muted small" }, jobStatusText(j));
|
||||
const logEl = h("pre", { class: "conv-log" }, (j.log || []).slice(-10).join("\n"));
|
||||
|
||||
const actions = [];
|
||||
if (running) {
|
||||
const c = h("button", { class: "btn ghost sm" }, "✕ " + t("import.cancel"));
|
||||
c.addEventListener("click", async () => {
|
||||
c.disabled = true;
|
||||
try {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "warn");
|
||||
refreshJobs();
|
||||
} catch (e) {
|
||||
toast(e.message, "err");
|
||||
c.disabled = false;
|
||||
}
|
||||
});
|
||||
actions.push(c);
|
||||
}
|
||||
if (j.state === "done") {
|
||||
const r = h("button", { class: "btn primary sm" }, "✓ " + t("import.register"));
|
||||
r.addEventListener("click", async () => {
|
||||
r.disabled = true;
|
||||
try {
|
||||
const rr = await api.registerImport(j.id);
|
||||
toast(t("import.registered", { id: rr.model_id }), "ok");
|
||||
} catch (e) {
|
||||
toast(e.message, "err");
|
||||
r.disabled = false;
|
||||
}
|
||||
});
|
||||
actions.push(r);
|
||||
}
|
||||
if (!running) {
|
||||
// delete the job + its converted .cmf file(s) from disk
|
||||
const d = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("import.delete"));
|
||||
d.addEventListener("click", async () => {
|
||||
if (!window.confirm(t("import.deleteConfirm"))) return;
|
||||
d.disabled = true;
|
||||
try {
|
||||
await api.deleteImport(j.id);
|
||||
toast(t("import.deleted"), "ok");
|
||||
refreshJobs();
|
||||
} catch (e) {
|
||||
toast(e.message, "err");
|
||||
d.disabled = false;
|
||||
}
|
||||
});
|
||||
actions.push(d);
|
||||
}
|
||||
|
||||
return h(
|
||||
"div",
|
||||
{ class: "job" },
|
||||
h("div", { class: "job-head" }, h("b", {}, j.repo || j.id), badge),
|
||||
barWrap,
|
||||
phase,
|
||||
logEl,
|
||||
actions.length ? h("div", { class: "actions" }, ...actions) : null
|
||||
);
|
||||
}
|
||||
|
||||
function renderJobs(jobs) {
|
||||
if (!jobs.length) {
|
||||
mount(jobsHost);
|
||||
return;
|
||||
}
|
||||
mount(
|
||||
jobsHost,
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-title" }, t("import.jobs")),
|
||||
...jobs.map(jobCard)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// ── featured: the infosave account, read live so new releases just show up ──
|
||||
const featuredGrid = h("div", { class: "hf-grid" }, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
const reloadBtn = h("button", { class: "btn ghost sm" }, "↻ " + t("common.refresh"));
|
||||
const featuredHost = h(
|
||||
"div",
|
||||
{ class: "card featured-section" },
|
||||
h("div", { class: "card-head", style: "display:flex; align-items:center; gap:10px" },
|
||||
h("div", { class: "card-title", style: "margin:0" }, "⚡ " + t("import.featuredTitle")),
|
||||
h("div", { style: "flex:1" }),
|
||||
reloadBtn),
|
||||
h("div", { class: "muted small", style: "margin:2px 0 10px" }, t("import.featuredSub")),
|
||||
featuredGrid
|
||||
);
|
||||
|
||||
async function startFeatured(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function deleteFeatured(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!window.confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await renderFeatured();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function renderFeatured(reload = true) {
|
||||
if (reload) featured = await loadFeatured();
|
||||
let sys = null;
|
||||
try { sys = await api.system(); } catch { /* metrics optional */ }
|
||||
featuredHosts = new Map();
|
||||
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured, sys };
|
||||
mount(featuredGrid, ...(featured.length
|
||||
? featured.map((m) => featuredCard(m, ctx))
|
||||
: [h("div", { class: "muted pad" }, t("import.noResults"))]));
|
||||
paintFeaturedHosts(featuredHosts, lastJobs, cancelJob);
|
||||
}
|
||||
reloadBtn.addEventListener("click", async () => {
|
||||
reloadBtn.disabled = true;
|
||||
mount(featuredGrid, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
await renderFeatured();
|
||||
reloadBtn.disabled = false;
|
||||
});
|
||||
|
||||
const root = h(
|
||||
"div",
|
||||
{},
|
||||
h("div", { class: "view-head" },
|
||||
h("h2", {}, t("nav.import")),
|
||||
h("div", { class: "muted" }, t("import.subtitle"))),
|
||||
jobsHost, // progress panel on top, above the search
|
||||
featuredHost, // featured ready-to-run models
|
||||
h("div", { class: "card" }, searchIn, results),
|
||||
formHost
|
||||
);
|
||||
doSearch(); // initial trending list (mounts into `results`)
|
||||
renderFeatured(); // ready-made .cmf models, fetched live
|
||||
refreshJobs(); // resume: show any running/recent conversions immediately
|
||||
return root;
|
||||
}
|
||||
|
||||
Vendored
+329
-14
@@ -1,30 +1,23 @@
|
||||
// API keys — list, create (with copy-once), revoke.
|
||||
import { h, mount, modal, toast, confirmDialog } from "../ui.js";
|
||||
import { h, mount, modal, toast, confirmDialog, moneyEl, num } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export async function renderKeys() {
|
||||
const modelsData = await api.listModels();
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
const root = h("div");
|
||||
let usageRange = "7d";
|
||||
|
||||
function openCreate(reload) {
|
||||
const accountIn = h("input", { placeholder: "team-a" });
|
||||
const rateIn = h("input", { type: "number", value: 600 });
|
||||
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
|
||||
const customIn = h("input", { placeholder: "sk-gw-…" });
|
||||
const allowInputs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox" }) }));
|
||||
const body = h(
|
||||
"div",
|
||||
{},
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
|
||||
h(
|
||||
"label",
|
||||
{ class: "field" },
|
||||
h("span", {}, t("keys.form.allow")),
|
||||
h("div", { class: "flex wrap" }, ...allowInputs.map(({ id, cb }) => h("label", { class: "check" }, cb, id))),
|
||||
h("div", { class: "hint" }, t("keys.form.allowHint"))
|
||||
),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
|
||||
);
|
||||
modal(t("keys.form.title"), body, async () => {
|
||||
@@ -33,7 +26,8 @@ export async function renderKeys() {
|
||||
key: customIn.value.trim(),
|
||||
account: accountIn.value.trim() || "default",
|
||||
rate_per_min: parseInt(rateIn.value) || 0,
|
||||
allow_models: allowInputs.filter(({ cb }) => cb.checked).map(({ id }) => id),
|
||||
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
|
||||
allow_models: [],
|
||||
});
|
||||
reload();
|
||||
showCreated(r.key);
|
||||
@@ -48,11 +42,310 @@ export async function renderKeys() {
|
||||
const input = h("input", { value: key, readonly: true, class: "mono" });
|
||||
const copyBtn = h("button", { class: "btn", onClick: () => { input.select(); navigator.clipboard?.writeText(key); toast(t("common.copied"), "good"); } }, t("common.copy"));
|
||||
modal(t("common.ok"), h("div", {}, h("p", {}, t("keys.created")), h("div", { class: "flex" }, input, copyBtn)), null);
|
||||
lastCreatedKey = key;
|
||||
reload();
|
||||
}
|
||||
|
||||
let lastCreatedKey = null;
|
||||
|
||||
// "Connect your app" — copy-paste snippets with this gateway's real URL.
|
||||
// Keys are stored hashed-masked in the list, so the key itself is a
|
||||
// placeholder unless one was just created on this page.
|
||||
function snippetsCard(openMode) {
|
||||
const origin = location.origin;
|
||||
const base = origin + "/v1";
|
||||
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
// Every scripty snippet exists in two OS flavors: bash heredocs for
|
||||
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
|
||||
// (python/js) and GUI instructions are OS-independent.
|
||||
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
|
||||
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
|
||||
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
|
||||
const buildSnips = (osKind) => osKind === "ps" ? {
|
||||
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
|
||||
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
|
||||
$env:ANTHROPIC_MODEL = "cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
|
||||
Add-Content "$HOME\\.codex\\config.toml" @'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
[model_providers.cortiq]
|
||||
name = "Cortiq Gateway"
|
||||
base_url = "${base}"
|
||||
env_key = "CORTIQ_API_KEY"
|
||||
wire_api = "chat"
|
||||
'@
|
||||
$env:CORTIQ_API_KEY = "${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
|
||||
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"cortiq": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Cortiq Gateway",
|
||||
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
opencode`,
|
||||
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
|
||||
Add-Content "$HOME\\.continue\\config.yaml" @'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
model: cortiq-auto
|
||||
apiBase: ${base}
|
||||
apiKey: ${key}
|
||||
'@`,
|
||||
Cline: `${t("keys.h.cline1")}
|
||||
API Provider: OpenAI Compatible
|
||||
Base URL: ${base}
|
||||
API Key: ${key}
|
||||
Model ID: cortiq-auto
|
||||
|
||||
${t("keys.h.cline2")}`,
|
||||
curl: `curl.exe ${base}/chat/completions \`
|
||||
-H "Authorization: Bearer ${key}" \`
|
||||
-H "Content-Type: application/json" \`
|
||||
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||
} : {
|
||||
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
|
||||
export ANTHROPIC_AUTH_TOKEN="${key}"
|
||||
export ANTHROPIC_MODEL="cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
[model_providers.cortiq]
|
||||
name = "Cortiq Gateway"
|
||||
base_url = "${base}"
|
||||
env_key = "CORTIQ_API_KEY"
|
||||
wire_api = "chat"
|
||||
EOF
|
||||
export CORTIQ_API_KEY="${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"cortiq": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Cortiq Gateway",
|
||||
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
opencode`,
|
||||
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
model: cortiq-auto
|
||||
apiBase: ${base}
|
||||
apiKey: ${key}
|
||||
EOF`,
|
||||
Cline: `${t("keys.h.cline1")}
|
||||
API Provider: OpenAI Compatible
|
||||
Base URL: ${base}
|
||||
API Key: ${key}
|
||||
Model ID: cortiq-auto
|
||||
|
||||
${t("keys.h.cline2")}`,
|
||||
curl: `curl ${base}/chat/completions \\
|
||||
-H "Authorization: Bearer ${key}" \\
|
||||
-H "Content-Type: application/json" \\
|
||||
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||
python: `from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="${base}", api_key="${key}")
|
||||
resp = client.chat.completions.create(
|
||||
model="cortiq-auto", # авто-роутинг; или точное имя модели
|
||||
messages=[{"role": "user", "content": "Привет!"}],
|
||||
)
|
||||
print(resp.choices[0].message.content)`,
|
||||
javascript: `import OpenAI from "openai";
|
||||
|
||||
const client = new OpenAI({ baseURL: "${base}", apiKey: "${key}" });
|
||||
const resp = await client.chat.completions.create({
|
||||
model: "cortiq-auto",
|
||||
messages: [{ role: "user", content: "Привет!" }],
|
||||
});
|
||||
console.log(resp.choices[0].message.content);`,
|
||||
};
|
||||
let shSnips, psSnips, snips;
|
||||
function rebuildSnips() {
|
||||
shSnips = buildSnips("sh");
|
||||
psSnips = buildSnips("ps");
|
||||
// python/js/Cline одинаковы — дополняем ps-набор из sh
|
||||
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
|
||||
snips = os === "ps" ? psSnips : shSnips;
|
||||
}
|
||||
rebuildSnips();
|
||||
let cur = "Claude Code";
|
||||
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
|
||||
// in a pasted snippet is not a comment in default interactive zsh, and a
|
||||
// "!" inside it triggers history expansion ("event not found")
|
||||
const HINTS = {
|
||||
"Claude Code": () => t("keys.h.claude"),
|
||||
Codex: () => t("keys.h.codex"),
|
||||
OpenCode: () => t("keys.h.opencode"),
|
||||
Continue: () => t("keys.h.continue"),
|
||||
};
|
||||
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
|
||||
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
|
||||
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
|
||||
paintHint();
|
||||
const seg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
...Object.keys(snips).map((k, i) =>
|
||||
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
|
||||
cur = k;
|
||||
pre.textContent = snips[k];
|
||||
paintHint();
|
||||
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
|
||||
paintStatus();
|
||||
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
|
||||
e.currentTarget.classList.add("active");
|
||||
} }, k)
|
||||
)
|
||||
);
|
||||
// Claude Code needs the anthropic_messages protocol — offer a one-click
|
||||
// enable when it is off, right where the user will hit the problem.
|
||||
const protoWarn = h("div", { style: "display:none; margin-top:8px" });
|
||||
api.getProtocols().then((prot) => {
|
||||
if (prot.anthropic_messages) return;
|
||||
const fixBtn = h("button", { class: "btn sm primary" }, t("keys.h.enableProto"));
|
||||
fixBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.putProtocols({ ...prot, anthropic_messages: true });
|
||||
protoWarn.style.display = "none";
|
||||
toast(t("common.saved"), "ok");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
});
|
||||
mount(protoWarn, h("div", { class: "callout warn" }, h("span", {}, "⚠ " + t("keys.h.protoOff")), " ", fixBtn));
|
||||
if (cur === "Claude Code") protoWarn.style.display = "";
|
||||
}).catch(() => {});
|
||||
|
||||
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
|
||||
// status line under the snippet ("installed" / "already configured") —
|
||||
// and a one-line INSTALL command (per OS) when it's missing.
|
||||
const INSTALL = {
|
||||
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
|
||||
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
|
||||
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
|
||||
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
|
||||
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
|
||||
};
|
||||
let detected = {};
|
||||
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
|
||||
function paintStatus() {
|
||||
const d = detected[cur];
|
||||
if (!d) {
|
||||
statusHost.textContent = "";
|
||||
return;
|
||||
}
|
||||
statusHost.innerHTML = "";
|
||||
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
|
||||
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
|
||||
else {
|
||||
const cmd = (INSTALL[cur] || {})[os];
|
||||
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
|
||||
await navigator.clipboard.writeText(cmd);
|
||||
toast(t("common.copied"), "good");
|
||||
} }, "⧉");
|
||||
mount(statusHost,
|
||||
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
|
||||
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
|
||||
h("span", { class: "muted" }, t("keys.h.install")),
|
||||
h("code", { class: "mono small cmd-chip" }, cmd),
|
||||
copyInstall) : null);
|
||||
}
|
||||
}
|
||||
api.harnesses().then((d) => {
|
||||
detected = d || {};
|
||||
seg.querySelectorAll("button").forEach((b) => {
|
||||
const st = detected[b.dataset.snip];
|
||||
if (st && st.installed) b.append(" ✓");
|
||||
});
|
||||
paintStatus();
|
||||
}).catch(() => {});
|
||||
|
||||
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
|
||||
const osSeg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
|
||||
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
|
||||
);
|
||||
osSeg.querySelectorAll("button").forEach((b, i) =>
|
||||
b.addEventListener("click", () => {
|
||||
os = i === 0 ? "sh" : "ps";
|
||||
rebuildSnips();
|
||||
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
|
||||
pre.textContent = snips[cur];
|
||||
paintStatus();
|
||||
}));
|
||||
|
||||
// paste-your-key: substitutes the key into every snippet immediately
|
||||
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
|
||||
const applyKey = () => {
|
||||
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
rebuildSnips();
|
||||
pre.textContent = snips[cur];
|
||||
};
|
||||
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
|
||||
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
|
||||
|
||||
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||
copy.addEventListener("click", async () => {
|
||||
await navigator.clipboard.writeText(pre.textContent);
|
||||
toast(t("common.copied"), "ok");
|
||||
});
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" },
|
||||
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
|
||||
hintHost,
|
||||
pre,
|
||||
statusHost,
|
||||
protoWarn,
|
||||
h("div", { class: "muted small", style: "margin-top:6px" }, t("keys.h.note"))
|
||||
);
|
||||
}
|
||||
|
||||
async function reload() {
|
||||
const data = await api.listKeys();
|
||||
const keys = data.keys || [];
|
||||
// per-account usage: window breakdown + cumulative-forever totals — the
|
||||
// "who spends what" view for departments/developers sharing one gateway
|
||||
let usage = {};
|
||||
let alltime = {};
|
||||
try {
|
||||
const st = await api.stats(usageRange, "account");
|
||||
for (const b of st.breakdown || []) usage[b.key] = b;
|
||||
alltime = st.alltime_by_account || {};
|
||||
} catch { /* stats disabled — columns show dashes */ }
|
||||
mount(
|
||||
root,
|
||||
h(
|
||||
@@ -62,6 +355,9 @@ export async function renderKeys() {
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
h("div", { class: "grow" }, h("h2", {}, t("keys.title")), h("p", {}, t("keys.subtitle"))),
|
||||
h("div", { class: "seg", style: "margin-right:10px" },
|
||||
...["24h", "7d", "30d"].map((r) =>
|
||||
h("button", { class: r === usageRange ? "active" : "", onClick: () => { usageRange = r; reload(); } }, r))),
|
||||
h("button", { class: "btn primary", onClick: () => openCreate(reload) }, "+ " + t("keys.add"))
|
||||
)
|
||||
),
|
||||
@@ -80,7 +376,16 @@ export async function renderKeys() {
|
||||
h(
|
||||
"thead",
|
||||
{},
|
||||
h("tr", {}, h("th", {}, t("keys.col.key")), h("th", {}, t("keys.col.account")), h("th", {}, t("keys.col.rate")), h("th", {}, t("keys.col.models")), h("th", { class: "right-align" }, t("common.actions")))
|
||||
h("tr", {},
|
||||
h("th", {}, t("keys.col.key")),
|
||||
h("th", {}, t("keys.col.account")),
|
||||
h("th", {}, t("keys.col.rate")),
|
||||
h("th", {}, t("keys.col.models")),
|
||||
h("th", { class: "right-align" }, t("keys.col.requests") + " (" + usageRange + ")"),
|
||||
h("th", { class: "right-align" }, t("keys.col.tokens") + " (" + usageRange + ")"),
|
||||
h("th", { class: "right-align" }, "$ (" + usageRange + ")"),
|
||||
h("th", { class: "right-align" }, t("keys.col.costAll")),
|
||||
h("th", { class: "right-align" }, t("common.actions")))
|
||||
),
|
||||
h(
|
||||
"tbody",
|
||||
@@ -93,6 +398,15 @@ export async function renderKeys() {
|
||||
h("td", {}, k.account),
|
||||
h("td", { class: "mono" }, k.rate_per_min || "—"),
|
||||
h("td", {}, (k.allow_models && k.allow_models.length) ? k.allow_models.join(", ") : h("span", { class: "muted" }, t("keys.allowAll"))),
|
||||
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
|
||||
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
|
||||
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
|
||||
h("td", { class: "right-align" },
|
||||
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
|
||||
k.monthly_budget_usd
|
||||
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
|
||||
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
|
||||
: null),
|
||||
h(
|
||||
"td",
|
||||
{ class: "right-align" },
|
||||
@@ -119,7 +433,8 @@ export async function renderKeys() {
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
),
|
||||
snippetsCard(!!data.open_mode)
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
Vendored
+7
-1
@@ -2,6 +2,7 @@
|
||||
import { h } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { setToken } from "../api.js";
|
||||
import { SITE_URL } from "../app.js";
|
||||
|
||||
export function renderLogin(onOk) {
|
||||
const input = h("input", {
|
||||
@@ -34,7 +35,12 @@ export function renderLogin(onOk) {
|
||||
h("p", { class: "sub" }, t("login.subtitle")),
|
||||
h("label", { class: "field" }, h("span", {}, t("login.token")), input),
|
||||
h("button", { class: "btn primary", style: "width:100%", onClick: submit }, t("login.submit")),
|
||||
h("p", { class: "hint", style: "margin-top:16px" }, t("login.hint"))
|
||||
h("p", { class: "hint", style: "margin-top:16px" }, t("login.hint")),
|
||||
h(
|
||||
"p",
|
||||
{ class: "hint", style: "margin-top:8px" },
|
||||
h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("nav.billing") + " · api.allaigate.com ↗")
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
Vendored
+95
@@ -0,0 +1,95 @@
|
||||
// Gateway log viewer — the in-memory ring served by /admin/api/logs, for
|
||||
// everyone who never sees the terminal (Docker, autostart, double-clicked exe).
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export function renderLogs() {
|
||||
const root = h("div");
|
||||
let timer = null;
|
||||
let paused = false;
|
||||
let filter = "";
|
||||
let level = "";
|
||||
|
||||
const pre = h("pre", { class: "conv-log logs-view" }, t("common.loading"));
|
||||
const filterIn = h("input", { placeholder: t("logs.filterPlaceholder"), style: "max-width:260px" });
|
||||
const levelSel = h(
|
||||
"select",
|
||||
{ style: "max-width:130px" },
|
||||
h("option", { value: "" }, t("common.all")),
|
||||
...["ERROR", "WARN", "INFO", "DEBUG"].map((l) => h("option", { value: l }, l))
|
||||
);
|
||||
const pauseBtn = h("button", { class: "btn ghost sm" }, "⏸ " + t("logs.pause"));
|
||||
const copyBtn = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||
const clearBtn = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("dash.clear"));
|
||||
clearBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.clearLogs();
|
||||
lines = [];
|
||||
paint();
|
||||
toast(t("dash.cleared"), "ok");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
});
|
||||
|
||||
let lines = [];
|
||||
function paint() {
|
||||
let shown = lines;
|
||||
if (level) shown = shown.filter((l) => l.includes(` ${level} `) || l.includes(`${level} `));
|
||||
if (filter) shown = shown.filter((l) => l.toLowerCase().includes(filter));
|
||||
pre.textContent = shown.length ? shown.join("\n") : t("logs.empty");
|
||||
if (!paused) pre.scrollTop = pre.scrollHeight;
|
||||
}
|
||||
async function refresh() {
|
||||
if (paused) return;
|
||||
try {
|
||||
lines = (await api.logs(500)).lines || [];
|
||||
paint();
|
||||
} catch {
|
||||
/* transient */
|
||||
}
|
||||
}
|
||||
filterIn.addEventListener("input", () => {
|
||||
filter = filterIn.value.trim().toLowerCase();
|
||||
paint();
|
||||
});
|
||||
levelSel.addEventListener("change", () => {
|
||||
level = levelSel.value;
|
||||
paint();
|
||||
});
|
||||
pauseBtn.addEventListener("click", () => {
|
||||
paused = !paused;
|
||||
pauseBtn.textContent = (paused ? "▶ " : "⏸ ") + t(paused ? "logs.resume" : "logs.pause");
|
||||
if (!paused) refresh();
|
||||
});
|
||||
copyBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await navigator.clipboard.writeText(pre.textContent);
|
||||
toast(t("common.copied"), "ok");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
});
|
||||
|
||||
timer = setInterval(() => {
|
||||
if (!root.isConnected) {
|
||||
clearInterval(timer);
|
||||
return;
|
||||
}
|
||||
refresh();
|
||||
}, 2000);
|
||||
refresh();
|
||||
|
||||
mount(
|
||||
root,
|
||||
h("div", { class: "view-head" }, h("h2", {}, t("nav.logs")), h("div", { class: "muted" }, t("logs.subtitle"))),
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "flex", style: "gap:8px; flex-wrap:wrap; margin-bottom:10px" }, filterIn, levelSel, pauseBtn, copyBtn, clearBtn),
|
||||
pre
|
||||
)
|
||||
);
|
||||
return root;
|
||||
}
|
||||
Vendored
+265
-38
@@ -1,5 +1,5 @@
|
||||
// Models — pool table with add/edit modal, probe, and per-model secret entry.
|
||||
import { h, mount, modal, toast, confirmDialog, money } from "../ui.js";
|
||||
import { h, mount, modal, toast, confirmDialog, money, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
@@ -7,61 +7,232 @@ import { appState } from "../app.js";
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
function keyBadge(src) {
|
||||
const map = { store: "store", env: "env", missing: "bad", none: "" };
|
||||
return h("span", { class: "badge " + (map[src] ?? "") }, src);
|
||||
}
|
||||
|
||||
// Verified per-provider defaults (base URL, key env, whether a key is needed).
|
||||
const PROVIDER_DEFAULTS = {
|
||||
openai: { base: "https://api.openai.com/v1", keyEnv: "OPENAI_API_KEY", needsKey: true, tier: "mid", caps: ["tools", "vision"] },
|
||||
anthropic: { base: "https://api.anthropic.com", keyEnv: "ANTHROPIC_API_KEY", needsKey: true, tier: "expensive", caps: ["tools", "vision"] },
|
||||
openrouter: { base: "https://openrouter.ai/api/v1", keyEnv: "OPENROUTER_API_KEY", needsKey: true, tier: "mid", caps: ["tools", "vision"] },
|
||||
lmstudio: { base: "http://localhost:1234/v1", keyEnv: "", needsKey: false, tier: "local", caps: ["tools", "vision"] },
|
||||
ollama: { base: "http://localhost:11434/v1", keyEnv: "", needsKey: false, tier: "local", caps: ["tools"] },
|
||||
http: { base: "", keyEnv: "", needsKey: false, tier: "cheap", caps: ["tools"] },
|
||||
};
|
||||
|
||||
// Derive a clean model id from a model name (last path segment, slugified).
|
||||
// e.g. "qwen/qwen3.7-max" → "qwen3-7-max".
|
||||
function slugId(name) {
|
||||
const base = (name || "").split("/").pop() || "";
|
||||
return base.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "");
|
||||
}
|
||||
|
||||
function modelForm(meta, existing) {
|
||||
const m = existing || { provider: "openai", kind: "chat", cost_tier: "cheap", caps: [], price_in: 0, price_out: 0 };
|
||||
const idIn = h("input", { value: m.id || "", placeholder: "local-qwen", disabled: existing ? true : null });
|
||||
const m = existing || { provider: "openai", kind: "chat", cost_tier: "mid", caps: ["tools"], price_in: 0, price_out: 0 };
|
||||
const idIn = h("input", { value: m.id || "", placeholder: "my-model", disabled: existing ? true : null });
|
||||
const providerSel = h("select", {}, ...(meta.providers || ["openai"]).map((p) => opt(p, p, p === m.provider)));
|
||||
const baseUrlIn = h("input", { value: m.base_url || "", placeholder: "http://localhost:8000/v1" });
|
||||
const modelIn = h("input", { value: m.model || "", placeholder: "qwen2.5-7b-instruct" });
|
||||
const baseUrlIn = h("input", { value: m.base_url || "", placeholder: "https://api.openai.com/v1" });
|
||||
|
||||
// API key — typed directly, stored securely as a secret under a provider env name.
|
||||
const keyIn = h("input", { type: "password", placeholder: t("models.form.apiKeyPlaceholder"), autocomplete: "new-password" });
|
||||
const testBtn = h("button", { class: "btn small", type: "button" }, t("models.form.testKey"));
|
||||
const keyStatus = h("span", { class: "small", style: "margin-left:8px" });
|
||||
|
||||
// Model — below the key (some providers list models from the key); searchable datalist.
|
||||
const dlId = "model-opts-" + (m.id || "new");
|
||||
const dl = h("datalist", { id: dlId });
|
||||
if (m.model) dl.appendChild(h("option", { value: m.model }));
|
||||
const modelIn = h("input", { value: m.model || "", placeholder: t("models.form.modelPlaceholder"), list: dlId, autocomplete: "off" });
|
||||
|
||||
const kindSel = h("select", {}, ...(meta.kinds || ["chat"]).map((k) => opt(k, k, k === m.kind)));
|
||||
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
|
||||
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
|
||||
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
|
||||
const keyEnvIn = h("input", { value: m.api_key_env || "", placeholder: "OPENAI_API_KEY" });
|
||||
const secretIn = h("input", { type: "password", placeholder: t("models.form.secretPlaceholder") });
|
||||
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
|
||||
const CAP_EMOJIS = {
|
||||
tools: "🔧 tools",
|
||||
vision: "👁️ vision",
|
||||
audio: "🎙️ audio",
|
||||
reasoning: "🧠 reasoning",
|
||||
json: "📄 json",
|
||||
caching: "⚡ caching"
|
||||
};
|
||||
|
||||
const capInputs = (meta.caps || Object.keys(CAP_EMOJIS)).map((c) => {
|
||||
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
|
||||
return { c, cb };
|
||||
});
|
||||
|
||||
const envFor = () =>
|
||||
(existing && existing.api_key_env) ||
|
||||
(PROVIDER_DEFAULTS[providerSel.value] || {}).keyEnv ||
|
||||
providerSel.value.toUpperCase().replace(/[^A-Z0-9]/g, "_") + "_API_KEY";
|
||||
|
||||
// Fill in a provider's verified defaults when it is selected.
|
||||
function applyDefaults() {
|
||||
const d = PROVIDER_DEFAULTS[providerSel.value];
|
||||
if (!d) return;
|
||||
baseUrlIn.value = d.base;
|
||||
tierSel.value = d.tier;
|
||||
capInputs.forEach(({ c, cb }) => { cb.checked = d.caps.includes(c); });
|
||||
keyIn.placeholder = d.needsKey ? t("models.form.apiKeyPlaceholder") : t("models.form.apiKeyNotNeeded");
|
||||
mount(keyStatus, "");
|
||||
// Automatically pre-fetch models (especially useful for OpenRouter which doesn't need a key)
|
||||
testKey(true);
|
||||
}
|
||||
providerSel.addEventListener("change", applyDefaults);
|
||||
if (!existing) applyDefaults();
|
||||
|
||||
let cachedModels = [];
|
||||
|
||||
const autofillData = () => {
|
||||
console.log("autofillData triggered. modelIn.value:", modelIn.value);
|
||||
console.log("cachedModels length:", cachedModels.length);
|
||||
if (!existing && (!idIn.value || idIn._auto)) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
console.log("auto-filled idIn:", idIn.value);
|
||||
}
|
||||
const found = cachedModels.find((m) => (m.id || m) === modelIn.value);
|
||||
console.log("found model:", found);
|
||||
if (found) {
|
||||
if (found.price_in !== undefined) {
|
||||
priceInIn.value = parseFloat(Number(found.price_in).toFixed(4));
|
||||
priceOutIn.value = parseFloat(Number(found.price_out).toFixed(4));
|
||||
console.log("Updated prices:", priceInIn.value, priceOutIn.value);
|
||||
}
|
||||
if (found.caps && Array.isArray(found.caps)) {
|
||||
capInputs.forEach(({ c, cb }) => {
|
||||
cb.checked = found.caps.includes(c);
|
||||
});
|
||||
console.log("Updated caps:", found.caps);
|
||||
}
|
||||
}
|
||||
};
|
||||
modelIn.addEventListener("input", autofillData);
|
||||
modelIn.addEventListener("change", autofillData);
|
||||
|
||||
if (!existing) {
|
||||
idIn.addEventListener("input", () => { idIn._auto = false; });
|
||||
} else {
|
||||
console.log("Editing existing model, fetching testKey(true)");
|
||||
testKey(true);
|
||||
}
|
||||
|
||||
async function testKey(silent = false) {
|
||||
if (!baseUrlIn.value.trim()) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?"));
|
||||
return;
|
||||
}
|
||||
testBtn.disabled = true;
|
||||
if (!silent) mount(keyStatus, t("models.form.testing"));
|
||||
try {
|
||||
const r = await api.providerModels({
|
||||
provider: providerSel.value,
|
||||
base_url: baseUrlIn.value.trim(),
|
||||
api_key: keyIn.value.trim() || null,
|
||||
api_key_env: envFor(),
|
||||
});
|
||||
if (r.ok) {
|
||||
cachedModels = r.models || [];
|
||||
dl.replaceChildren(...cachedModels.map((mm) => h("option", { value: mm.id || mm })));
|
||||
|
||||
let msg = "✓ " + t("models.form.foundModels", { n: r.count });
|
||||
if (r.balance !== undefined) {
|
||||
msg += ` (Balance: $${r.balance.toFixed(4)})`;
|
||||
}
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, msg));
|
||||
if (!silent) modelIn.focus();
|
||||
} else {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
}
|
||||
} catch (e) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
}
|
||||
testBtn.disabled = false;
|
||||
}
|
||||
testBtn.addEventListener("click", () => testKey(false));
|
||||
keyIn.addEventListener("change", () => testKey(true));
|
||||
|
||||
const keyHint =
|
||||
existing && existing.key_source === "store" ? t("models.form.keyStored")
|
||||
: existing && existing.key_source === "env" ? t("models.form.keyFromEnv")
|
||||
: t("models.form.apiKeyHint");
|
||||
|
||||
// ⚙ Generation & Thinking Parameters
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: m.temperature ?? "" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: m.top_p ?? "" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: m.max_tokens ?? "" });
|
||||
const thinkIn = h("input", { type: "number", min: "0", placeholder: "1024", value: m.think_budget ?? "" });
|
||||
const o1Sel = h("select", {},
|
||||
h("option", { value: "" }, t("import.adv.auto")),
|
||||
h("option", { value: "all" }, "all — O(1) Attention"),
|
||||
h("option", { value: "deep" }, "deep — O(1) Deep"),
|
||||
h("option", { value: "off" }, "off — Standard"));
|
||||
if (m.o1) o1Sel.value = m.o1;
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: m.skip_mtp ? true : null });
|
||||
const sysPromptIn = h("textarea", { rows: 2, placeholder: t("models.form.systemPromptPlaceholder"), value: m.system_prompt || "" });
|
||||
|
||||
const advParams = h(
|
||||
"details",
|
||||
{ class: "adv-params", style: "margin-top:12px;padding:10px;border:1px solid var(--border);border-radius:6px;" },
|
||||
h("summary", { style: "font-weight:600;cursor:pointer;" }, "⚙ " + t("models.form.paramsTitle")),
|
||||
h("div", { class: "row", style: "margin-top:8px" },
|
||||
field(t("models.form.temperature"), tempIn, t("models.form.temperatureHint")),
|
||||
field(t("models.form.topP"), topPIn, t("models.form.topPHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("models.form.maxTokens"), maxTokIn, t("models.form.maxTokensHint")),
|
||||
field(t("models.form.thinkBudget"), thinkIn, t("models.form.thinkBudgetHint"))),
|
||||
h("div", { class: "row" },
|
||||
field("O(1) Attention", o1Sel, t("import.adv.o1Hint")),
|
||||
field("MTP", h("label", { class: "check" }, skipMtpCb, t("import.adv.skipMtp")))),
|
||||
field(t("models.form.systemPrompt"), sysPromptIn, t("models.form.systemPromptHint"))
|
||||
);
|
||||
|
||||
const node = h(
|
||||
"div",
|
||||
{},
|
||||
h("div", { class: "row" }, field(t("models.form.id"), idIn, existing ? null : t("models.form.idHint")), field(t("models.form.provider"), providerSel)),
|
||||
field(t("models.form.baseUrl"), baseUrlIn),
|
||||
field(t("models.form.model"), modelIn),
|
||||
field(t("models.form.apiKey"), h("div", { class: "flex wrap", style: "gap:8px;align-items:center" }, keyIn, testBtn, keyStatus), keyHint),
|
||||
field(t("models.form.model"), modelIn, t("models.form.modelHint")),
|
||||
dl,
|
||||
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
|
||||
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c)))),
|
||||
field(t("models.form.apiKeyEnv"), keyEnvIn, t("models.form.apiKeyEnvHint")),
|
||||
field(t("models.form.secret"), secretIn)
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap", style: "gap:10px" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check", style: "margin:0" }, cb, CAP_EMOJIS[c] || c)))),
|
||||
advParams
|
||||
);
|
||||
|
||||
const getValue = () => ({
|
||||
model: {
|
||||
id: existing ? m.id : idIn.value.trim(),
|
||||
provider: providerSel.value,
|
||||
base_url: baseUrlIn.value.trim(),
|
||||
model: modelIn.value.trim(),
|
||||
kind: kindSel.value,
|
||||
cost_tier: tierSel.value,
|
||||
price_in: parseFloat(priceInIn.value) || 0,
|
||||
price_out: parseFloat(priceOutIn.value) || 0,
|
||||
api_key_env: keyEnvIn.value.trim() || null,
|
||||
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
|
||||
},
|
||||
secret: secretIn.value,
|
||||
keyEnv: keyEnvIn.value.trim(),
|
||||
});
|
||||
const getValue = () => {
|
||||
const typedKey = keyIn.value.trim();
|
||||
const hasKey = !!typedKey || !!(existing && existing.api_key_env);
|
||||
const keyEnv = envFor();
|
||||
return {
|
||||
model: {
|
||||
id: existing ? m.id : idIn.value.trim(),
|
||||
provider: providerSel.value,
|
||||
base_url: baseUrlIn.value.trim(),
|
||||
model: modelIn.value.trim(),
|
||||
kind: kindSel.value,
|
||||
cost_tier: tierSel.value,
|
||||
price_in: parseFloat(priceInIn.value) || 0,
|
||||
price_out: parseFloat(priceOutIn.value) || 0,
|
||||
api_key_env: hasKey ? keyEnv : null,
|
||||
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: thinkIn.value !== "" ? parseInt(thinkIn.value) : null,
|
||||
o1: o1Sel.value || null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysPromptIn.value.trim() || null,
|
||||
},
|
||||
secret: typedKey,
|
||||
keyEnv,
|
||||
};
|
||||
};
|
||||
|
||||
return { node, getValue };
|
||||
}
|
||||
@@ -70,12 +241,14 @@ async function openModal(meta, existing, reload) {
|
||||
const { node, getValue } = modelForm(meta, existing);
|
||||
modal(existing ? t("models.form.editTitle") : t("models.form.addTitle"), node, async () => {
|
||||
const { model, secret, keyEnv } = getValue();
|
||||
// Forgot the id? Derive it from the model name so saving never errors on a
|
||||
// blank id.
|
||||
if (!model.id) model.id = slugId(model.model);
|
||||
if (!model.id) {
|
||||
toast(t("models.form.id") + "?", "bad");
|
||||
toast(t("models.form.idOrModel"), "bad");
|
||||
return false;
|
||||
}
|
||||
// strip null api_key_env so serde keeps it optional
|
||||
if (!model.api_key_env) delete model.api_key_env;
|
||||
if (!model.api_key_env) delete model.api_key_env; // keep serde field optional
|
||||
try {
|
||||
if (existing) await api.updateModel(existing.id, model);
|
||||
else await api.createModel(model);
|
||||
@@ -142,10 +315,63 @@ export async function renderModels() {
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
),
|
||||
benchCard()
|
||||
);
|
||||
}
|
||||
|
||||
// ── one-click benchmark of local .cmf files (cortiq bench --json) ──
|
||||
function benchCard() {
|
||||
const host = h("div", { class: "card" }, h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
|
||||
h("div", { class: "empty" }, t("common.loading")));
|
||||
let timer = null;
|
||||
async function paint() {
|
||||
let files = [];
|
||||
let runs = {};
|
||||
try {
|
||||
files = (await api.mediaModels()).models || [];
|
||||
for (const r of (await api.benchList()).runs || []) runs[r.model] = r;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const chat = files.filter((m) => m.modality === "chat");
|
||||
const anyRunning = Object.values(runs).some((r) => r.state === "running");
|
||||
const rows = chat.map((m) => {
|
||||
const r = runs[m.name];
|
||||
const go = h("button", { class: "btn sm ghost", disabled: anyRunning ? true : null }, "⏱ " + t("bench.run"));
|
||||
go.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.benchStart(m.name);
|
||||
paint();
|
||||
if (!timer) timer = setInterval(() => {
|
||||
if (!host.isConnected) { clearInterval(timer); return; }
|
||||
paint();
|
||||
}, 3000);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
});
|
||||
let result = h("span", { class: "muted" }, "—");
|
||||
if (r?.state === "running") result = h("span", { class: "flex" }, h("span", { class: "spinner" }), " ", t("bench.running"));
|
||||
else if (r?.state === "done") result = h("b", { class: "mono" }, (r.tok_s != null ? r.tok_s.toFixed(1) : "?") + " tok/s");
|
||||
else if (r?.state === "error") result = h("span", { class: "err", title: (r.log || []).slice(-2).join(" · ") }, t("bench.failed"));
|
||||
return h("div", { class: "flex bench-row" },
|
||||
h("span", { class: "mono grow" }, m.name),
|
||||
h("span", { class: "muted small" }, (m.size_bytes / 1e9).toFixed(2) + " GB"),
|
||||
result,
|
||||
go);
|
||||
});
|
||||
mount(host,
|
||||
h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
|
||||
...(rows.length ? rows : [h("div", { class: "empty" }, t("bench.none"))]));
|
||||
if (!anyRunning && timer) { clearInterval(timer); timer = null; }
|
||||
}
|
||||
paint();
|
||||
return host;
|
||||
}
|
||||
|
||||
await reload();
|
||||
return root;
|
||||
}
|
||||
@@ -168,7 +394,8 @@ function row(m, meta, reload) {
|
||||
return h(
|
||||
"tr",
|
||||
{},
|
||||
h("td", { class: "mono" }, m.id),
|
||||
h("td", { class: "mono" }, m.id,
|
||||
m.managed ? h("span", { class: "badge", style: "margin-left:6px", title: t("models.managedHint") }, t("models.managed")) : null),
|
||||
h("td", {}, m.provider),
|
||||
h("td", { class: "mono" }, m.model),
|
||||
h("td", {}, h("span", { class: "badge " + (m.cost_tier || "") }, m.cost_tier || "—")),
|
||||
@@ -182,7 +409,7 @@ function row(m, meta, reload) {
|
||||
"div",
|
||||
{ class: "flex", style: "justify-content:flex-end" },
|
||||
probeBtn,
|
||||
h("button", { class: "btn sm ghost", onClick: () => openModal(meta, m, reload) }, t("common.edit")),
|
||||
m.managed ? null : h("button", { class: "btn sm ghost", onClick: () => openModal(meta, m, reload) }, t("common.edit")),
|
||||
h(
|
||||
"button",
|
||||
{
|
||||
@@ -192,7 +419,7 @@ function row(m, meta, reload) {
|
||||
try {
|
||||
await api.deleteModel(m.id);
|
||||
toast(t("toast.deleted"), "good");
|
||||
reload();
|
||||
await reload();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
|
||||
Vendored
+335
@@ -0,0 +1,335 @@
|
||||
// First-run onboarding wizard: language → featured .cmf models (infosave) → HF search+convert → done.
|
||||
// No manual TOML edits: every action goes through the admin API.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t, getLang, setLang, LANGS } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured as fetchFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobBar,
|
||||
jobStatusText,
|
||||
fmtBytes,
|
||||
} from "./featured.js";
|
||||
|
||||
let step = 0; // 0 lang, 1 featured, 2 hf convert, 3 done
|
||||
let featured = [];
|
||||
let featuredLoaded = false;
|
||||
|
||||
function langStep(onNext) {
|
||||
const cards = LANGS.map((l) =>
|
||||
h("button", {
|
||||
class: "onb-lang" + (getLang() === l.code ? " sel" : ""),
|
||||
onClick: () => {
|
||||
setLang(l.code);
|
||||
cards.forEach((c) => c.classList.toggle("sel", c.dataset.code === l.code));
|
||||
toast(l.flag + " " + l.label, "ok");
|
||||
},
|
||||
"data-code": l.code,
|
||||
title: l.label,
|
||||
"aria-label": l.label,
|
||||
}, h("span", { class: "flag" }, l.flag), h("b", {}, l.label))
|
||||
);
|
||||
return h("div", { class: "onb-step" },
|
||||
h("h2", {}, t("onboarding.welcome")),
|
||||
h("p", { class: "muted" }, t("onboarding.welcomeSub")),
|
||||
h("div", { class: "onb-langs" }, ...cards),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn primary", onClick: onNext }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
export function renderOnboarding() {
|
||||
const root = h("div", { class: "onb-wrap" });
|
||||
let jobsPoll = null;
|
||||
let jobs = [];
|
||||
let sysSnapshot = null; // disk/RAM для плашек "не влезет" на карточках
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
let hosts = new Map(); // job id → card progress host (rebuilt on every render)
|
||||
let jobsHost = h("div", {});
|
||||
|
||||
// Job rows: bar, live status, and the actions that finish or undo a download.
|
||||
function jobsMini(list) {
|
||||
if (!list.length) return h("div", { class: "muted small" }, t("onboarding.noJobs"));
|
||||
return h("div", {},
|
||||
...list.map((j) => {
|
||||
const act = [];
|
||||
if (j.state === "done") {
|
||||
const btn = h("button", { class: "btn primary sm" }, "✓ " + t("import.register"));
|
||||
btn.addEventListener("click", async () => {
|
||||
btn.disabled = true;
|
||||
try {
|
||||
const r = await api.registerImport(j.id);
|
||||
toast(t("import.registered", { id: r.model_id }), "ok");
|
||||
} catch (e) { toast(e.message, "err"); btn.disabled = false; }
|
||||
});
|
||||
act.push(btn);
|
||||
}
|
||||
if (j.state === "running") {
|
||||
const cancel = h("button", { class: "btn ghost sm" }, "✕ " + t("import.cancel"));
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await api.cancelImport(j.id); toast(t("import.cancelling"), "ok"); await refreshJobs(); }
|
||||
catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
act.push(cancel);
|
||||
} else {
|
||||
// done / error / cancelled — remove the .cmf from disk and redo the step
|
||||
const del = h("button", { class: "btn ghost sm danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete"));
|
||||
del.addEventListener("click", () => deleteJob(j));
|
||||
act.push(del);
|
||||
}
|
||||
const known = ["running", "done", "error", "cancelled"].includes(j.state);
|
||||
return h("div", { class: "job", style: "margin:8px 0" },
|
||||
h("div", { class: "job-head" },
|
||||
h("b", {}, j.name || j.repo || j.id),
|
||||
h("span", { class: "badge" + (j.state === "done" ? " ok" : j.state === "running" ? "" : " bad") }, known ? t("import.state." + j.state) : j.state)
|
||||
),
|
||||
jobBar(j),
|
||||
h("div", { class: "muted small" }, jobStatusText(j)),
|
||||
h("pre", { class: "conv-log", style: "max-height:80px" }, (j.log || []).slice(-6).join("\n")),
|
||||
act.length ? h("div", { class: "actions" }, ...act) : null
|
||||
);
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
// Repaint only the live bits (card hosts + job list) — cheap enough for 1.5s.
|
||||
function paintJobs() {
|
||||
mount(jobsHost, jobs.length
|
||||
? h("div", { class: "card" }, h("div", { class: "card-title" }, t("import.jobs")), jobsMini(jobs))
|
||||
: h("div", {}));
|
||||
paintFeaturedHosts(hosts, jobs, async (j) => {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "ok");
|
||||
await refreshJobs();
|
||||
});
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
try { jobs = (await api.listImports()).jobs || []; } catch { jobs = []; }
|
||||
// A job that just settled changes what the cards offer (✓ + delete), so the
|
||||
// featured list has to be re-read — but only on the transition, not per poll.
|
||||
const settled = jobs.some((j) => {
|
||||
const prev = jobStates.get(j.id);
|
||||
return prev === "running" && j.state !== "running";
|
||||
});
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) {
|
||||
await loadFeatured();
|
||||
render(); // rebuilds the cards (✓ + delete) and repaints the job rows
|
||||
} else {
|
||||
paintJobs();
|
||||
}
|
||||
if (jobs.some((j) => j.state === "running")) {
|
||||
if (!jobsPoll) jobsPoll = setInterval(refreshJobs, 1500);
|
||||
} else {
|
||||
if (jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; }
|
||||
}
|
||||
}
|
||||
|
||||
async function loadFeatured() {
|
||||
featured = await fetchFeatured();
|
||||
try { sysSnapshot = await api.system(); } catch { /* optional */ }
|
||||
featuredLoaded = true;
|
||||
}
|
||||
|
||||
function topNav(prev, next, extra) {
|
||||
return h("div", { class: "actions", style: "margin-bottom:12px; position:sticky; top:0; background:var(--bg); z-index:2; padding:8px 0; border-bottom:1px solid var(--border-soft); backdrop-filter:blur(4px)" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
extra ? extra : h("div", { style: "flex:1" }),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
}
|
||||
|
||||
async function startDownload(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function handleDelete(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
// Delete straight from a job row: the gateway knows that job's output path.
|
||||
async function deleteJob(j) {
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: j.name || j.repo }))) return;
|
||||
try {
|
||||
await api.deleteImport(j.id);
|
||||
toast(t("import.deleted"), "ok");
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
function render() {
|
||||
const dots = [0, 1, 2, 3].map((i) => h("span", { class: "dot" + (i === step ? " on" : i < step ? " done" : "") }));
|
||||
const header = h("div", { class: "onb-header" },
|
||||
h("div", { class: "onb-dots" }, ...dots),
|
||||
h("div", { class: "onb-title" }, t("onboarding.title") + ` — ${step + 1}/4`)
|
||||
);
|
||||
|
||||
let body;
|
||||
const next = () => { step = Math.min(3, step + 1); render(); };
|
||||
const prev = () => { step = Math.max(0, step - 1); render(); };
|
||||
hosts = new Map(); // rebuilt by featuredCard below
|
||||
|
||||
if (step === 0) {
|
||||
const top = h("div", { class: "actions", style: "margin-bottom:12px; justify-content:flex-end" },
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
body = h("div", {},
|
||||
top,
|
||||
langStep(next)
|
||||
);
|
||||
} else if (step === 1) {
|
||||
const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip")));
|
||||
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete, sys: sysSnapshot };
|
||||
// the list is fetched from HuggingFace — say so instead of showing a blank grid
|
||||
const grid = featured.length
|
||||
? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx)))
|
||||
: h("div", { class: "muted pad" }, featuredLoaded ? t("import.noResults") : t("import.searching"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
top,
|
||||
h("h2", {}, "📦 " + t("onboarding.featuredTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.featuredSub")),
|
||||
grid,
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn", onClick: async () => { await loadFeatured(); render(); await refreshJobs(); } }, t("common.refresh")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →"),
|
||||
h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip"))
|
||||
)
|
||||
);
|
||||
} else if (step === 2) {
|
||||
const top2 = topNav(prev, next);
|
||||
// lightweight HF search inline
|
||||
const inp = h("input", { placeholder: t("import.searchPlaceholder"), style: "width:100%" });
|
||||
const res = h("div", { class: "hf-grid" });
|
||||
const quantSel = h("select", {},
|
||||
h("option", { value: "Q8_2F" }, "Q8_2F — " + t("import.q.q8_2f")),
|
||||
h("option", { value: "Q4_BLOCK" }, "Q4 — " + t("import.q.q4")),
|
||||
h("option", { value: "Q1T" }, "Q1T — " + t("import.q.q1t")),
|
||||
h("option", { value: "Q1" }, "Q1 — " + t("import.q.q1")),
|
||||
);
|
||||
let selected = null;
|
||||
let debounce = null;
|
||||
async function doSearch() {
|
||||
const q = inp.value.trim();
|
||||
mount(res, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
try {
|
||||
const r = await api.hfSearch(q, 16);
|
||||
const models = r.models || [];
|
||||
if (!models.length) { mount(res, h("div", { class: "muted pad" }, t("import.noResults"))); return; }
|
||||
mount(res, ...models.map((m) => {
|
||||
const card = h("div", { class: "hf-card" + (selected && selected.id === m.id ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, m.id),
|
||||
h("div", { class: "hf-meta" }, h("span", { class: "badge" }, m.pipeline_tag || "model"), h("span", { class: "muted" }, "♥ " + (m.likes ?? 0)))
|
||||
);
|
||||
card.addEventListener("click", () => { selected = m; doSearch(); });
|
||||
return card;
|
||||
}));
|
||||
} catch (e) { mount(res, h("div", { class: "err" }, String(e.message))); }
|
||||
}
|
||||
inp.addEventListener("input", () => { clearTimeout(debounce); debounce = setTimeout(doSearch, 300); });
|
||||
const go = h("button", { class: "btn primary" }, "⬇ " + t("import.convert"));
|
||||
go.addEventListener("click", async () => {
|
||||
if (!selected) { toast(t("import.noResults"), "warn"); return; }
|
||||
go.disabled = true;
|
||||
try {
|
||||
await api.startImport({ repo: selected.id, quant: quantSel.value, name: selected.id.split("/").pop().toLowerCase() });
|
||||
toast(t("import.started"), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
go.disabled = false;
|
||||
});
|
||||
// auto initial search
|
||||
setTimeout(doSearch, 100);
|
||||
body = h("div", { class: "onb-step" },
|
||||
top2,
|
||||
h("h2", {}, "🔍 " + t("onboarding.hfTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.hfSub")),
|
||||
h("div", { class: "card" }, inp, res),
|
||||
h("div", { class: "card", style: "margin-top:12px" },
|
||||
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("import.quant")), quantSel)),
|
||||
h("div", { class: "actions" }, go)
|
||||
),
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
} else {
|
||||
// done — finishing connects every completed download as a local model
|
||||
const ready = jobs.filter((j) => j.state === "done");
|
||||
const doneBtn = h("button", { class: "btn primary" }, t("onboarding.finish"));
|
||||
doneBtn.addEventListener("click", async () => {
|
||||
doneBtn.disabled = true;
|
||||
let r = null;
|
||||
try { r = await api.setupComplete(getLang(), true); } catch (e) { toast(e.message, "err"); }
|
||||
if (r && r.registered && r.registered.length) {
|
||||
toast(t("onboarding.registeredN", { n: r.registered.length, ids: r.registered.join(", ") }), "ok");
|
||||
if (!r.runtime_installed) toast(t("onboarding.runtimeMissing"), "warn");
|
||||
// let the toast be readable before the reload swaps the page
|
||||
await new Promise((res) => setTimeout(res, 900));
|
||||
}
|
||||
// land where the reward is: try the model right away
|
||||
location.hash = "#/playground";
|
||||
location.reload();
|
||||
});
|
||||
const skipBtn = h("button", { class: "btn ghost", onClick: async () => {
|
||||
try { await api.setupComplete(getLang(), false); } catch {}
|
||||
location.hash = "#/dashboard";
|
||||
} }, t("onboarding.skipSetup"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
h("h2", {}, "✅ " + t("onboarding.doneTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.doneSub")),
|
||||
ready.length
|
||||
? h("div", { class: "card" },
|
||||
h("div", { class: "card-title" }, t("onboarding.willRegister")),
|
||||
...ready.map((j) => h("div", { class: "muted small" }, "• cmf-" + (j.name || j.repo) + (j.size_bytes ? ` · ${fmtBytes(j.size_bytes)}` : "")))
|
||||
)
|
||||
: h("div", { class: "muted small" }, t("onboarding.nothingToRegister")),
|
||||
jobsHost,
|
||||
h("div", { class: "callout" }, t("onboarding.doneHint")),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
skipBtn,
|
||||
doneBtn
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
mount(root, h("div", { class: "onb" }, header, body));
|
||||
paintJobs(); // fill the freshly built hosts with whatever we already know
|
||||
if (jobs.some((j) => j.state === "running") && !jobsPoll) {
|
||||
jobsPoll = setInterval(refreshJobs, 1500);
|
||||
}
|
||||
}
|
||||
|
||||
loadFeatured().then(() => { render(); refreshJobs(); });
|
||||
render();
|
||||
// cleanup poll on detach
|
||||
const obs = new MutationObserver(() => { if (!root.isConnected && jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; obs.disconnect(); } });
|
||||
obs.observe(document.body, { childList: true, subtree: true });
|
||||
return root;
|
||||
}
|
||||
Vendored
+544
-39
@@ -1,17 +1,64 @@
|
||||
// Playground — send a prompt through the live pipeline, inspect routing decision.
|
||||
import { h, mount, money, ms } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api, testStream } from "../api.js";
|
||||
// Playground — three tabs over the live gateway:
|
||||
// chat → the routing pipeline (with streaming, token counts, tok/s, TTFT)
|
||||
// image → `cortiq imagine` over a local text-to-image .cmf (Lumina)
|
||||
// video → `cortiq animate` over a local text-to-video .cmf (MiniMax-H3)
|
||||
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
|
||||
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
|
||||
import { h, mount, toast, money, moneyEl, ms } from "../ui.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
|
||||
|
||||
function fmtBytes(n) {
|
||||
if (!n && n !== 0) return "—";
|
||||
if (n >= 1e9) return (n / 1e9).toFixed(2) + " GB";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + " MB";
|
||||
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
|
||||
return n + " B";
|
||||
}
|
||||
function kv(k, v) {
|
||||
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
|
||||
}
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
|
||||
export async function renderPlayground() {
|
||||
const meta = appState.meta || (await api.meta());
|
||||
const modelsData = await api.listModels();
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
let mediaModels = [];
|
||||
try {
|
||||
mediaModels = (await api.mediaModels()).models || [];
|
||||
} catch {
|
||||
/* media API unavailable — tabs will show the hint */
|
||||
}
|
||||
const imageModels = mediaModels.filter((m) => m.modality === "image");
|
||||
const videoModels = mediaModels.filter((m) => m.modality === "video");
|
||||
|
||||
let mode = "auto";
|
||||
const root = h("div");
|
||||
let tab = "chat";
|
||||
|
||||
// ── tabs ──
|
||||
const tabsSeg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
h("button", { class: "active", "data-tab": "chat" }, "💬 " + t("pg.tab.chat")),
|
||||
h("button", { "data-tab": "image" }, "🖼 " + t("pg.tab.image")),
|
||||
h("button", { "data-tab": "video" }, "🎬 " + t("pg.tab.video"))
|
||||
);
|
||||
const panelHost = h("div", {});
|
||||
tabsSeg.querySelectorAll("button").forEach((b) =>
|
||||
b.addEventListener("click", () => {
|
||||
tab = b.dataset.tab;
|
||||
tabsSeg.querySelectorAll("button").forEach((x) => x.classList.toggle("active", x === b));
|
||||
mount(panelHost, panels[tab]);
|
||||
})
|
||||
);
|
||||
|
||||
// ════════════════════════════ chat panel ════════════════════════════
|
||||
let mode = "auto";
|
||||
const sysIn = h("textarea", { placeholder: t("pg.systemPlaceholder"), rows: 2 });
|
||||
const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 });
|
||||
const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 });
|
||||
@@ -44,16 +91,33 @@ export async function renderPlayground() {
|
||||
modeExtra = next;
|
||||
}
|
||||
|
||||
const out = h("div", { class: "answer" }, h("div", { class: "empty" }, t("pg.empty")));
|
||||
const chatLog = []; // the running conversation: {role, content}
|
||||
const out = h("div", { class: "answer chat-log" }, h("div", { class: "empty" }, t("pg.empty")));
|
||||
const decision = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
|
||||
|
||||
function bubble(role, text) {
|
||||
return h("div", { class: "chat-msg " + role }, h("div", { class: "chat-bubble" }, text));
|
||||
}
|
||||
function renderChat(liveText) {
|
||||
const items = chatLog.map((m) => bubble(m.role, m.content));
|
||||
if (liveText != null) items.push(bubble("assistant", liveText || "…"));
|
||||
mount(out, ...(items.length ? items : [h("div", { class: "empty" }, t("pg.empty"))]));
|
||||
out.scrollTop = out.scrollHeight;
|
||||
}
|
||||
const clearBtn = h("button", { class: "btn ghost sm" }, "🧹 " + t("pg.clearChat"));
|
||||
clearBtn.addEventListener("click", () => {
|
||||
chatLog.length = 0;
|
||||
renderChat();
|
||||
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
|
||||
});
|
||||
|
||||
const sendBtn = h("button", { class: "btn primary" }, t("pg.send"));
|
||||
sendBtn.addEventListener("click", async () => {
|
||||
const prompt = promptIn.value.trim();
|
||||
if (!prompt) return;
|
||||
const messages = [];
|
||||
if (sysIn.value.trim()) messages.push({ role: "system", content: sysIn.value.trim() });
|
||||
messages.push({ role: "user", content: prompt });
|
||||
messages.push(...chatLog, { role: "user", content: prompt });
|
||||
let model = "cortiq-auto";
|
||||
if (mode === "auto") {
|
||||
if (profileSel.value) model = "cortiq-auto:" + profileSel.value;
|
||||
@@ -62,7 +126,9 @@ export async function renderPlayground() {
|
||||
}
|
||||
sendBtn.disabled = true;
|
||||
sendBtn.textContent = t("pg.sending");
|
||||
mount(out, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.sending")));
|
||||
chatLog.push({ role: "user", content: prompt });
|
||||
promptIn.value = "";
|
||||
renderChat("");
|
||||
const body = {
|
||||
model,
|
||||
messages,
|
||||
@@ -73,12 +139,13 @@ export async function renderPlayground() {
|
||||
if (streamCb.checked) {
|
||||
const started = performance.now();
|
||||
let acc = "";
|
||||
out.textContent = "";
|
||||
const info = await testStream(body, (d) => {
|
||||
acc += d;
|
||||
out.textContent = acc;
|
||||
renderChat(acc);
|
||||
});
|
||||
out.textContent = acc || "—";
|
||||
chatLog.push({ role: "assistant", content: acc || "—" });
|
||||
renderChat();
|
||||
const latency = Math.round(performance.now() - started);
|
||||
renderDecision({
|
||||
cortiq: {
|
||||
task_label: info.task_label,
|
||||
@@ -88,37 +155,58 @@ export async function renderPlayground() {
|
||||
cost_usd: info.cost_usd,
|
||||
failover: false,
|
||||
},
|
||||
usage: {},
|
||||
latency_ms: Math.round(performance.now() - started),
|
||||
usage: info.usage || {},
|
||||
usage_estimated: info.usage_estimated,
|
||||
latency_ms: latency,
|
||||
ttft_ms: info.ttft_ms,
|
||||
});
|
||||
} else {
|
||||
const r = await api.test(body);
|
||||
if (r.ok) {
|
||||
mount(out, r.answer || h("span", { class: "muted" }, "—"));
|
||||
chatLog.push({ role: "assistant", content: r.answer || "—" });
|
||||
renderChat();
|
||||
renderDecision(r);
|
||||
} else {
|
||||
mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
|
||||
chatLog.pop(); // the user turn failed — let them retry it
|
||||
renderChat();
|
||||
promptIn.value = prompt;
|
||||
mount(out, out.firstChild || "", h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
|
||||
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
mount(out, h("div", { class: "callout" }, String(e.message || e)));
|
||||
chatLog.pop();
|
||||
renderChat();
|
||||
promptIn.value = prompt;
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
sendBtn.disabled = false;
|
||||
sendBtn.textContent = t("pg.send");
|
||||
});
|
||||
// Cmd/Ctrl+Enter sends
|
||||
promptIn.addEventListener("keydown", (e) => {
|
||||
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
|
||||
e.preventDefault();
|
||||
sendBtn.click();
|
||||
}
|
||||
});
|
||||
|
||||
function kv(k, v) {
|
||||
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
|
||||
}
|
||||
function renderDecision(r) {
|
||||
const c = r.cortiq || {};
|
||||
const score = c.complexity?.score ?? 0;
|
||||
const u = r.usage || {};
|
||||
const haveUsage = u.completion_tokens != null;
|
||||
// generation speed over the produce phase (after first token when known)
|
||||
let tokS = null;
|
||||
if (haveUsage && r.latency_ms) {
|
||||
const genMs = r.ttft_ms != null ? Math.max(1, r.latency_ms - r.ttft_ms) : r.latency_ms;
|
||||
tokS = (u.completion_tokens / (genMs / 1000)).toFixed(1);
|
||||
}
|
||||
mount(
|
||||
decision,
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
|
||||
kv(t("pg.task"), c.task_label || "—"),
|
||||
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
|
||||
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, trTier(c.complexity.tier)) : "—"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "kv", style: "display:block" },
|
||||
@@ -127,17 +215,278 @@ export async function renderPlayground() {
|
||||
),
|
||||
kv(t("pg.selected"), h("span", { class: "mono" }, c.selected_model || "—")),
|
||||
kv(t("pg.source"), c.route_source || "—"),
|
||||
kv(t("pg.tokens"), `${r.usage?.prompt_tokens ?? 0}/${r.usage?.completion_tokens ?? 0}`),
|
||||
kv(t("pg.cost"), money(c.cost_usd)),
|
||||
kv(t("pg.tokens"), haveUsage ? `${r.usage_estimated ? "≈ " : ""}${u.prompt_tokens ?? 0} / ${u.completion_tokens ?? 0}` : "—"),
|
||||
kv(t("pg.tokSpeed"), tokS != null ? tokS + " tok/s" : "—"),
|
||||
kv(t("pg.ttft"), r.ttft_ms != null ? ms(r.ttft_ms) : "—"),
|
||||
kv(t("pg.cost"), moneyEl(c.cost_usd)),
|
||||
kv(t("pg.latency"), ms(r.latency_ms)),
|
||||
kv(t("pg.failover"), c.failover ? "yes" : "no")
|
||||
);
|
||||
}
|
||||
|
||||
mount(
|
||||
root,
|
||||
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
|
||||
const chatPanel = h(
|
||||
"div",
|
||||
{ class: "pg" },
|
||||
h(
|
||||
"div",
|
||||
{},
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
field(t("pg.system"), sysIn),
|
||||
field(t("pg.prompt"), promptIn),
|
||||
field(t("pg.mode"), modeSeg),
|
||||
modeExtra,
|
||||
h("div", { class: "row" }, field(t("pg.temp"), tempIn), field(t("pg.maxTokens"), maxTokIn)),
|
||||
h(
|
||||
"div",
|
||||
{ class: "flex", style: "justify-content:space-between; gap:8px" },
|
||||
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
|
||||
h("div", { class: "flex", style: "gap:8px" }, clearBtn, sendBtn)
|
||||
)
|
||||
),
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
|
||||
),
|
||||
decision
|
||||
);
|
||||
|
||||
// ═══════════════════════ shared media machinery ═══════════════════════
|
||||
let mediaPoll = null;
|
||||
function stopMediaPoll() {
|
||||
if (mediaPoll) clearInterval(mediaPoll);
|
||||
mediaPoll = null;
|
||||
}
|
||||
// one generation at a time server-side → one active job client-side
|
||||
function watchJob(id, ui, onDone) {
|
||||
stopMediaPoll();
|
||||
const started = performance.now();
|
||||
mediaPoll = setInterval(async () => {
|
||||
if (!root.isConnected) return stopMediaPoll();
|
||||
let j;
|
||||
try {
|
||||
j = await api.mediaJob(id);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const elapsed = (performance.now() - started) / 1000;
|
||||
const pct = j.progress != null ? Math.round(j.progress * 100) : null;
|
||||
// honest ETA once there is enough signal: total ≈ elapsed / progress
|
||||
let eta = "";
|
||||
if (j.state === "running" && j.progress > 0.03) {
|
||||
const left = elapsed / j.progress - elapsed;
|
||||
if (left > 1) eta = ` · ${t("pg.eta")} ~${left > 90 ? Math.round(left / 60) + " " + t("pg.min") : Math.round(left) + " s"}`;
|
||||
}
|
||||
let statusLine;
|
||||
if (j.state === "queued") {
|
||||
const pos = (j.queue_position ?? 0) + 1;
|
||||
statusLine = t("pg.queued", { n: pos });
|
||||
} else {
|
||||
statusLine =
|
||||
(j.phase || j.state) +
|
||||
(pct != null && j.state === "running" ? ` · ${pct}%` : "") +
|
||||
` · ${elapsed.toFixed(0)}s` + eta;
|
||||
}
|
||||
mount(
|
||||
ui.bar,
|
||||
h("div", { class: "pbar" },
|
||||
j.state === "running"
|
||||
? (pct != null
|
||||
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
|
||||
: h("div", { class: "pbar-fill run" }))
|
||||
: j.state === "queued"
|
||||
? h("div", { class: "pbar-fill run" })
|
||||
: h("div", { class: "pbar-fill " + (j.state === "done" ? "ok" : "err"), style: "width:100%" })
|
||||
),
|
||||
h("div", { class: "muted small" }, statusLine),
|
||||
(j.log || []).length && j.state === "running" ? h("pre", { class: "conv-log", style: "max-height:90px" }, j.log.slice(-5).join("\n")) : null
|
||||
);
|
||||
if (j.state !== "running" && j.state !== "queued") {
|
||||
stopMediaPoll();
|
||||
ui.goBtn.disabled = false;
|
||||
ui.cancelBtn.style.display = "none";
|
||||
if (j.state === "done") onDone(j);
|
||||
else if (j.state === "error") toast(t("pg.genFailed"), "err");
|
||||
}
|
||||
}, 1000);
|
||||
}
|
||||
function mediaControls(goBtn, onCancelId) {
|
||||
const cancelBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
|
||||
cancelBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.mediaCancel(onCancelId.id);
|
||||
} catch (e) {
|
||||
toast(e.message, "err");
|
||||
}
|
||||
});
|
||||
return cancelBtn;
|
||||
}
|
||||
function noModelsHint(kindLabel) {
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "callout" },
|
||||
t("pg.noMediaModels", { kind: kindLabel }),
|
||||
" ",
|
||||
h("a", { href: "#/import" }, t("settings.cmf.importLink") + " ↗"))
|
||||
);
|
||||
}
|
||||
function dice(seedIn) {
|
||||
const b = h("button", { class: "btn ghost sm", title: t("pg.randomSeed") }, "🎲");
|
||||
b.addEventListener("click", () => (seedIn.value = Math.floor(Math.random() * 1e9)));
|
||||
return b;
|
||||
}
|
||||
|
||||
// ── gallery: past generations of one kind, loaded lazily ──
|
||||
function buildGallery(kind, onOpen) {
|
||||
const host = h("div", {});
|
||||
async function refresh() {
|
||||
let jobs = [];
|
||||
try {
|
||||
jobs = ((await api.mediaJobs()).jobs || []).filter((j) => j.kind === kind && j.state === "done");
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!jobs.length) {
|
||||
mount(host);
|
||||
return;
|
||||
}
|
||||
const grid = h("div", { class: "gal-grid" });
|
||||
for (const j of jobs.slice(0, 12)) {
|
||||
const del = h("button", { class: "gal-del", title: t("import.delete") }, "×");
|
||||
del.addEventListener("click", async (e) => {
|
||||
e.stopPropagation();
|
||||
if (!window.confirm(t("pg.galDeleteConfirm"))) return;
|
||||
try {
|
||||
await api.mediaDelete(j.id);
|
||||
refresh();
|
||||
} catch (err) {
|
||||
toast(err.message, "err");
|
||||
}
|
||||
});
|
||||
const cap = h("div", { class: "gal-cap" }, `${j.width}×${j.height} · seed ${j.seed}`);
|
||||
const item = h("div", { class: "gal-item", title: j.prompt }, del, cap);
|
||||
if (kind === "image") {
|
||||
// lazy thumbnail: decode the PPM only when the item scrolls into view
|
||||
const thumb = h("canvas", { class: "gal-thumb" });
|
||||
item.prepend(thumb);
|
||||
const io = new IntersectionObserver(async (entries) => {
|
||||
if (!entries.some((x) => x.isIntersecting)) return;
|
||||
io.disconnect();
|
||||
try {
|
||||
const buf = await mediaFileBytes(j.id, "image");
|
||||
const full = ppmToCanvas(buf);
|
||||
thumb.width = 128;
|
||||
thumb.height = Math.round((128 * full.height) / full.width) || 128;
|
||||
thumb.getContext("2d").drawImage(full, 0, 0, thumb.width, thumb.height);
|
||||
} catch {
|
||||
/* file gone */
|
||||
}
|
||||
});
|
||||
io.observe(item);
|
||||
} else {
|
||||
item.prepend(h("div", { class: "gal-thumb gal-video" }, "🎬"));
|
||||
}
|
||||
item.addEventListener("click", () => onOpen(j));
|
||||
grid.appendChild(item);
|
||||
}
|
||||
mount(host, h("div", { class: "card-title", style: "margin-top:14px" }, t("pg.gallery")), grid);
|
||||
}
|
||||
refresh();
|
||||
return { host, refresh };
|
||||
}
|
||||
|
||||
// ════════════════════════════ image panel ════════════════════════════
|
||||
let imagePanel;
|
||||
if (!imageModels.length) {
|
||||
imagePanel = noModelsHint(t("pg.tab.image"));
|
||||
} else {
|
||||
const imModel = h("select", {}, ...imageModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
|
||||
const imPrompt = h("textarea", { placeholder: t("pg.imagePromptPlaceholder"), rows: 3 });
|
||||
const imW = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
|
||||
const imH = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
|
||||
const imSteps = h("input", { type: "number", min: 1, max: 100, value: 30 });
|
||||
const imCfg = h("input", { type: "number", step: "0.5", value: 4 });
|
||||
const imSeed = h("input", { type: "number", value: 42 });
|
||||
const imBar = h("div", {});
|
||||
const imOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.imageEmpty")));
|
||||
const imMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
|
||||
const imGo = h("button", { class: "btn primary" }, "🖼 " + t("pg.generate"));
|
||||
const imCancelRef = { id: null };
|
||||
const imCancel = mediaControls(imGo, imCancelRef);
|
||||
|
||||
imGo.addEventListener("click", async () => {
|
||||
const prompt = imPrompt.value.trim();
|
||||
if (!prompt) return;
|
||||
imGo.disabled = true;
|
||||
imCancel.style.display = "";
|
||||
mount(imOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
|
||||
const params = {
|
||||
kind: "image",
|
||||
model: imModel.value,
|
||||
prompt,
|
||||
width: +imW.value,
|
||||
height: +imH.value,
|
||||
steps: +imSteps.value || 30,
|
||||
cfg: parseFloat(imCfg.value) || 4,
|
||||
seed: +imSeed.value || 42,
|
||||
};
|
||||
try {
|
||||
const r = await api.mediaGenerate(params);
|
||||
imCancelRef.id = r.job;
|
||||
const t0 = performance.now();
|
||||
watchJob(r.job, { bar: imBar, goBtn: imGo, cancelBtn: imCancel }, async (j) => {
|
||||
mount(imBar);
|
||||
const secs = (performance.now() - t0) / 1000;
|
||||
await showImageResult(j, secs);
|
||||
imGallery.refresh();
|
||||
});
|
||||
} catch (e) {
|
||||
imGo.disabled = false;
|
||||
imCancel.style.display = "none";
|
||||
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
});
|
||||
|
||||
async function showImageResult(j, secs) {
|
||||
try {
|
||||
const buf = await mediaFileBytes(j.id, "image");
|
||||
const canvas = ppmToCanvas(buf);
|
||||
canvas.className = "media-canvas";
|
||||
const dl = h("button", { class: "btn sm" }, "⬇ PNG");
|
||||
dl.addEventListener("click", () =>
|
||||
canvas.toBlob((b) => {
|
||||
const a = h("a", { href: URL.createObjectURL(b), download: `cortiq-${j.id}.png` });
|
||||
a.click();
|
||||
}, "image/png")
|
||||
);
|
||||
mount(imOut, canvas, h("div", { class: "actions" }, dl));
|
||||
const rows = [
|
||||
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
|
||||
kv(t("pg.resolution"), `${j.width}×${j.height}`),
|
||||
kv(t("pg.steps"), String(j.steps)),
|
||||
kv(t("pg.seed"), String(j.seed)),
|
||||
kv(t("pg.fileSize"), fmtBytes(j.out_bytes)),
|
||||
];
|
||||
if (secs != null) {
|
||||
rows.push(kv(t("pg.genTime"), secs.toFixed(1) + " s"));
|
||||
rows.push(kv(t("pg.timePerStep"), (secs / j.steps).toFixed(2) + " s"));
|
||||
} else if (j.finished && j.started) {
|
||||
const dur = j.finished - j.started;
|
||||
rows.push(kv(t("pg.genTime"), dur + " s"));
|
||||
rows.push(kv(t("pg.timePerStep"), (dur / j.steps).toFixed(2) + " s"));
|
||||
}
|
||||
mount(imMetrics, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), ...rows);
|
||||
} catch (e) {
|
||||
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
}
|
||||
const imGallery = buildGallery("image", (j) => {
|
||||
imSeed.value = j.seed;
|
||||
imSteps.value = j.steps;
|
||||
showImageResult(j, null);
|
||||
});
|
||||
|
||||
imagePanel = h(
|
||||
"div",
|
||||
{ class: "pg" },
|
||||
h(
|
||||
@@ -146,22 +495,178 @@ export async function renderPlayground() {
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("label", { class: "field" }, h("span", {}, t("pg.system")), sysIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("pg.prompt")), promptIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("pg.mode")), modeSeg),
|
||||
modeExtra,
|
||||
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("pg.temp")), tempIn), h("label", { class: "field" }, h("span", {}, t("pg.maxTokens")), maxTokIn)),
|
||||
h(
|
||||
"div",
|
||||
{ class: "flex", style: "justify-content:space-between" },
|
||||
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
|
||||
sendBtn
|
||||
)
|
||||
field(t("pg.model"), imModel),
|
||||
field(t("pg.prompt"), imPrompt),
|
||||
h("div", { class: "row" }, field(t("pg.width"), imW), field(t("pg.height"), imH)),
|
||||
h("div", { class: "row" },
|
||||
field(t("pg.steps"), imSteps),
|
||||
field("CFG", imCfg, t("pg.cfgHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, imSeed, dice(imSeed)))),
|
||||
h("div", { class: "actions" }, imCancel, imGo),
|
||||
imBar
|
||||
),
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), imOut, imGallery.host)
|
||||
),
|
||||
decision
|
||||
)
|
||||
imMetrics
|
||||
);
|
||||
}
|
||||
|
||||
// ════════════════════════════ video panel ════════════════════════════
|
||||
let videoPanel;
|
||||
if (!videoModels.length) {
|
||||
videoPanel = noModelsHint(t("pg.tab.video"));
|
||||
} else {
|
||||
const vModel = h("select", {}, ...videoModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
|
||||
const vPrompt = h("textarea", { placeholder: t("pg.videoPromptPlaceholder"), rows: 3 });
|
||||
const vW = h("input", { type: "number", step: 32, value: 512 });
|
||||
const vH = h("input", { type: "number", step: 32, value: 288 });
|
||||
const vFrames = h("input", { type: "number", value: 39 });
|
||||
const vSteps = h("input", { type: "number", value: 4 });
|
||||
const vSeed = h("input", { type: "number", value: 42 });
|
||||
const vQuality = h("input", { type: "number", min: 10, max: 100, value: 92 });
|
||||
const vFirst = h("input", { type: "file", accept: "image/*" });
|
||||
const vLast = h("input", { type: "file", accept: "image/*" });
|
||||
const vBar = h("div", {});
|
||||
const vOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.videoEmpty")));
|
||||
const vMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
|
||||
const vGo = h("button", { class: "btn primary" }, "🎬 " + t("pg.generate"));
|
||||
const vCancelRef = { id: null };
|
||||
const vCancel = mediaControls(vGo, vCancelRef);
|
||||
|
||||
vGo.addEventListener("click", async () => {
|
||||
const prompt = vPrompt.value.trim();
|
||||
if (!prompt) return;
|
||||
vGo.disabled = true;
|
||||
vCancel.style.display = "";
|
||||
mount(vOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
|
||||
const params = {
|
||||
kind: "video",
|
||||
model: vModel.value,
|
||||
prompt,
|
||||
width: +vW.value || 512,
|
||||
height: +vH.value || 288,
|
||||
frames: +vFrames.value || 39,
|
||||
steps: +vSteps.value || 4,
|
||||
seed: +vSeed.value || 42,
|
||||
quality: +vQuality.value || 92,
|
||||
};
|
||||
try {
|
||||
if (vFirst.files[0]) params.first_frame_b64 = await fileToPpmB64(vFirst.files[0], params.width, params.height);
|
||||
if (vLast.files[0]) params.last_frame_b64 = await fileToPpmB64(vLast.files[0], params.width, params.height);
|
||||
const r = await api.mediaGenerate(params);
|
||||
vCancelRef.id = r.job;
|
||||
const t0 = performance.now();
|
||||
watchJob(r.job, { bar: vBar, goBtn: vGo, cancelBtn: vCancel }, async (j) => {
|
||||
mount(vBar);
|
||||
vGallery.refresh();
|
||||
const secs = (performance.now() - t0) / 1000;
|
||||
try {
|
||||
await showVideoResult(j, vOut);
|
||||
const durS = (j.frames || 0) / 24;
|
||||
mount(
|
||||
vMetrics,
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))),
|
||||
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
|
||||
kv(t("pg.resolution"), `${j.width}×${j.height}`),
|
||||
kv(t("pg.frames"), `${j.frames} @ 24 fps`),
|
||||
kv(t("pg.duration"), durS.toFixed(1) + " s"),
|
||||
kv(t("pg.steps"), String(j.steps)),
|
||||
kv(t("pg.seed"), String(j.seed)),
|
||||
kv(t("pg.genTime"), secs.toFixed(1) + " s"),
|
||||
kv(t("pg.timePerFrame"), j.frames ? (secs / j.frames).toFixed(2) + " s" : "—"),
|
||||
kv(t("pg.fileSize"), fmtBytes(j.out_bytes))
|
||||
);
|
||||
} catch (e) {
|
||||
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
});
|
||||
} catch (e) {
|
||||
vGo.disabled = false;
|
||||
vCancel.style.display = "none";
|
||||
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
});
|
||||
|
||||
async function showVideoResult(j, host) {
|
||||
const downloads = h("div", { class: "actions" });
|
||||
for (const kind of j.outputs || []) {
|
||||
const label = { video: "⬇ AVI", audio: "⬇ WAV", mp4: "⬇ MP4" }[kind];
|
||||
if (!label) continue;
|
||||
const b = h("button", { class: "btn sm" }, label);
|
||||
b.addEventListener("click", async () => {
|
||||
const { url } = await mediaFileUrl(j.id, kind);
|
||||
const ext = kind === "video" ? "avi" : kind === "audio" ? "wav" : "mp4";
|
||||
h("a", { href: url, download: `cortiq-${j.id}.${ext}` }).click();
|
||||
});
|
||||
downloads.appendChild(b);
|
||||
}
|
||||
if ((j.outputs || []).includes("mp4")) {
|
||||
// ffmpeg was present server-side → native playback
|
||||
const { url } = await mediaFileUrl(j.id, "mp4");
|
||||
mount(host, h("video", { src: url, controls: true, class: "media-canvas" }), downloads);
|
||||
return;
|
||||
}
|
||||
// no ffmpeg anywhere: decode the MJPEG AVI ourselves, sync to the wav
|
||||
const buf = await mediaFileBytes(j.id, "video");
|
||||
const frames = aviJpegFrames(buf);
|
||||
let audio = null;
|
||||
if ((j.outputs || []).includes("audio")) {
|
||||
const { url } = await mediaFileUrl(j.id, "audio");
|
||||
audio = h("audio", { src: url });
|
||||
}
|
||||
const player = mjpegPlayer(frames, 24, audio);
|
||||
player.canvas.className = "media-canvas";
|
||||
await player.seek0();
|
||||
const playBtn = h("button", { class: "btn primary sm" }, "▶ " + t("pg.play"));
|
||||
playBtn.addEventListener("click", async () => {
|
||||
if (player.state.playing) {
|
||||
player.pause();
|
||||
playBtn.textContent = "▶ " + t("pg.play");
|
||||
} else {
|
||||
playBtn.textContent = "⏸";
|
||||
await player.play();
|
||||
}
|
||||
});
|
||||
mount(host, player.canvas, h("div", { class: "actions" }, playBtn, ...downloads.childNodes ? [downloads] : []));
|
||||
}
|
||||
|
||||
const vGallery = buildGallery("video", (j) => showVideoResult(j, vOut));
|
||||
|
||||
videoPanel = h(
|
||||
"div",
|
||||
{ class: "pg" },
|
||||
h(
|
||||
"div",
|
||||
{},
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
field(t("pg.model"), vModel),
|
||||
field(t("pg.prompt"), vPrompt),
|
||||
h("div", { class: "row" }, field(t("pg.width"), vW, t("pg.videoSizeHint")), field(t("pg.height"), vH)),
|
||||
h("div", { class: "row" }, field(t("pg.frames"), vFrames, t("pg.framesHint")), field(t("pg.steps"), vSteps, t("pg.videoStepsHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, vSeed, dice(vSeed))),
|
||||
field(t("pg.quality"), vQuality)),
|
||||
h("div", { class: "row" }, field(t("pg.firstFrame"), vFirst, t("pg.keyframeHint")), field(t("pg.lastFrame"), vLast)),
|
||||
h("div", { class: "actions" }, vCancel, vGo),
|
||||
vBar
|
||||
),
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), vOut, vGallery.host)
|
||||
),
|
||||
vMetrics
|
||||
);
|
||||
}
|
||||
|
||||
const panels = { chat: chatPanel, image: imagePanel, video: videoPanel };
|
||||
|
||||
mount(
|
||||
root,
|
||||
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
|
||||
tabsSeg,
|
||||
panelHost
|
||||
);
|
||||
mount(panelHost, panels.chat);
|
||||
return root;
|
||||
}
|
||||
|
||||
Vendored
+262
-32
@@ -1,74 +1,293 @@
|
||||
// Routing — tier editor (ordered model chips), default model, and policy.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
|
||||
// Known labels of the hosted taxonomy — used for localized names/descriptions
|
||||
// and as the offline fallback; the live list comes from /routing/labels.
|
||||
const TASK_META = {
|
||||
chitchat: { icon: "💬" },
|
||||
code: { icon: "⌨" },
|
||||
"creative-writing": { icon: "✍" },
|
||||
extraction: { icon: "📑" },
|
||||
math: { icon: "∑" },
|
||||
qa: { icon: "❓" },
|
||||
summarization: { icon: "📋" },
|
||||
translation: { icon: "🌍" },
|
||||
};
|
||||
|
||||
export async function renderRouting() {
|
||||
const meta = appState.meta || (await api.meta());
|
||||
const [routing, modelsData] = await Promise.all([api.getRouting(), api.listModels()]);
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
|
||||
// Task labels: live from the router when reachable, built-in set otherwise;
|
||||
// labels that already have a rule are always shown (custom taxonomies).
|
||||
let taskLabels = Object.keys(TASK_META);
|
||||
let labelsSource = "builtin";
|
||||
try {
|
||||
const r = await api.routingLabels();
|
||||
if (r.labels && r.labels.length) {
|
||||
taskLabels = r.labels;
|
||||
labelsSource = "router";
|
||||
}
|
||||
} catch {
|
||||
/* offline — builtin list */
|
||||
}
|
||||
for (const k of Object.keys(routing.tasks || {})) {
|
||||
if (!taskLabels.includes(k)) taskLabels.push(k);
|
||||
}
|
||||
|
||||
// Share of recent traffic per task — shows which rules actually matter.
|
||||
let taskShare = {};
|
||||
try {
|
||||
const stat = await api.stats("7d", "label");
|
||||
const total = (stat.breakdown || []).reduce((a, b) => a + (b.requests || 0), 0);
|
||||
if (total > 0) {
|
||||
for (const b of stat.breakdown || []) taskShare[b.key] = (b.requests || 0) / total;
|
||||
}
|
||||
} catch {
|
||||
/* no stats yet */
|
||||
}
|
||||
|
||||
// local editable state — ensure standard tiers exist for a friendly editor
|
||||
const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {});
|
||||
const tasks = {};
|
||||
// A task rule is edited as ordered PAIRS (model, tier): "this model serves
|
||||
// this task at this complexity" (tier "any" = at every complexity).
|
||||
// GET returns {any:[], low:[], medium:[], high:[]}; legacy arrays = any.
|
||||
const taskPairs = {};
|
||||
for (const l of taskLabels) {
|
||||
const v = (routing.tasks || {})[l];
|
||||
const pairs = [];
|
||||
if (Array.isArray(v)) for (const m of v) pairs.push({ model: m, tier: "any" });
|
||||
else if (v && typeof v === "object") {
|
||||
for (const tier of ["low", "medium", "high", "any"])
|
||||
for (const m of v[tier] || []) pairs.push({ model: m, tier });
|
||||
}
|
||||
taskPairs[l] = pairs;
|
||||
}
|
||||
const st = {
|
||||
tiers,
|
||||
taskPairs,
|
||||
bands: routing.bands || null, // null = пороги роутера как есть
|
||||
default: routing.default || ids[0] || "",
|
||||
policy: routing.policy || { mode: "fixed_table", min_class: {} },
|
||||
};
|
||||
|
||||
const root = h("div");
|
||||
|
||||
function moveChip(tier, i, dir) {
|
||||
const arr = st.tiers[tier];
|
||||
function moveChip(map, key, i, dir) {
|
||||
const arr = map[key];
|
||||
const ni = i + dir;
|
||||
if (ni < 0 || ni >= arr.length) return;
|
||||
[arr[i], arr[ni]] = [arr[ni], arr[i]];
|
||||
render();
|
||||
}
|
||||
function removeChip(tier, i) {
|
||||
st.tiers[tier].splice(i, 1);
|
||||
function removeChip(map, key, i) {
|
||||
map[key].splice(i, 1);
|
||||
render();
|
||||
}
|
||||
function addChip(tier, id) {
|
||||
if (id && !st.tiers[tier].includes(id)) st.tiers[tier].push(id);
|
||||
function addChip(map, key, id) {
|
||||
if (id && !map[key].includes(id)) map[key].push(id);
|
||||
render();
|
||||
}
|
||||
|
||||
function tierRow(tier) {
|
||||
const arr = st.tiers[tier];
|
||||
function chipEditor(map, key, emptyLabel) {
|
||||
const arr = map[key];
|
||||
const avail = ids.filter((id) => !arr.includes(id));
|
||||
const addSel = h(
|
||||
"select",
|
||||
{
|
||||
style: "width:auto",
|
||||
onChange: (e) => {
|
||||
addChip(tier, e.target.value);
|
||||
},
|
||||
},
|
||||
{ style: "width:auto", onChange: (e) => addChip(map, key, e.target.value) },
|
||||
h("option", { value: "" }, t("routing.addModel")),
|
||||
...avail.map((id) => h("option", { value: id }, id))
|
||||
);
|
||||
return h(
|
||||
"div",
|
||||
{ class: "chips" },
|
||||
arr.length === 0 && emptyLabel ? h("span", { class: "muted small chips-empty" }, emptyLabel) : null,
|
||||
...arr.map((id, i) =>
|
||||
h(
|
||||
"div",
|
||||
{ class: "chip" },
|
||||
h("span", { class: "ord" }, i + 1),
|
||||
id,
|
||||
h("span", { class: "arrow", title: "←", onClick: () => moveChip(map, key, i, -1) }, "◄"),
|
||||
h("span", { class: "arrow", title: "→", onClick: () => moveChip(map, key, i, 1) }, "►"),
|
||||
h("span", { class: "x", onClick: () => removeChip(map, key, i) }, "×")
|
||||
)
|
||||
),
|
||||
avail.length ? addSel : null
|
||||
);
|
||||
}
|
||||
|
||||
// Linked boundary sliders: one track, two handles (low|medium boundaries),
|
||||
// three colored zones. Handles push each other, numbers update live —
|
||||
// nobody types score ranges by hand.
|
||||
function bandsCard() {
|
||||
const DEF = { low: 0.33, medium: 0.48 };
|
||||
const enabled = !!st.bands;
|
||||
const b = st.bands || { ...DEF };
|
||||
const MIN_GAP = 0.02;
|
||||
|
||||
const track = h("div", { class: "bands-track" });
|
||||
const zoneLow = h("div", { class: "bands-zone z-low" });
|
||||
const zoneMed = h("div", { class: "bands-zone z-med" });
|
||||
const zoneHigh = h("div", { class: "bands-zone z-high" });
|
||||
const hLow = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
|
||||
const hMed = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
|
||||
const legend = h("div", { class: "bands-legend" });
|
||||
track.append(zoneLow, zoneMed, zoneHigh, hLow, hMed);
|
||||
|
||||
function paint() {
|
||||
const lo = b.low, me = b.medium;
|
||||
zoneLow.style.width = lo * 100 + "%";
|
||||
zoneMed.style.left = lo * 100 + "%";
|
||||
zoneMed.style.width = (me - lo) * 100 + "%";
|
||||
zoneHigh.style.left = me * 100 + "%";
|
||||
zoneHigh.style.width = (1 - me) * 100 + "%";
|
||||
hLow.style.left = lo * 100 + "%";
|
||||
hMed.style.left = me * 100 + "%";
|
||||
hLow.querySelector(".bands-val").textContent = lo.toFixed(2);
|
||||
hMed.querySelector(".bands-val").textContent = me.toFixed(2);
|
||||
// подписи не наезжают, когда ручки рядом
|
||||
hMed.querySelector(".bands-val").classList.toggle("raised", me - lo < 0.09);
|
||||
mount(legend,
|
||||
h("span", { class: "badge tier-low" }, `${trTier("low")} ≤ ${lo.toFixed(2)}`),
|
||||
h("span", { class: "badge tier-medium" }, `${trTier("medium")} ≤ ${me.toFixed(2)}`),
|
||||
h("span", { class: "badge tier-high" }, `${trTier("high")} > ${me.toFixed(2)}`));
|
||||
track.classList.toggle("disabled", !st.bands);
|
||||
}
|
||||
|
||||
function drag(handle, which) {
|
||||
handle.addEventListener("pointerdown", (e) => {
|
||||
if (!st.bands) return;
|
||||
handle.setPointerCapture(e.pointerId);
|
||||
const rect = track.getBoundingClientRect();
|
||||
const move = (ev) => {
|
||||
let v = (ev.clientX - rect.left) / rect.width;
|
||||
v = Math.min(0.98, Math.max(0.02, Math.round(v * 100) / 100));
|
||||
if (which === "low") {
|
||||
b.low = v;
|
||||
// связанность: low толкает medium перед собой
|
||||
if (b.medium < b.low + MIN_GAP) b.medium = Math.min(0.99, b.low + MIN_GAP);
|
||||
} else {
|
||||
b.medium = v;
|
||||
// и medium толкает low назад
|
||||
if (b.low > b.medium - MIN_GAP) b.low = Math.max(0.01, b.medium - MIN_GAP);
|
||||
}
|
||||
st.bands = { low: b.low, medium: b.medium };
|
||||
paint();
|
||||
};
|
||||
const up = () => {
|
||||
handle.removeEventListener("pointermove", move);
|
||||
handle.removeEventListener("pointerup", up);
|
||||
};
|
||||
handle.addEventListener("pointermove", move);
|
||||
handle.addEventListener("pointerup", up);
|
||||
});
|
||||
}
|
||||
drag(hLow, "low");
|
||||
drag(hMed, "medium");
|
||||
|
||||
const toggle = h("input", { type: "checkbox", checked: enabled ? "" : null });
|
||||
toggle.checked = enabled;
|
||||
toggle.addEventListener("change", () => {
|
||||
st.bands = toggle.checked ? { low: b.low, medium: b.medium } : null;
|
||||
paint();
|
||||
});
|
||||
paint();
|
||||
return h(
|
||||
"div",
|
||||
{ class: "bands-wrap" },
|
||||
h("label", { class: "check", style: "margin-bottom:8px" }, toggle, t("routing.bands.enable")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.bands.hint")),
|
||||
track,
|
||||
legend
|
||||
);
|
||||
}
|
||||
|
||||
function tierRow(tier) {
|
||||
return h(
|
||||
"div",
|
||||
{ class: "tierrow" },
|
||||
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
|
||||
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, trTier(tier))),
|
||||
chipEditor(st.tiers, tier)
|
||||
);
|
||||
}
|
||||
|
||||
function taskRow(label) {
|
||||
const known = TASK_META[label];
|
||||
const name = known ? t("task." + label) : label;
|
||||
const share = taskShare[label];
|
||||
return h(
|
||||
"div",
|
||||
{ class: "tierrow taskrow" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "chips" },
|
||||
...arr.map((id, i) =>
|
||||
h(
|
||||
"div",
|
||||
{ class: "chip" },
|
||||
h("span", { class: "ord" }, i + 1),
|
||||
id,
|
||||
h("span", { class: "arrow", title: "←", onClick: () => moveChip(tier, i, -1) }, "◄"),
|
||||
h("span", { class: "arrow", title: "→", onClick: () => moveChip(tier, i, 1) }, "►"),
|
||||
h("span", { class: "x", onClick: () => removeChip(tier, i) }, "×")
|
||||
)
|
||||
),
|
||||
avail.length ? addSel : null
|
||||
)
|
||||
{ class: "tname taskname" },
|
||||
h("div", {}, (known ? known.icon + " " : "") + name),
|
||||
h("div", { class: "muted small mono" }, label),
|
||||
share != null ? h("div", { class: "muted small" }, t("routing.tasks.share", { p: Math.round(share * 100) })) : null
|
||||
),
|
||||
pairsEditor(label)
|
||||
);
|
||||
}
|
||||
|
||||
// Pairs editor: each row is [model ▾][complexity ▾][×]; "any" serves every
|
||||
// tier. Order matters — it is the fallback order inside the tier.
|
||||
function pairsEditor(label) {
|
||||
const pairs = st.taskPairs[label];
|
||||
const wrap = h("div", { class: "pairs grow" });
|
||||
function paint() {
|
||||
mount(
|
||||
wrap,
|
||||
...pairs.map((p, i) => {
|
||||
const modelSel = h(
|
||||
"select",
|
||||
{ class: "pair-model" },
|
||||
...ids.map((id) => h("option", { value: id }, id))
|
||||
);
|
||||
modelSel.value = p.model;
|
||||
modelSel.addEventListener("change", () => { p.model = modelSel.value; });
|
||||
const tierSel = h(
|
||||
"select",
|
||||
{ class: "pair-tier", title: t("routing.tasks.tierHint") },
|
||||
h("option", { value: "any" }, t("routing.tasks.tierAny")),
|
||||
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, trTier(tr)))
|
||||
);
|
||||
tierSel.value = p.tier;
|
||||
tierSel.addEventListener("change", () => { p.tier = tierSel.value; });
|
||||
const del = h("button", { class: "btn sm ghost danger", onClick: () => { pairs.splice(i, 1); paint(); } }, "×");
|
||||
return h("div", { class: "pair-row" }, modelSel, tierSel, del);
|
||||
}),
|
||||
pairs.length === 0 ? h("div", { class: "muted small", style: "align-self:center" }, t("routing.tasks.inherit")) : null,
|
||||
h("button", { class: "btn ghost sm", onClick: () => { pairs.push({ model: ids[0] || "", tier: "any" }); paint(); } }, "+ " + t("routing.tasks.addModel"))
|
||||
);
|
||||
}
|
||||
paint();
|
||||
return wrap;
|
||||
}
|
||||
|
||||
function tasksCard() {
|
||||
const addIn = h("input", { placeholder: t("routing.tasks.addPlaceholder"), style: "max-width:220px" });
|
||||
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("routing.tasks.addCustom"));
|
||||
addBtn.addEventListener("click", () => {
|
||||
const v = addIn.value.trim().toLowerCase();
|
||||
if (!v || st.taskPairs[v]) return;
|
||||
st.taskPairs[v] = [];
|
||||
render();
|
||||
});
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" },
|
||||
h("h3", {}, "🎯 " + t("routing.tasks.title")),
|
||||
h("span", { class: "card-head sub" },
|
||||
labelsSource === "router" ? t("routing.tasks.fromRouter") : t("routing.tasks.builtin"))),
|
||||
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.tasks.hint")),
|
||||
...Object.keys(st.taskPairs).map(taskRow),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-top:10px" }, addIn, addBtn)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -116,7 +335,16 @@ export async function renderRouting() {
|
||||
saveBtn.addEventListener("click", async () => {
|
||||
saveBtn.disabled = true;
|
||||
try {
|
||||
await api.putRouting({ tiers: st.tiers, default: st.default, policy: st.policy });
|
||||
const tasks = {};
|
||||
for (const [k, pairs] of Object.entries(st.taskPairs)) {
|
||||
const rule = {};
|
||||
for (const p of pairs) {
|
||||
if (!p.model) continue;
|
||||
(rule[p.tier] = rule[p.tier] || []).push(p.model);
|
||||
}
|
||||
if (Object.keys(rule).length) tasks[k] = rule;
|
||||
}
|
||||
await api.putRouting({ tiers: st.tiers, tasks, bands: st.bands, default: st.default, policy: st.policy });
|
||||
toast(t("toast.saved"), "good");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
@@ -141,8 +369,10 @@ export async function renderRouting() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
|
||||
bandsCard(),
|
||||
...Object.keys(st.tiers).map(tierRow)
|
||||
),
|
||||
tasksCard(),
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
|
||||
Vendored
+523
-27
@@ -1,20 +1,66 @@
|
||||
// Settings — router/route/breaker/log/telemetry/stats + export/import config.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { h, mount, toast, confirmDialog } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
import { api, getToken, postRaw } from "../api.js";
|
||||
import { appState, SITE_URL } from "../app.js";
|
||||
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
// A small "?" badge with a hover/focus tooltip explaining a setting.
|
||||
function help(text) {
|
||||
return text ? h("span", { class: "help", title: text, tabindex: "0", role: "img", "aria-label": text }, "?") : null;
|
||||
}
|
||||
function check(label, checked) {
|
||||
function field(label, control, hint, helpText) {
|
||||
return h(
|
||||
"label",
|
||||
{ class: "field" },
|
||||
h("span", {}, label, helpText ? " " : null, help(helpText)),
|
||||
control,
|
||||
hint ? h("div", { class: "hint" }, hint) : null
|
||||
);
|
||||
}
|
||||
function check(label, checked, helpText) {
|
||||
const cb = h("input", { type: "checkbox", checked: checked ? true : null });
|
||||
return { node: h("label", { class: "check" }, cb, label), cb };
|
||||
return { node: h("label", { class: "check" }, cb, label, helpText ? " " : null, help(helpText)), cb };
|
||||
}
|
||||
|
||||
// Flatten router usage (nested account/usage objects) into readable, wrapping
|
||||
// key: value rows — a raw JSON.stringify dump overflows the card horizontally.
|
||||
function usageRows(obj, prefix) {
|
||||
const rows = [];
|
||||
for (const [k, v] of Object.entries(obj || {})) {
|
||||
const key = prefix ? prefix + "." + k : k;
|
||||
if (v && typeof v === "object" && !Array.isArray(v)) {
|
||||
rows.push(...usageRows(v, key));
|
||||
} else {
|
||||
rows.push(
|
||||
h("div", { class: "mono kv" },
|
||||
h("span", { class: "kv-k" }, key + ": "),
|
||||
h("span", { class: "kv-v" }, Array.isArray(v) ? JSON.stringify(v) : String(v)))
|
||||
);
|
||||
}
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
// Probe status → i18n key suffix (see /admin/api/router/probe).
|
||||
function probeMsgKey(status) {
|
||||
const map = {
|
||||
ok: "ok",
|
||||
no_key: "noKey",
|
||||
auth: "auth",
|
||||
payment: "payment",
|
||||
quota: "quota",
|
||||
timeout: "timeout",
|
||||
unreachable: "unreachable",
|
||||
};
|
||||
return "router.probe." + (map[status] || "error");
|
||||
}
|
||||
|
||||
// Statuses where the fix is on the site (get / pay for a key).
|
||||
const PROBE_SITE_STATUSES = ["no_key", "auth", "payment", "quota"];
|
||||
|
||||
export async function renderSettings() {
|
||||
const meta = appState.meta || (await api.meta());
|
||||
const s = await api.getSettings();
|
||||
const [s, secretsData] = await Promise.all([api.getSettings(), api.listSecrets()]);
|
||||
const root = h("div");
|
||||
|
||||
const listenIn = h("input", { value: s.listen || "" });
|
||||
@@ -23,7 +69,109 @@ export async function renderSettings() {
|
||||
const rKeyEnv = h("input", { value: s.router?.api_key_env || "", placeholder: "CORTIQ_ROUTER_KEY" });
|
||||
const rTimeout = h("input", { type: "number", value: s.router?.timeout_ms ?? 800 });
|
||||
const rTax = h("input", { value: s.router?.taxonomy_id || "" });
|
||||
const rVerify = check(t("settings.router.verifyTls"), s.router?.verify_tls);
|
||||
const rVerify = check(t("settings.router.verifyTls"), s.router?.verify_tls, t("settings.router.verifyTlsHelp"));
|
||||
const rEnabled = check(t("settings.router.enabled"), s.router?.enabled !== false, t("settings.router.enabledHelp"));
|
||||
// router key value (write-only; stored in the secret store, like model keys)
|
||||
const rKeySource =
|
||||
(secretsData.secrets || []).find((x) => x.name === (s.router?.api_key_env || "CORTIQ_ROUTER_KEY"))?.source ||
|
||||
"missing";
|
||||
const keyStored = rKeySource === "store" || rKeySource === "env";
|
||||
// When a key is already saved, show a masked "saved" hint instead of an empty
|
||||
// field (an empty field reads as "nothing is set").
|
||||
const rSecret = h("input", {
|
||||
type: "password",
|
||||
placeholder: keyStored ? t("settings.router.keyStoredMask") : t("models.form.secretPlaceholder"),
|
||||
});
|
||||
const rKeyBadge = h(
|
||||
"span",
|
||||
{ class: "badge " + (rKeySource === "store" ? "store" : rKeySource === "env" ? "env" : "bad") },
|
||||
rKeySource === "store"
|
||||
? t("models.form.secretStored")
|
||||
: rKeySource === "env"
|
||||
? t("models.form.secretEnv")
|
||||
: t("models.form.secretMissing")
|
||||
);
|
||||
const rSecretHint = h(
|
||||
"div",
|
||||
{},
|
||||
h(
|
||||
"div",
|
||||
{},
|
||||
rKeyBadge,
|
||||
" · ",
|
||||
t("settings.router.getKey") + " ",
|
||||
h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, "api.allaigate.com")
|
||||
),
|
||||
h("div", { class: "muted", style: "margin-top:4px" }, t("settings.router.secretHelp"))
|
||||
);
|
||||
// The key lives in the secret store under this env name (config default: CORTIQ_ROUTER_KEY).
|
||||
const keyName = () => rKeyEnv.value.trim() || "CORTIQ_ROUTER_KEY";
|
||||
// Persist just the pasted router key (independent of the big Save at page top).
|
||||
async function saveKey() {
|
||||
const v = rSecret.value.trim();
|
||||
if (!v) return false;
|
||||
if (!rKeyEnv.value.trim()) rKeyEnv.value = "CORTIQ_ROUTER_KEY";
|
||||
await api.setSecret(keyName(), v);
|
||||
rSecret.value = "";
|
||||
rSecret.placeholder = t("settings.router.keyStoredMask"); // show "saved", not empty
|
||||
rKeyBadge.textContent = t("models.form.secretStored");
|
||||
rKeyBadge.className = "badge store";
|
||||
return true;
|
||||
}
|
||||
// Shared connection test (real /v1/route call) — tells key problems apart from
|
||||
// a down router. Rendered into probeOut; also used right after saving the key.
|
||||
const probeOut = h("div", { class: "hint", style: "margin-top:8px" });
|
||||
async function runProbe() {
|
||||
mount(probeOut, h("span", { class: "spinner" }));
|
||||
try {
|
||||
const r = await api.probeRouter();
|
||||
const parts = [t(probeMsgKey(r.status), { ms: r.latency_ms ?? "—" })];
|
||||
if (r.message) parts.push(" — " + r.message);
|
||||
const out = [h("span", {}, ...parts)];
|
||||
if (PROBE_SITE_STATUSES.includes(r.status)) {
|
||||
out.push(" · ", h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("dash.health.getKey") + " ↗"));
|
||||
}
|
||||
if (r.usage && typeof r.usage === "object") {
|
||||
out.push(
|
||||
h("div", { class: "usage-box", style: "margin-top:4px" },
|
||||
h("b", {}, t("router.usage")), ...usageRows(r.usage))
|
||||
);
|
||||
}
|
||||
mount(probeOut, h("div", {}, h("span", { class: "badge " + (r.ok ? "ok" : "bad") }, r.ok ? "OK" : "×"), " ", ...out));
|
||||
toast(t(probeMsgKey(r.status), { ms: r.latency_ms ?? "—" }), r.ok ? "good" : "bad");
|
||||
return r.ok;
|
||||
} catch (e) {
|
||||
mount(probeOut, String(e.message || e));
|
||||
return false;
|
||||
}
|
||||
}
|
||||
const saveKeyBtn = h("button", { class: "btn sm" }, "✓ " + t("settings.router.saveKey"));
|
||||
saveKeyBtn.addEventListener("click", async () => {
|
||||
saveKeyBtn.disabled = true;
|
||||
try {
|
||||
if (await saveKey()) {
|
||||
toast(t("settings.router.keySaved"), "good");
|
||||
// Immediately test the connection so the user sees reachable/error at once,
|
||||
// instead of a stale "unavailable" until something re-checks.
|
||||
if (rEnabled.cb.checked) await runProbe();
|
||||
} else {
|
||||
toast(t("settings.router.keyEmpty"), "warn");
|
||||
}
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
saveKeyBtn.disabled = false;
|
||||
});
|
||||
const probeBtn = h("button", { class: "btn" }, t("settings.router.test"));
|
||||
probeBtn.addEventListener("click", async () => {
|
||||
probeBtn.disabled = true;
|
||||
// If a key was pasted but not saved, persist it first so the test uses it.
|
||||
if (rSecret.value.trim()) {
|
||||
try { await saveKey(); } catch (_) { /* probe will report no_key/auth */ }
|
||||
}
|
||||
await runProbe();
|
||||
probeBtn.disabled = false;
|
||||
});
|
||||
// route
|
||||
const strategySel = h("select", {}, ...(meta.text_strategies || []).map((x) => h("option", { value: x, selected: x === s.route?.text_strategy }, x)));
|
||||
const maxChars = h("input", { type: "number", value: s.route?.max_chars ?? 4000 });
|
||||
@@ -34,33 +182,264 @@ export async function renderSettings() {
|
||||
const brCool = h("input", { value: s.breaker?.cooldown || "30s" });
|
||||
// log
|
||||
const logSel = h("select", {}, ...["error", "warn", "info", "debug", "trace"].map((x) => h("option", { value: x, selected: x === s.log?.level }, x)));
|
||||
const logBodies = check(t("settings.log.bodies"), s.log?.bodies);
|
||||
const logBodies = check(t("settings.log.bodies"), s.log?.bodies, t("settings.log.bodiesHelp"));
|
||||
// telemetry / cortiq
|
||||
const metricsCb = check(t("settings.telemetry.metrics"), s.telemetry?.metrics);
|
||||
const echoCb = check(t("settings.cortiq.echo"), s.cortiq?.echo);
|
||||
const metricsCb = check(t("settings.telemetry.metrics"), s.telemetry?.metrics, t("settings.telemetry.metricsHelp"));
|
||||
const echoCb = check(t("settings.cortiq.echo"), s.cortiq?.echo, t("settings.cortiq.echoHelp"));
|
||||
// stats
|
||||
const statsEnabled = check(t("settings.stats.enabled"), s.stats?.enabled);
|
||||
const statsEnabled = check(t("settings.stats.enabled"), s.stats?.enabled, t("settings.stats.enabledHelp"));
|
||||
const statsFile = h("input", { value: s.stats?.file || "" });
|
||||
const statsRet = h("input", { value: s.stats?.retention || "7d" });
|
||||
// cache
|
||||
const modelsData = await api.listModels();
|
||||
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
|
||||
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled);
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
|
||||
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
|
||||
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
|
||||
// Live proof-of-life for the cache card: entries/hits/saved — and an
|
||||
// honest warning when the cache is on but has no embedding model to work
|
||||
// with (it silently does nothing in that state).
|
||||
let cacheStats = null;
|
||||
try {
|
||||
cacheStats = (await api.stats("24h", "model")).cache || null;
|
||||
} catch { /* stats off */ }
|
||||
function cacheStatus() {
|
||||
const wrap = h("div", { style: "margin-bottom:10px" });
|
||||
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
|
||||
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
|
||||
return wrap;
|
||||
}
|
||||
if (cacheStats && s.cache?.enabled) {
|
||||
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
|
||||
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
|
||||
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
|
||||
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
|
||||
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
|
||||
}
|
||||
return wrap;
|
||||
}
|
||||
const cacheEmbedSel = h(
|
||||
"select",
|
||||
{},
|
||||
h("option", { value: "" }, "auto"),
|
||||
...embedModels.map((id) => h("option", { value: id, selected: id === s.cache?.embed_model }, id))
|
||||
);
|
||||
// local models (CMF) — run a local .cmf server and/or force local-only routing
|
||||
const cmfLocalOnly = check(t("settings.cmf.localOnly"), s.cmf?.local_only, t("settings.cmf.localOnlyHelp"));
|
||||
const cmfManage = check(t("settings.cmf.manage"), s.cmf?.manage_server, t("settings.cmf.manageHelp"));
|
||||
const cmfAutoInstall = check(t("settings.cmf.autoInstall"), s.cmf?.auto_install, t("settings.cmf.autoInstallHelp"));
|
||||
const cmfAutoUpdate = check(t("settings.cmf.autoUpdate"), s.cmf?.auto_update, t("settings.cmf.autoUpdateHelp"));
|
||||
// ── managed local models: one supervised `cortiq serve` per row ──
|
||||
const serverRows = [];
|
||||
const serversWrap = h("div", {});
|
||||
const legacyServers = s.cmf?.local_model
|
||||
? [{ id: s.cmf.model_id || "cmf-local", model: s.cmf.local_model, port: s.cmf.local_port ?? 8081, threads: s.cmf.threads ?? 8, gpu: !!s.cmf.gpu }]
|
||||
: [];
|
||||
const initServers = (s.cmf?.servers && s.cmf.servers.length) ? s.cmf.servers : legacyServers;
|
||||
// available .cmf files under models_dir → offer as a dropdown, not a typed path
|
||||
const cmfFiles = ((await api.cmfFiles().catch(() => ({ files: [] }))).files) || [];
|
||||
const fileLabel = (f) => f.name + (f.size ? ` (${(f.size / 1073741824).toFixed(f.size >= 1073741824 ? 1 : 2)} GB)` : f.is_dir ? " (dir)" : "");
|
||||
function autoId(path) {
|
||||
const base = (path || "").split("/").pop().replace(/\.cmf$/i, "");
|
||||
return base ? "cmf-" + base.replace(/[^a-zA-Z0-9_-]/g, "-") : "";
|
||||
}
|
||||
function makeServerRow(sv) {
|
||||
sv = sv || { id: "", model: "", port: 8090, threads: 8, gpu: false };
|
||||
const idIn = h("input", { value: sv.id || "", placeholder: "cmf-id", style: "width:130px" });
|
||||
// pick from existing .cmf files rather than typing a path
|
||||
const modelSel = h("select", { style: "flex:2;min-width:150px" });
|
||||
const seen = new Set();
|
||||
modelSel.appendChild(h("option", { value: "" }, "— " + t("settings.cmf.pick") + " —"));
|
||||
cmfFiles.forEach((f) => { modelSel.appendChild(h("option", { value: f.path }, fileLabel(f))); seen.add(f.path); });
|
||||
if (sv.model && !seen.has(sv.model)) modelSel.appendChild(h("option", { value: sv.model }, sv.model));
|
||||
modelSel.value = sv.model || "";
|
||||
const portIn = h("input", { type: "number", value: sv.port ?? 8090, style: "width:82px" });
|
||||
const threadsIn = h("input", { type: "number", min: 0, value: sv.threads ?? 8, style: "width:60px", title: t("settings.cmf.threadsHelp") });
|
||||
const gpuIn = h("input", { type: "checkbox", checked: sv.gpu ? true : null, title: t("settings.cmf.gpuHelp") });
|
||||
const checkBtn = h("button", { class: "btn small", type: "button" }, t("settings.cmf.checkPort"));
|
||||
const removeBtn = h("button", { class: "btn small danger", type: "button", title: t("common.delete") }, "✕");
|
||||
const st = h("span", { class: "small" });
|
||||
modelSel.addEventListener("change", () => { if (!idIn.value || idIn._auto) { idIn.value = autoId(modelSel.value); idIn._auto = true; } });
|
||||
idIn.addEventListener("input", () => { idIn._auto = false; });
|
||||
checkBtn.addEventListener("click", async () => {
|
||||
const p = parseInt(portIn.value, 10); if (!p) return;
|
||||
checkBtn.disabled = true; mount(st, t("status.checking"));
|
||||
try {
|
||||
const r = await api.cmfPortCheck(p);
|
||||
if (r.available) mount(st, h("span", { style: "color:var(--good)" }, "✓ " + t("settings.cmf.portFree")));
|
||||
else {
|
||||
const kids = [h("span", { style: "color:var(--bad)" }, "✗ " + t("settings.cmf.portBusy"))];
|
||||
if (r.suggested) { const b = h("button", { class: "btn small", type: "button" }, t("settings.cmf.useSuggested", { port: r.suggested })); b.addEventListener("click", () => { portIn.value = r.suggested; mount(st, ""); }); kids.push(b); }
|
||||
mount(st, h("span", {}, ...kids));
|
||||
}
|
||||
} catch (e) { mount(st, String(e.message || e)); }
|
||||
checkBtn.disabled = false;
|
||||
});
|
||||
|
||||
// ── O(1) Nyström Attention ──
|
||||
// NOT a "reasoning mode". This is a memory-efficient streaming attention
|
||||
// kernel (Nyström/landmark) that replaces standard KV-cache attention,
|
||||
// giving O(1) memory per decode step instead of O(n).
|
||||
const o1Sel = h("select", { style: "width:120px", title: "O(1) Nyström attention layer spec: which layers replace KV-cache with streaming Nyström attention" },
|
||||
h("option", { value: "" }, "O(1) авто"),
|
||||
h("option", { value: "all" }, "O(1) все слои"),
|
||||
h("option", { value: "deep" }, "O(1) глубокие"),
|
||||
h("option", { value: "off" }, "O(1) выкл."));
|
||||
if (sv.o1) o1Sel.value = sv.o1;
|
||||
const o1mIn = h("input", { type: "number", min: "4", placeholder: "32", value: sv.o1_m ?? "", style: "width:55px", title: "Landmark budget (≥4, default 32). More landmarks ≠ better: m=64 measured worse." });
|
||||
const o1wIn = h("input", { type: "number", min: "1", placeholder: "128", value: sv.o1_window ?? "", style: "width:60px", title: "Exact-window width — main quality lever (default 128)" });
|
||||
const o1sinkIn = h("input", { type: "number", min: "0", placeholder: "4", value: sv.o1_sink ?? "", style: "width:50px", title: "Permanent exact sink keys, StreamingLLM discipline (default 4)" });
|
||||
|
||||
// ── Generation parameters ──
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: sv.temperature ?? "", style: "width:60px" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: sv.top_p ?? "", style: "width:60px" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: sv.max_tokens ?? "", style: "width:75px" });
|
||||
// think_budget = 0 means reasoning disabled (enable_thinking=false);
|
||||
// null = model decides; N>0 = token budget for the <think> block.
|
||||
const noThinkCb = h("input", { type: "checkbox", checked: (sv.think_budget === 0) ? true : null, title: "Отключить режим размышлений (think_budget=0, enable_thinking=false для Qwen3/3.5)" });
|
||||
const thinkIn = h("input", { type: "number", min: "1", placeholder: "авто", value: (sv.think_budget && sv.think_budget > 0) ? sv.think_budget : "", style: "width:75px", title: "Бюджет токенов на размышления (think_budget). Пустое = модель решает сама." });
|
||||
if (sv.think_budget === 0) thinkIn.disabled = true;
|
||||
noThinkCb.addEventListener("change", () => {
|
||||
thinkIn.disabled = noThinkCb.checked;
|
||||
if (noThinkCb.checked) thinkIn.value = "";
|
||||
});
|
||||
const sysIn = h("input", { placeholder: "System prompt (optional)", value: sv.system_prompt || "", style: "flex:1;min-width:200px" });
|
||||
|
||||
// ── MTP (Multi-Token Prediction): speculative decoding, NOT reasoning ──
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: sv.skip_mtp ? true : null, title: "Disable MTP speculative decoding (env CMF_MTP=0)" });
|
||||
|
||||
// ── Settings drawer ──
|
||||
const drawerStyle = "display:none;width:100%;margin-top:6px;padding:10px 12px;background:var(--bg-subtle);border-radius:6px;border:1px solid var(--border)";
|
||||
const groupStyle = "display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:4px 0";
|
||||
const labelStyle = "font-size:11px;color:var(--text-muted);font-weight:600;text-transform:uppercase;letter-spacing:0.5px;width:100%;margin-bottom:2px";
|
||||
|
||||
const drawer = h("div", { style: drawerStyle },
|
||||
// Group 1: O(1) Nyström Attention
|
||||
h("div", { style: labelStyle }, "O(1) Nyström Attention"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Слои:"), o1Sel,
|
||||
h("span", { class: "small muted" }, "m:"), o1mIn,
|
||||
h("span", { class: "small muted" }, "Window:"), o1wIn,
|
||||
h("span", { class: "small muted" }, "Sink:"), o1sinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 2: Generation
|
||||
h("div", { style: labelStyle }, "Генерация"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Temp:"), tempIn,
|
||||
h("span", { class: "small muted" }, "Top-P:"), topPIn,
|
||||
h("span", { class: "small muted" }, "Max токенов:"), maxTokIn
|
||||
),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "System prompt:"), sysIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 3: Reasoning (thinking)
|
||||
h("div", { style: labelStyle }, "Режим размышлений"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "Для моделей Qwen3/3.5: enable_thinking=false — модель отвечает напрямую без блока <think>" }, noThinkCb, "Отключить размышления"),
|
||||
h("span", { class: "small muted" }, "Think budget:"), thinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 4: Runtime
|
||||
h("div", { style: labelStyle }, "Runtime"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "MTP — Multi-Token Prediction (спекулятивное декодирование). Отключение замедляет генерацию, но стабилизирует output." }, skipMtpCb, "Отключить MTP")
|
||||
)
|
||||
);
|
||||
|
||||
const gearBtn = h("button", { class: "btn small", type: "button", title: "⚙ Настройки модели" }, "⚙");
|
||||
gearBtn.addEventListener("click", () => {
|
||||
drawer.style.display = drawer.style.display === "none" ? "block" : "none";
|
||||
});
|
||||
|
||||
const row = h("div", { class: "flex wrap cmf-server-row", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), gearBtn, removeBtn, st, drawer);
|
||||
const entry = { node: row, read: () => ({
|
||||
id: idIn.value.trim() || autoId(modelSel.value),
|
||||
model: modelSel.value.trim(),
|
||||
port: parseInt(portIn.value) || 8090,
|
||||
threads: parseInt(threadsIn.value) || 0,
|
||||
gpu: gpuIn.checked,
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: noThinkCb.checked ? 0 : (thinkIn.value !== "" ? parseInt(thinkIn.value) : null),
|
||||
o1: o1Sel.value || null,
|
||||
o1_m: o1mIn.value !== "" ? parseInt(o1mIn.value) : null,
|
||||
o1_window: o1wIn.value !== "" ? parseInt(o1wIn.value) : null,
|
||||
o1_sink: o1sinkIn.value !== "" ? parseInt(o1sinkIn.value) : null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysIn.value.trim() || null,
|
||||
}) };
|
||||
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
|
||||
serverRows.push(entry);
|
||||
serversWrap.appendChild(row);
|
||||
return entry;
|
||||
}
|
||||
initServers.forEach((sv) => makeServerRow(sv));
|
||||
const addServerBtn = h("button", { class: "btn small", type: "button" }, t("settings.cmf.addServer"));
|
||||
addServerBtn.addEventListener("click", () => makeServerRow());
|
||||
// "Install cortiq" — the CLI that RUNS the .cmf format. Must be present before
|
||||
// a local model server can start; this is step 1 for a new user.
|
||||
const cmfStatusEl = h("span", { class: "muted" }, "…");
|
||||
const cmfInstallBtn = h("button", { class: "btn" }, t("settings.cmf.install"));
|
||||
// live "serving <model> on :<port>" indicator for the managed local server
|
||||
const cmfServingEl = h("div", { class: "hint" });
|
||||
async function refreshCmfStatus() {
|
||||
try {
|
||||
const st = await api.cmfStatus();
|
||||
cmfInstallBtn.textContent = st.installed ? t("settings.cmf.reinstall") : t("settings.cmf.install");
|
||||
mount(
|
||||
cmfStatusEl,
|
||||
st.installed
|
||||
? h("span", {}, h("span", { class: "badge ok" }, "✓"), " cortiq " + (st.version || ""))
|
||||
: h("span", {}, h("span", { class: "badge bad" }, "×"), " " + t("settings.cmf.notInstalled"))
|
||||
);
|
||||
// per-server serving indicators
|
||||
const servers = st.servers || [];
|
||||
if (!servers.length) {
|
||||
mount(cmfServingEl, "");
|
||||
} else {
|
||||
mount(cmfServingEl, h("div", {}, ...servers.map((sv) => {
|
||||
if (sv.last_error) return h("div", { style: "color:var(--bad)" }, "✗ " + sv.id + ": " + sv.last_error);
|
||||
if (sv.running) return h("div", { style: "color:var(--good)" },
|
||||
"▶ " + t("settings.cmf.serving", { model: sv.id, port: sv.port }) + (sv.healthy ? "" : " …"));
|
||||
return h("div", { class: "muted" }, "• " + sv.id + " :" + sv.port);
|
||||
})));
|
||||
}
|
||||
} catch (_) {
|
||||
mount(cmfStatusEl, "—");
|
||||
}
|
||||
}
|
||||
cmfInstallBtn.addEventListener("click", async () => {
|
||||
cmfInstallBtn.disabled = true;
|
||||
mount(cmfStatusEl, h("span", {}, h("span", { class: "spinner" }), " " + t("settings.cmf.installing")));
|
||||
try {
|
||||
await api.cmfInstall();
|
||||
// install compiles from crates.io — poll status for a few minutes
|
||||
for (let i = 0; i < 120; i++) {
|
||||
await new Promise((r) => setTimeout(r, 3000));
|
||||
const st = await api.cmfStatus();
|
||||
if (st.installed) { toast(t("settings.cmf.installedOk"), "good"); break; }
|
||||
}
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
await refreshCmfStatus();
|
||||
cmfInstallBtn.disabled = false;
|
||||
});
|
||||
refreshCmfStatus();
|
||||
|
||||
const saveBtn = h("button", { class: "btn primary" }, t("common.save"));
|
||||
saveBtn.addEventListener("click", async () => {
|
||||
saveBtn.disabled = true;
|
||||
// a pasted router key needs an env name to live under — default it
|
||||
if (rSecret.value.trim() && !rKeyEnv.value.trim()) rKeyEnv.value = "CORTIQ_ROUTER_KEY";
|
||||
const patch = {
|
||||
listen: listenIn.value.trim(),
|
||||
router: {
|
||||
enabled: rEnabled.cb.checked,
|
||||
url: rUrl.value.trim(),
|
||||
api_key_env: rKeyEnv.value.trim() || null,
|
||||
timeout_ms: parseInt(rTimeout.value) || 800,
|
||||
@@ -90,6 +469,16 @@ export async function renderSettings() {
|
||||
max_entries: s.cache?.max_entries ?? 1000,
|
||||
embed_model: cacheEmbedSel.value || null,
|
||||
},
|
||||
cmf: {
|
||||
...(s.cmf || {}),
|
||||
local_only: cmfLocalOnly.cb.checked,
|
||||
manage_server: cmfManage.cb.checked,
|
||||
auto_install: cmfAutoInstall.cb.checked,
|
||||
auto_update: cmfAutoUpdate.cb.checked,
|
||||
// the managed-models list is the source of truth; clear the legacy single
|
||||
servers: serverRows.map((r) => r.read()).filter((sv) => sv.model && sv.id),
|
||||
local_model: "",
|
||||
},
|
||||
};
|
||||
// serde: drop null optionals
|
||||
if (!patch.router.api_key_env) delete patch.router.api_key_env;
|
||||
@@ -98,6 +487,13 @@ export async function renderSettings() {
|
||||
if (!patch.cache.embed_model) delete patch.cache.embed_model;
|
||||
try {
|
||||
const r = await api.putSettings(patch);
|
||||
const secretVal = rSecret.value.trim();
|
||||
if (secretVal) {
|
||||
await api.setSecret(rKeyEnv.value.trim(), secretVal);
|
||||
rSecret.value = "";
|
||||
rKeyBadge.textContent = t("models.form.secretStored");
|
||||
rKeyBadge.className = "badge store";
|
||||
}
|
||||
toast(r.needs_restart ? t("settings.needsRestart") : t("toast.saved"), r.needs_restart ? "warn" : "good");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
@@ -106,22 +502,33 @@ export async function renderSettings() {
|
||||
});
|
||||
|
||||
async function exportConfig() {
|
||||
const cfg = await api.getConfig();
|
||||
const blob = new Blob([JSON.stringify(cfg, null, 2)], { type: "application/json" });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = h("a", { href: url, download: "gateway-config.json" });
|
||||
// the real gateway.toml from disk — comments and all
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch("/admin/api/config/export", { headers });
|
||||
if (!r.ok) {
|
||||
toast(await r.text(), "bad");
|
||||
return;
|
||||
}
|
||||
const url = URL.createObjectURL(await r.blob());
|
||||
const a = h("a", { href: url, download: "gateway.toml" });
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
URL.revokeObjectURL(url);
|
||||
}
|
||||
const fileIn = h("input", { type: "file", accept: ".json,application/json", style: "display:none" });
|
||||
const fileIn = h("input", { type: "file", accept: ".toml,.json,application/json", style: "display:none" });
|
||||
fileIn.addEventListener("change", async () => {
|
||||
const f = fileIn.files[0];
|
||||
if (!f) return;
|
||||
try {
|
||||
const cfg = JSON.parse(await f.text());
|
||||
await api.putConfig(cfg);
|
||||
const text = await f.text();
|
||||
if (f.name.endsWith(".json")) {
|
||||
await api.putConfig(JSON.parse(text)); // legacy JSON export
|
||||
} else {
|
||||
await postRaw("/config/import", text); // gateway.toml — validated + hot-reloaded
|
||||
}
|
||||
toast(t("toast.imported"), "good");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
@@ -129,6 +536,54 @@ export async function renderSettings() {
|
||||
fileIn.value = "";
|
||||
});
|
||||
|
||||
const restartBtn = h("button", { class: "btn danger" }, "⟳ " + t("settings.restart"));
|
||||
restartBtn.addEventListener("click", async () => {
|
||||
if (!(await confirmDialog(t("settings.restartConfirm")))) return;
|
||||
restartBtn.disabled = true;
|
||||
try {
|
||||
await api.restart();
|
||||
toast(t("settings.restarting"), "good");
|
||||
// the gateway comes back on the same port — reload once it answers
|
||||
const poll = setInterval(async () => {
|
||||
try {
|
||||
await api.health();
|
||||
clearInterval(poll);
|
||||
location.reload();
|
||||
} catch {
|
||||
/* still restarting */
|
||||
}
|
||||
}, 1000);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
restartBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
const stopBtn = h("button", { class: "btn danger" }, "\u23fb " + t("settings.shutdown"));
|
||||
stopBtn.addEventListener("click", async () => {
|
||||
if (!(await confirmDialog(t("settings.shutdownConfirm")))) return;
|
||||
stopBtn.disabled = true;
|
||||
restartBtn.disabled = true;
|
||||
try {
|
||||
await api.shutdown();
|
||||
// the server is gone in ~300ms — replace the app with a static farewell
|
||||
document.body.innerHTML = "";
|
||||
document.body.appendChild(
|
||||
h(
|
||||
"div",
|
||||
{ class: "shutdown-screen" },
|
||||
h("div", { class: "shutdown-icon" }, "\u23fb"),
|
||||
h("h2", {}, t("settings.shutdownDone")),
|
||||
h("p", { class: "hint" }, t("settings.shutdownHint"))
|
||||
)
|
||||
);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
stopBtn.disabled = false;
|
||||
restartBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
mount(
|
||||
root,
|
||||
h(
|
||||
@@ -150,16 +605,23 @@ export async function renderSettings() {
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.router"))),
|
||||
field(t("settings.router.url"), rUrl),
|
||||
field(t("settings.router.keyEnv"), rKeyEnv),
|
||||
field(t("settings.router.secret"), h("div", { class: "key-row" }, rSecret, saveKeyBtn), rSecretHint),
|
||||
h("div", { class: "row" }, field(t("settings.router.timeout"), rTimeout), field(t("settings.router.taxonomy"), rTax)),
|
||||
rVerify.node
|
||||
rEnabled.node,
|
||||
rVerify.node,
|
||||
h("div", { class: "divider" }),
|
||||
h("div", { class: "flex" }, probeBtn),
|
||||
probeOut
|
||||
),
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.route"))),
|
||||
field(t("settings.route.strategy"), strategySel),
|
||||
h("div", { class: "row" }, field(t("settings.route.maxChars"), maxChars), field(t("settings.route.cacheTtl"), cacheTtl)),
|
||||
field(t("settings.route.profile"), profileSel)
|
||||
field(t("settings.route.strategy"), strategySel, null, t("settings.route.strategyHelp")),
|
||||
h("div", { class: "row" },
|
||||
field(t("settings.route.maxChars"), maxChars, null, t("settings.route.maxCharsHelp")),
|
||||
field(t("settings.route.cacheTtl"), cacheTtl, null, t("settings.route.cacheTtlHelp"))),
|
||||
field(t("settings.route.profile"), profileSel, null, t("settings.route.profileHelp"))
|
||||
),
|
||||
h(
|
||||
"div",
|
||||
@@ -195,10 +657,41 @@ export async function renderSettings() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
|
||||
cacheStatus(),
|
||||
cacheEnabled.node,
|
||||
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
|
||||
field(t("settings.cache.embed"), cacheEmbedSel),
|
||||
h("div", { class: "hint" }, t("settings.cache.note"))
|
||||
),
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.cmf"))),
|
||||
h("div", { class: "hint" }, t("settings.cmf.intro")),
|
||||
// Step 1 — install the cortiq runtime (the CLI that runs .cmf models).
|
||||
h(
|
||||
"label",
|
||||
{ class: "field" },
|
||||
h("span", {}, "1. " + t("settings.cmf.runtime"), " ", help(t("settings.cmf.runtimeHelp"))),
|
||||
h("div", { class: "flex", style: "gap:8px;align-items:center;flex-wrap:wrap" }, cmfInstallBtn, cmfStatusEl)
|
||||
),
|
||||
h("div", { class: "row" }, cmfAutoInstall.node, cmfAutoUpdate.node),
|
||||
h("div", { class: "divider" }),
|
||||
// Step 2 — managed local models: one supervised cortiq serve per row, each
|
||||
// on its own port. Add as many as you have .cmf files + free ports.
|
||||
h("div", { class: "field" },
|
||||
h("span", {}, "2. " + t("settings.cmf.models"), " ", help(t("settings.cmf.modelsHelp"))),
|
||||
h("div", { class: "hint" }, t("settings.cmf.rowHint")),
|
||||
h("div", { class: "hint" }, t("settings.cmf.importNote"),
|
||||
" ", h("a", { href: "#/import" }, t("settings.cmf.importLink") + " ↗"))),
|
||||
serversWrap,
|
||||
addServerBtn,
|
||||
cmfServingEl,
|
||||
h("div", { class: "hint" }, t("settings.cmf.speedNote")),
|
||||
h("div", { class: "divider" }),
|
||||
// Step 3 — how the gateway uses them.
|
||||
cmfManage.node,
|
||||
cmfLocalOnly.node
|
||||
)
|
||||
),
|
||||
h(
|
||||
@@ -212,7 +705,10 @@ export async function renderSettings() {
|
||||
{ class: "flex wrap" },
|
||||
h("button", { class: "btn", onClick: exportConfig }, t("settings.export")),
|
||||
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
|
||||
fileIn
|
||||
fileIn,
|
||||
h("div", { class: "grow" }),
|
||||
restartBtn,
|
||||
stopBtn
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user