Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9fd4bdaa85 | ||
|
|
9c15109f26 | ||
|
|
f65f131c41 | ||
|
|
b339c2b622 | ||
|
|
34093330f6 | ||
|
|
faddf945d7 | ||
|
|
255d989e83 | ||
|
|
0d9d24125a | ||
|
|
1c59906b49 | ||
|
|
19f666e7b1 | ||
|
|
dbcd9827f6 | ||
|
|
b689180717 | ||
|
|
f385c2254b | ||
|
|
9607a38792 | ||
|
|
239ddc767c | ||
|
|
057abaa95a | ||
|
|
dd864eed22 | ||
|
|
c1ff675d80 | ||
|
|
9ef1af3ae1 | ||
|
|
6f4d83ce13 | ||
|
|
0040bfd7e9 | ||
|
|
10b5101757 | ||
|
|
788c87c565 | ||
|
|
179c414a0e | ||
|
|
115b04f20e | ||
|
|
b24d4cdbab | ||
|
|
06813b3e97 | ||
|
|
95e560dc72 | ||
|
|
ef28a6633d | ||
|
|
b4d36334bf | ||
|
|
dd9ccaea94 | ||
|
|
ff8415d7d4 | ||
|
|
7479cc51c2 | ||
|
|
b1e0dc76ec | ||
|
|
c3dd251cce | ||
|
|
9af51b679e | ||
|
|
1bba427042 | ||
|
|
54b73f1365 | ||
|
|
ec92bdddb7 | ||
|
|
1120d8f11a | ||
|
|
914ff42fc5 | ||
|
|
2ede0ee452 | ||
|
|
15de36be3b | ||
|
|
a77beeabab | ||
|
|
32ce2f7ed7 | ||
|
|
9ec8861464 | ||
|
|
ad438a5dcb | ||
|
|
45def5c6b6 | ||
|
|
5d8455b4a0 | ||
|
|
b255c60b0b | ||
|
|
a75cacd0ec | ||
|
|
850a54acdf | ||
|
|
22110ac524 | ||
|
|
b3b37d27f4 | ||
|
|
a7c8ed72c7 | ||
|
|
f2e6047bff | ||
|
|
0b28aa3d59 | ||
|
|
1414ad1b52 | ||
|
|
94c8551206 | ||
|
|
6ecc709fd2 | ||
|
|
1661c19550 | ||
|
|
66ee32b233 | ||
|
|
f6ee045989 | ||
|
|
59e2eb3215 | ||
|
|
5b743a4691 | ||
|
|
c59ed9c949 | ||
|
|
b55e72789f | ||
|
|
0d1283c3fa | ||
|
|
0105a43ef5 | ||
|
|
b8d48c31f7 | ||
|
|
196ff52f0f | ||
|
|
e25814a54c | ||
|
|
0c20eb34f0 | ||
|
|
f8c1bb0914 | ||
|
|
0f299b77e6 | ||
|
|
7c49510c46 | ||
|
|
31da84cc45 | ||
|
|
749a47ce9b | ||
|
|
d2b1d50eb8 | ||
|
|
d48349a144 | ||
|
|
f05d26bfbb | ||
|
|
39f1fcb600 | ||
|
|
42377a75d4 | ||
|
|
6cf2608fc8 | ||
|
|
e6d1a630c9 | ||
|
|
8dd77ca7ff | ||
|
|
630149d141 | ||
|
|
85fc508f04 | ||
|
|
010171e05c | ||
|
|
39c56145ed | ||
|
|
17c8186383 | ||
|
|
e773e18efd | ||
|
|
6466006de6 | ||
|
|
6a25d8ac08 | ||
|
|
e486617e51 | ||
|
|
729a8bfbba | ||
|
|
5759de153b | ||
|
|
bf8534f13d | ||
|
|
70d8cde258 | ||
|
|
3d71b03788 | ||
|
|
492953d4c2 | ||
|
|
3b0231b32d | ||
|
|
fd2f2a165b | ||
|
|
cbc385f6c7 |
@@ -1,4 +1,6 @@
|
||||
target
|
||||
models
|
||||
*.cmf
|
||||
.git
|
||||
.github
|
||||
docs
|
||||
|
||||
@@ -21,9 +21,20 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
# Skip when this version is already on crates.io — a version can never be
|
||||
# re-uploaded, so publishing one that exists (released by hand, or a re-run
|
||||
# of this job) is a guaranteed failure rather than a real problem.
|
||||
- name: Publish to crates.io
|
||||
if: ${{ env.CARGO_REGISTRY_TOKEN != '' }}
|
||||
run: cargo publish
|
||||
run: |
|
||||
name=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].name')
|
||||
version=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].version')
|
||||
if curl -sf -H 'User-Agent: cortiq-gateway-release' \
|
||||
"https://crates.io/api/v1/crates/$name/$version" >/dev/null; then
|
||||
echo "$name $version is already published — skipping"
|
||||
else
|
||||
cargo publish
|
||||
fi
|
||||
|
||||
# --- prebuilt binaries + GitHub Release -----------------------------------
|
||||
release:
|
||||
|
||||
+409
@@ -3,8 +3,417 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.48] - 2026-08-21
|
||||
|
||||
Verified end to end: Open WebUI 0.11.0 → gateway → managed CMF/Qwen, native
|
||||
function calling, tool executed and its result fed back into the reply.
|
||||
|
||||
### Fixed
|
||||
- **A tool call is never delivered twice.** Qwen-family templates write the call into the
|
||||
generated text as `<tool_call>{…}</tool_call>`; some runtimes parse it into structured
|
||||
`tool_calls` *and* still stream the raw markup as `content`, so the client sees the call
|
||||
once as prose it must not show and once as the real thing. The gateway now strips the
|
||||
markup from `content` — including when the marker is split across SSE chunks — while
|
||||
passing the structured call through untouched. Ordinary text containing `<` is not
|
||||
affected, and an unterminated block is handed back as text rather than swallowed.
|
||||
- **`finish_reason` agrees with the reply.** A response carrying tool calls now always ends
|
||||
with `finish_reason: "tool_calls"`, whatever the upstream called it — clients branch on
|
||||
that field to decide whether to run the tool.
|
||||
- **Streamed `tool_calls` are brought to the OpenAI shape**: `index` on every entry, `id`
|
||||
and `type` exactly once per call (continuation deltas that carry only an argument
|
||||
fragment are left alone, as the spec requires), and `function.arguments` always a string
|
||||
even when the runtime sends an object.
|
||||
|
||||
### Added
|
||||
- **Unparsed `<tool_call>` markup is promoted into a real call.** When the upstream never
|
||||
parsed it, the gateway turns it into a proper `tool_calls` entry with a generated
|
||||
`call_…` id, so the call is executable instead of decorative. Markup that is not a JSON
|
||||
object with a `name` is ignored — a stray marker must not become a bogus call.
|
||||
|
||||
## [0.2.47] - 2026-08-21
|
||||
|
||||
### Changed
|
||||
- **The hosted allaigate router is addressed by hostname:
|
||||
`https://router.allaigate.com`** is now the default `[router].url`, and
|
||||
`verify_tls` defaults to `true`. The endpoint used to be reachable only by bare IP,
|
||||
whose certificate is issued for the hostname — so every hosted setup had to run with
|
||||
certificate verification switched off. It no longer does.
|
||||
- **Existing configs are migrated on load.** A `[router].url` still pointing at the old
|
||||
`138.226.222.209` (with or without a scheme or trailing slash) is rewritten to the
|
||||
hostname, `verify_tls` is turned back on, and the file is saved — the endpoint is the
|
||||
same machine, and this is the difference between a verified connection and a blind one.
|
||||
A self-hosted router URL is never touched.
|
||||
- README, `docs/ROUTER.md` (both languages), `config/gateway.example.toml` and
|
||||
`bench/accuracy.py` updated to the hostname.
|
||||
|
||||
## [0.2.46] - 2026-08-21
|
||||
|
||||
Verified end to end against a real Open WebUI container talking to the gateway.
|
||||
|
||||
### Fixed
|
||||
- **Offered `tools` no longer make a local model unroutable.** Open WebUI attaches its ~26
|
||||
built-in tools (knowledge bases, notes, calendar, …) to *every* message, and the gateway
|
||||
treated that offer as a hard capability requirement — so a plain "hello" against a pool of
|
||||
tools-less local models failed with «No healthy model available that satisfies required
|
||||
capabilities». In the OpenAI contract `tools` is an offer: only `tool_choice: "required"`
|
||||
(or a named function) makes the call impossible without them. The gateway now still
|
||||
prefers a tool-capable model, but falls back to serving the request with the tools
|
||||
stripped (along with `tool_choice`/`parallel_tool_calls`, which a provider rejects on
|
||||
their own). `tool_choice: "required"` still fails loudly, as it must.
|
||||
- **A pinned model that merely lacks tool support is honoured** instead of being reported as
|
||||
«not available — it is not in the pool (deleted?)». An unknown pin still fails loudly.
|
||||
|
||||
### Added
|
||||
- `X-Cortiq-Tools-Dropped: true` (and `cortiq.tools_dropped` in the echo block) whenever the
|
||||
gateway answers without the tools the client offered, so the downgrade is never silent.
|
||||
|
||||
## [0.2.45] - 2026-08-21
|
||||
|
||||
### Fixed
|
||||
- **Open WebUI (and every strict OpenAI client) can finally chat with the gateway.**
|
||||
`POST /v1/chat/completions` rejected perfectly ordinary payloads with a bare HTTP 422 —
|
||||
a plain-text body no OpenAI client can parse, which is why the chat window just spun with
|
||||
no answer and no error. The request parser now accepts every shape real clients send:
|
||||
`content` as a string, `null`, or an array of content parts (text parts concatenated,
|
||||
images skipped), `tools: null`, `stream: null`, a string `reasoning_effort`
|
||||
(`low`/`medium`/`high`) as well as a numeric budget, `max_completion_tokens` as an alias
|
||||
for `max_tokens`, `messages[].name` and `messages[].tool_call_id` proxied through, and a
|
||||
missing `Content-Type`. A body that genuinely cannot be read now returns an OpenAI error
|
||||
envelope with HTTP 400.
|
||||
- **Strict OpenAI response JSON, no optional `null`s.** Absent tool calls no longer emit
|
||||
`"tool_calls": null` (the key is omitted entirely); `finish_reason` is mapped to the
|
||||
spec's closed set, so `cortiq serve`'s `max_tokens` reaches clients as `length`; the
|
||||
response always carries a non-empty `id`.
|
||||
- **`model` echoes the id the client requested** (`cortiq-auto` stays `cortiq-auto`, not the
|
||||
upstream's `cortiq`), matching OpenAI's own behaviour. The model that actually answered
|
||||
stays in `cortiq.selected_model` and `X-Cortiq-Selected-Model`. Applies to
|
||||
`/v1/chat/completions`, `/v1/completions` and `/v1/messages`.
|
||||
- **Streaming is re-normalised instead of forwarded verbatim.** Every chunk is reissued as a
|
||||
strict `chat.completion.chunk`: the stream opens with a `role` delta, `message`-shaped
|
||||
chunks are converted to `delta`, empty deltas are dropped, and it always closes with one
|
||||
terminal `finish_reason` and `data: [DONE]` — even when the upstream stops early or dies
|
||||
mid-flight, in which case the failure arrives as an SSE `error` event rather than a client
|
||||
waiting forever. SSE responses also carry `X-Accel-Buffering: no` so nginx/Traefik cannot
|
||||
buffer the stream into a hang.
|
||||
- **Client bookkeeping keys are no longer proxied upstream.** `chat_id`, `session_id`,
|
||||
`metadata`, `features`, `background_tasks` and friends (plus a non-string `user`) are
|
||||
stripped before the provider call, so strict backends stop rejecting the request.
|
||||
|
||||
### Added
|
||||
- **`GET /v1/models` lists managed local CMF models.** Every `[[cmf.servers]]` entry now
|
||||
appears alongside `cortiq-auto` and the `[[models]]` pool, with live `running`/`healthy`
|
||||
state in a non-standard `cortiq` field, so Open WebUI and other clients can pin a local
|
||||
`.cmf` directly. Entries also carry `created`.
|
||||
|
||||
### Changed
|
||||
- Dropped leftover `[to_wire]` debug prints from the OpenAI provider — they wrote to stderr
|
||||
on every single request.
|
||||
|
||||
## [0.2.44] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Power controls in Settings**: «Stop everything & power off» stops every managed local
|
||||
model server and exits the gateway (the panel shows a clear "gateway is off" screen);
|
||||
«Restart everything» (relabeled) re-execs the gateway with its local servers. Both behind
|
||||
confirm dialogs, translated in all seven languages.
|
||||
|
||||
### Fixed
|
||||
- `cortiq-gateway` typed in any directory picks up your configuration. Resolution order:
|
||||
`--config` → a readable `./config/gateway.toml` → `$CORTIQ_GATEWAY_HOME` →
|
||||
`~/.cortiq-gateway/gateway.toml`. An unrelated project that merely contains `Cargo.toml`
|
||||
and a `config/` folder no longer hijacks the config path ("cannot read config
|
||||
config/gateway.toml").
|
||||
|
||||
## [0.2.43] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
|
||||
`/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
|
||||
account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
|
||||
per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
|
||||
configured = open mode, as before.
|
||||
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
|
||||
advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
|
||||
when the server becomes healthy, so a local model whose chat template can drive tool
|
||||
calling serves agent (tools) traffic with zero manual configuration. Manual override:
|
||||
`caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
|
||||
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
|
||||
remaining credit is a KPI card, red under $5.
|
||||
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
|
||||
per harness, an install command chip when the harness is missing, and a paste-your-key
|
||||
field that substitutes the real key into every snippet.
|
||||
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
|
||||
Caddy TLS fronting, security checklist, backup list.
|
||||
|
||||
### Fixed
|
||||
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
|
||||
route's chain has no model with the required capabilities (tools), the gateway escalates
|
||||
through medium→high chains and then the rest of the pool instead of returning
|
||||
"No healthy model available". Explicit pins still fail loudly.
|
||||
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
|
||||
settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
|
||||
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
|
||||
no history-expansion `!`), and the install chips are readable in the light theme.
|
||||
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
|
||||
into the sidecar totals file — per-account all-time and month spend survive compaction
|
||||
and restarts.
|
||||
|
||||
### Breaking
|
||||
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
|
||||
will now be rejected with 401 — either remove the keys from the config or start sending
|
||||
`Authorization: Bearer <key>`.
|
||||
|
||||
## [0.2.42] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **The continuous complexity score is visible**: the benchmark's routing table and the
|
||||
dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
|
||||
see what different prompts actually weigh and tune the boundary sliders from real traffic.
|
||||
|
||||
### Changed
|
||||
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
|
||||
«Рабочий стол».
|
||||
|
||||
## [0.2.41] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
|
||||
(the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
|
||||
other, the numbers ride on the handles and the legend updates live — no hand-typed score
|
||||
ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
|
||||
(`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
|
||||
stats; unchecked = the router's own bands, untouched behavior.
|
||||
|
||||
### Fixed
|
||||
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
|
||||
selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
|
||||
facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
|
||||
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
|
||||
fresh template — difficulty is judged in the background, so the second occurrence is the
|
||||
honest steady-state answer.
|
||||
|
||||
## [0.2.40] - 2026-08-08
|
||||
|
||||
### Added
|
||||
- **Per-key spend accounting** on the API-keys page: requests, tokens in/out and cost per key
|
||||
for a selectable window (24h / 7d / 30d) plus a cumulative all-time cost column — "which
|
||||
department spends what" at a glance.
|
||||
- **Bounded stats on disk**: once `stats.jsonl` outgrows `stats.max_file_mb` (default 16, 0
|
||||
disables), records older than the retention window fold into cumulative per-account totals in
|
||||
a small sidecar file (tmp+rename on both) and leave the log. All-time accounting survives
|
||||
compaction and restarts; nothing is ever counted twice.
|
||||
- **Per-complexity models inside a task rule**: each task is edited as rows of
|
||||
"model + complexity" pairs — `[routing.tasks.code] low=["cheap"] high=["best"] any=[...]`.
|
||||
The tier-specific pick leads its own tier, `any` (and legacy plain arrays) leads everywhere,
|
||||
the global tier chain and `default` always remain as fallback, and high verdicts still
|
||||
escalate over any-tier pins.
|
||||
|
||||
### Changed
|
||||
- Benchmark hard probes are generated from wider interdisciplinary pools (6 families each) and
|
||||
retry once with a fresh template — the router's novelty decay no longer flips them red.
|
||||
|
||||
### Fixed
|
||||
- The `task-rule` badge is honest: it appears only when the rule actually chose the head of the
|
||||
candidate chain, not whenever a rule merely exists for the task.
|
||||
|
||||
## [0.2.39] - 2026-08-07
|
||||
|
||||
### Added
|
||||
- **Benchmark, redesigned into the console** (⏱ sidebar entry): two explicit modes. **Smart
|
||||
routing** (default) sends every probe through `cortiq-auto` and judges the ROUTER — task label
|
||||
vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge —
|
||||
with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a
|
||||
"requests went to" per-model distribution. **Individual models** compares pinned models
|
||||
head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run ×
|
||||
and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
|
||||
- **Hard probes that exercise the high tier**: two route-judged tests (science / ops pools)
|
||||
generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings
|
||||
decay with the router's novelty signal. Pass = the router said `high` and the request escalated;
|
||||
the tiny token budget keeps them at cents even on the priciest model.
|
||||
- **One-paste harness setup** on the API-keys page: Claude Code (native Anthropic `/v1/messages`,
|
||||
now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline —
|
||||
each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses
|
||||
already installed on its machine (✓ on the tab, "already configured for this gateway" status).
|
||||
- **Stale-tab detection**: open consoles notice a gateway restart via the meta start marker and
|
||||
show a reload banner instead of silently running an old interface.
|
||||
- Log viewer Clear button (`DELETE /admin/api/logs`).
|
||||
|
||||
### Changed
|
||||
- **High complexity escalates over task rules**: at tier `high` the tier chain leads and the task
|
||||
rule demotes to fallback — a `code → cheap-model` rule no longer pins the hardest prompts to
|
||||
the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
|
||||
- **Honest stream costs**: the final cost rides a closing SSE event (`cortiq_cost_usd`) since the
|
||||
initial header is written before generation; money renders with 4 decimals (last two smaller)
|
||||
and grows precision below $0.0001 instead of collapsing to $0.0000.
|
||||
- **Oracle-sized timeouts**: router default 800 ms → 15 s (hosted escalations take ~10 s);
|
||||
provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall
|
||||
cap so long generations and streams are never cut.
|
||||
- Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-`<think>`),
|
||||
router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on
|
||||
reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
|
||||
- New-key dialog simplified to account / rate limit / optional custom value.
|
||||
|
||||
### Fixed
|
||||
- `cortiq bench` results showed "error" with newer cortiq releases that pretty-print `--json`:
|
||||
the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers
|
||||
steady decode speed (the old heuristic could report `allocs_per_token` as tok/s).
|
||||
|
||||
## [0.2.38] - 2026-08-07
|
||||
|
||||
### Added
|
||||
- **Image & video generation in the Playground**: three tabs now — Chat, Image, Video. Image runs
|
||||
`cortiq imagine` over a local text-to-image `.cmf` (Lumina) with width/height/steps/CFG/seed,
|
||||
live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser
|
||||
with one-click PNG export. Video runs `cortiq animate` (MiniMax-H3) with frames/steps/quality
|
||||
and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded
|
||||
in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens
|
||||
opportunistically when ffmpeg exists). Local `.cmf` files are classified chat/image/video via
|
||||
`cortiq info`; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56.
|
||||
- **Task-based routing**: `[routing.tasks]` maps a router task label to an ordered model list.
|
||||
A matching rule's models lead the candidate chain and the complexity chain follows as fallback,
|
||||
so a task pin never weakens failover. The Routing page builds the editor from the account's
|
||||
live taxonomy (`GET /v1/taxonomies`), shows each task's share of 7-day traffic, and supports
|
||||
custom labels. Responses whose chain was headed by a rule report `route_source "router+task-rule"`.
|
||||
- **Generation gallery** under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete),
|
||||
and generations **queue up** (with position shown) instead of being rejected.
|
||||
- **Chat is a real dialogue**: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live
|
||||
metrics: token counts, tok/s and time-to-first-token.
|
||||
- **Exact stream token counts** when the model server provides usage; the gateway fallback counts
|
||||
content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate.
|
||||
- **Jobs survive restarts**: import and media job lists persist to `<data>/jobs-*.json` and are
|
||||
reconciled on start — the Register button no longer vanishes because the gateway restarted.
|
||||
- **Admin console, ops batch**: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy);
|
||||
a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at
|
||||
the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download;
|
||||
a Restart-gateway button; an update-available notice (crates.io checked twice a day); config
|
||||
export as the real `gateway.toml` and TOML import (validated, hot-reloaded); "Connect your app"
|
||||
snippets (curl/Python/JS) on the keys page; one-click `cortiq bench --json` on the Models page.
|
||||
- **Docker ships the cortiq binary** (built in the builder stage, plus `libvulkan1`): local `.cmf`
|
||||
chat models and media generation now work in containers, where auto-install never could.
|
||||
- **Security**: an open admin console listening on 0.0.0.0 outside Docker warns in the log and
|
||||
shows a dismissible banner.
|
||||
|
||||
### Fixed
|
||||
- The default config pointed at a placeholder `localhost:8080` router with a nonexistent taxonomy,
|
||||
leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now
|
||||
targets the hosted allaigate router and degrades gracefully until a key is set.
|
||||
- The wizard's Finish lands on the Playground instead of the Models list.
|
||||
|
||||
## [0.2.37] - 2026-08-06
|
||||
|
||||
### Added
|
||||
- **Zero-argument startup**: `cortiq-gateway` now runs with no flags. The config is created on
|
||||
first launch (repo `config/gateway.toml` when run from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml`, or `$CORTIQ_GATEWAY_HOME`), the browser opens by itself,
|
||||
and a busy port is swapped for the next free one. `--no-browser` / `--no-port-search` opt out;
|
||||
`--config` still takes an explicit path.
|
||||
- **No admin token required by default**: the console opens without auth. Set `--admin-token`,
|
||||
`[admin].token` or `[admin].token_env` to protect it.
|
||||
- **First-run wizard** (`/admin#/onboarding`): pick a language, download a ready-made `.cmf`
|
||||
model, and finish — Finish connects every completed download as a local model, starts its
|
||||
`cortiq serve` and points `routing.default` at it. Nothing to edit by hand.
|
||||
- **Ready-made models are read live** from the `infosave` HuggingFace account in both the wizard
|
||||
and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a
|
||||
delete button. Newly published models appear without a gateway update.
|
||||
- **Windows-friendly data dir**: `%APPDATA%` / `%LOCALAPPDATA%` / `%USERPROFILE%` are used before
|
||||
falling back, so running the exe from `Downloads` never writes into `Program Files`.
|
||||
- **Full localization**: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr).
|
||||
Import, Settings → Local models, the model form and the whole first-run wizard were English-only
|
||||
in de/fr/es/zh/tr before this release.
|
||||
|
||||
### Fixed
|
||||
- **Download progress jumped straight to 100% and then hung**: with `-L`, curl prints a complete
|
||||
`0→100%` progress bar for the redirect hop before the transfer starts, and that bar was being
|
||||
parsed as the download's progress. Progress is now derived from bytes on disk against the size
|
||||
reported by the HuggingFace tree (or a `HEAD` request), so the bar and the reported
|
||||
`done / total` bytes track the real transfer.
|
||||
- **Re-downloading over an existing file reported 100% instantly**: the previous copy is now
|
||||
removed before the transfer starts, so the size poller cannot read stale bytes.
|
||||
- **Downloaded models were not recognised**: the admin UI derived the output file name
|
||||
independently of the gateway, so a finished download did not mark its card as downloaded and the
|
||||
delete button never appeared. The gateway now returns the exact output stem (`local_name`) and
|
||||
the UI sends it back verbatim for download, progress and delete.
|
||||
- **Registering a local model produced an unusable pool entry**: with `manage_server = false` the
|
||||
model was added to the static pool pointing at `cortiq_server_url`, where nothing listens —
|
||||
every call failed with `upstream unavailable`. Registration now always creates a managed server
|
||||
the gateway runs itself, and repairs a stale static entry for the same file.
|
||||
- **Cancel did nothing during a sharded download**: cancellation was only checked between part
|
||||
files, so a multi-GB part had to finish first. The running transfer is now aborted immediately
|
||||
and the partial output removed.
|
||||
- **Deleting a model left the config broken**: removing a `.cmf` now also unregisters the model
|
||||
and stops its managed server instead of leaving a pool entry pointing at a missing file.
|
||||
- **Two variants of one repo collapsed into a single job row**; jobs are now keyed by repo + variant.
|
||||
- Curl's stderr is captured into the job log, so a failed download explains itself instead of
|
||||
showing a bare exit code.
|
||||
|
||||
## [0.2.36] - 2026-07-25
|
||||
|
||||
### Fixed
|
||||
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
|
||||
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
|
||||
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
|
||||
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.
|
||||
|
||||
## [0.2.35] - 2026-07-23
|
||||
|
||||
### Fixed
|
||||
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
|
||||
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.
|
||||
|
||||
## [0.2.34] - 2026-07-23
|
||||
|
||||
### Added
|
||||
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
|
||||
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.
|
||||
|
||||
### Fixed
|
||||
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
|
||||
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.
|
||||
|
||||
## [0.2.31] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
|
||||
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.
|
||||
|
||||
### Fixed
|
||||
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
|
||||
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.
|
||||
|
||||
## [0.2.30] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
|
||||
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.
|
||||
|
||||
|
||||
### Added
|
||||
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
|
||||
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
### Added
|
||||
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
|
||||
gateway + a local Ollama model in one command. GHCR badge in the READMEs.
|
||||
|
||||
Generated
+249
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.10"
|
||||
version = "0.2.48"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
@@ -297,10 +297,12 @@ dependencies = [
|
||||
"futures",
|
||||
"getrandom 0.2.17",
|
||||
"mime_guess",
|
||||
"open",
|
||||
"reqwest",
|
||||
"rust-embed",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sysinfo",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tokio-stream",
|
||||
@@ -340,6 +342,16 @@ dependencies = [
|
||||
"crypto-common",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dispatch2"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "displaydoc"
|
||||
version = "0.2.6"
|
||||
@@ -743,6 +755,25 @@ version = "2.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
|
||||
|
||||
[[package]]
|
||||
name = "is-docker"
|
||||
version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "928bae27f42bc99b60d9ac7334e3a21d10ad8f1835a4e12ec3ec0464765ed1b3"
|
||||
dependencies = [
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is-wsl"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "173609498df190136aa7dea1a91db051746d339e18476eed5ca40521f02d7aa5"
|
||||
dependencies = [
|
||||
"is-docker",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is_terminal_polyfill"
|
||||
version = "1.70.2"
|
||||
@@ -853,6 +884,15 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ntapi"
|
||||
version = "0.4.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae"
|
||||
dependencies = [
|
||||
"winapi",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nu-ansi-term"
|
||||
version = "0.50.3"
|
||||
@@ -862,6 +902,63 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2"
|
||||
version = "0.6.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3a12a8ed07aefc768292f076dc3ac8c48f3781c8f2d5851dd3d98950e8c5a89f"
|
||||
dependencies = [
|
||||
"objc2-encode",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-core-foundation"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"dispatch2",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-encode"
|
||||
version = "4.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33"
|
||||
|
||||
[[package]]
|
||||
name = "objc2-foundation"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"objc2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-io-kit"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"objc2-core-foundation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-open-directory"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bb82bed227edf5201dfedf072bba4015a33d3d4a98519837295a90f0a23f676d"
|
||||
dependencies = [
|
||||
"objc2",
|
||||
"objc2-core-foundation",
|
||||
"objc2-foundation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
@@ -874,6 +971,16 @@ version = "1.70.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
||||
|
||||
[[package]]
|
||||
name = "open"
|
||||
version = "5.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f9cfef937e9c486488c7e3d949ae31c0f1d06bdacd75b99c086cb35356e30408"
|
||||
dependencies = [
|
||||
"is-wsl",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot"
|
||||
version = "0.12.5"
|
||||
@@ -1400,6 +1507,21 @@ dependencies = [
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sysinfo"
|
||||
version = "0.39.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d2071df9448915b71c4fe6d25deaf1c22f12bd234f01540b77312bb8e41361e6"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"memchr",
|
||||
"ntapi",
|
||||
"objc2-core-foundation",
|
||||
"objc2-io-kit",
|
||||
"objc2-open-directory",
|
||||
"windows",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "thiserror"
|
||||
version = "1.0.69"
|
||||
@@ -1884,6 +2006,22 @@ dependencies = [
|
||||
"rustls-pki-types",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winapi"
|
||||
version = "0.3.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
|
||||
dependencies = [
|
||||
"winapi-i686-pc-windows-gnu",
|
||||
"winapi-x86_64-pc-windows-gnu",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winapi-i686-pc-windows-gnu"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
|
||||
|
||||
[[package]]
|
||||
name = "winapi-util"
|
||||
version = "0.1.11"
|
||||
@@ -1893,12 +2031,113 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winapi-x86_64-pc-windows-gnu"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
|
||||
|
||||
[[package]]
|
||||
name = "windows"
|
||||
version = "0.62.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580"
|
||||
dependencies = [
|
||||
"windows-collections",
|
||||
"windows-core",
|
||||
"windows-future",
|
||||
"windows-numerics",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-collections"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-core"
|
||||
version = "0.62.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
|
||||
dependencies = [
|
||||
"windows-implement",
|
||||
"windows-interface",
|
||||
"windows-link",
|
||||
"windows-result",
|
||||
"windows-strings",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-future"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-link",
|
||||
"windows-threading",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-implement"
|
||||
version = "0.60.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-interface"
|
||||
version = "0.59.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-link"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
|
||||
|
||||
[[package]]
|
||||
name = "windows-numerics"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26"
|
||||
dependencies = [
|
||||
"windows-core",
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-result"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-strings"
|
||||
version = "0.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.52.0"
|
||||
@@ -1959,6 +2198,15 @@ dependencies = [
|
||||
"windows_x86_64_msvc 0.53.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-threading"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_gnullvm"
|
||||
version = "0.52.6"
|
||||
|
||||
+18
-5
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.10"
|
||||
version = "0.2.48"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
@@ -10,12 +10,16 @@ homepage = "https://api.allaigate.com"
|
||||
readme = "README.md"
|
||||
keywords = ["llm", "gateway", "proxy", "openai", "routing"]
|
||||
categories = ["web-programming::http-server", "command-line-utilities"]
|
||||
# Files shipped in the published crate. The embedded admin SPA (web/) MUST be
|
||||
# included because rust-embed compiles it in; docs/screenshots are excluded to
|
||||
# keep the crate lean.
|
||||
# Files shipped in the published crate. The embedded admin SPA MUST be included
|
||||
# because rust-embed compiles it in; docs/screenshots are excluded to keep the
|
||||
# crate lean. The web/ subfolders are listed one by one rather than as web/**/*
|
||||
# so stray scratch files dropped next to the SPA never end up published.
|
||||
include = [
|
||||
"src/**/*",
|
||||
"web/**/*",
|
||||
"web/index.html",
|
||||
"web/favicon.svg",
|
||||
"web/css/**/*",
|
||||
"web/js/**/*",
|
||||
"config/gateway.example.toml",
|
||||
"README.md",
|
||||
"CHANGELOG.md",
|
||||
@@ -54,6 +58,11 @@ tokio-stream = "0.1"
|
||||
|
||||
# CLI / config
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
# open browser on first run (optional)
|
||||
open = { version = "5", optional = true }
|
||||
|
||||
# host metrics for the admin dashboard (CPU / RAM / disk)
|
||||
sysinfo = "0.39"
|
||||
|
||||
# errors / logging / metrics
|
||||
thiserror = "1"
|
||||
@@ -61,6 +70,10 @@ anyhow = "1"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
|
||||
[features]
|
||||
default = ["open-browser"]
|
||||
open-browser = ["open"]
|
||||
|
||||
[profile.release]
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
|
||||
+19
-5
@@ -1,6 +1,12 @@
|
||||
# ---- build stage ------------------------------------------------------------
|
||||
FROM rust:1-slim AS builder
|
||||
# pin builder and runtime to the same Debian release — a floating `rust:1-slim`
|
||||
# drifts to a newer glibc than the runtime image and the binary refuses to start
|
||||
FROM rust:1-slim-trixie AS builder
|
||||
WORKDIR /app
|
||||
# the cortiq runtime CANNOT be auto-installed inside the container (no cargo in
|
||||
# the runtime image), so it is compiled here and shipped alongside the gateway —
|
||||
# local .cmf chat models and image/video generation work in Docker out of the box
|
||||
RUN cargo install cortiq-cli --locked
|
||||
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY src ./src
|
||||
@@ -8,18 +14,26 @@ COPY web ./web
|
||||
RUN cargo build --release
|
||||
|
||||
# ---- runtime stage ----------------------------------------------------------
|
||||
FROM debian:bookworm-slim
|
||||
FROM debian:trixie-slim
|
||||
# curl — model imports download ready .cmf files with it;
|
||||
# libvulkan1 — optional GPU offload for cortiq (falls back to CPU without a GPU)
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl libvulkan1 \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& useradd -r -u 10001 cortiq \
|
||||
&& mkdir -p /app/config
|
||||
&& mkdir -p /app/config /app/data/models
|
||||
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
|
||||
COPY --from=builder /usr/local/cargo/bin/cortiq /usr/local/bin/cortiq
|
||||
# ship the example as the default config so `docker run` works out of the box
|
||||
COPY config/gateway.example.toml /app/config/gateway.toml
|
||||
RUN chown -R 10001:10001 /app
|
||||
WORKDIR /app
|
||||
USER cortiq
|
||||
# all gateway state (downloaded models, stats, secrets) lives under /app/data —
|
||||
# mount a volume there to keep it across container recreates
|
||||
ENV CORTIQ_GATEWAY_HOME=/app/data
|
||||
# runs as root so writes to bind-mounted host folders (Docker Desktop on
|
||||
# Windows/macOS) don't fail with permission errors (curl exit code 23);
|
||||
# add `--user 10001` to docker run to harden when not using host bind mounts
|
||||
EXPOSE 9000
|
||||
ENTRYPOINT ["cortiq-gateway"]
|
||||
CMD ["--config", "/app/config/gateway.toml"]
|
||||
|
||||
@@ -36,10 +36,12 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Zero-config start.** `cortiq-gateway` with no flags: the config is created for you, a busy port is swapped for a free one, the browser opens on a setup wizard that downloads a ready-made local model and wires it into routing — no TOML, no token, no separate model server.
|
||||
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
|
||||
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
|
||||
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down.
|
||||
- **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down. Plus **per-task rules** (`[routing.tasks]`): pin translations to a local model and code to your strongest coder — the complexity chain stays as fallback ([docs](docs/ROUTING.md)).
|
||||
- **Image & video generation.** The Playground drives local text-to-image (Lumina) and text-to-video (MiniMax-H3) `.cmf` models via `cortiq imagine`/`animate`: live progress with ETA, a generation gallery, PNG export, in-browser AVI playback — no ffmpeg anywhere.
|
||||
- **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard).
|
||||
- **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes.
|
||||
- **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint.
|
||||
@@ -54,11 +56,13 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
```bash
|
||||
# Docker (GitHub Container Registry)
|
||||
docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
-v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
-v "$PWD/config:/app/config" -v cortiq-data:/app/data \
|
||||
ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
# → admin console at http://localhost:9000/admin
|
||||
|
||||
# from crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # no flags, no config file, no token — see Quick start
|
||||
|
||||
# or build from source
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -67,7 +71,10 @@ cargo build --release # ./target/release/cortiq-gateway
|
||||
```
|
||||
|
||||
The image ships a default config and runs out of the box; mount `/app/config` (with a
|
||||
`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
|
||||
`gateway.toml`) to use your own. Downloaded models and stats live in `/app/data` — mount
|
||||
a volume there so they survive container recreates (a host folder works too, e.g.
|
||||
`-v "$PWD/models:/app/data/models"`). Tags: `:latest`, `:X.Y.Z`, `:edge`
|
||||
(linux/amd64 + arm64).
|
||||
|
||||
**Docker Compose** — gateway + a local Ollama model in one command:
|
||||
|
||||
@@ -80,6 +87,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # pull a model once
|
||||
See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
|
||||
and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
|
||||
|
||||
Deploying for a team or an office (systemd/launchd/Windows service, TLS reverse proxy, API keys with rate limits and monthly budgets, backups): [docs/DEPLOY.md](docs/DEPLOY.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Admin console
|
||||
@@ -98,32 +107,54 @@ The gateway ships with an embedded web console at **`/admin`**:
|
||||
|  |
|
||||
| test the live pipeline and see the routing decision |
|
||||
|
||||
The console is **open by default** — no token needed. To protect it, set any one of
|
||||
`--admin-token <TOKEN>`, `[admin].token`, or `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <YOUR_TOKEN>
|
||||
cortiq-gateway --admin-token <YOUR_TOKEN>
|
||||
# open http://localhost:9000/admin?token=<YOUR_TOKEN>
|
||||
```
|
||||
|
||||
If `--admin-token` / `[admin].token_env` are not set, a token is generated at startup
|
||||
and printed to the log. All `/admin/api/*` endpoints require a Bearer admin token;
|
||||
secret values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
Once a token is set, every `/admin/api/*` endpoint requires it as a Bearer token. Secret
|
||||
values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick start
|
||||
|
||||
```bash
|
||||
# 1. start a router (the hosted allaigate router, or a local cortiq-router)
|
||||
# it listens e.g. on http://localhost:8080 (or https://api.allaigate.com)
|
||||
|
||||
# 2. describe your model pool
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. run the gateway
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
# Gateway listens on 0.0.0.0:9000 and serves the admin console at /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
That is the whole command. On first run the gateway:
|
||||
|
||||
1. **creates its config** — `config/gateway.toml` when started from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml` (override with `$CORTIQ_GATEWAY_HOME` or `--config`);
|
||||
2. **picks a free port** if 9000 is taken, and writes the choice back to the config;
|
||||
3. **opens your browser** on the setup wizard at `/admin#/onboarding`.
|
||||
|
||||
The wizard takes you from nothing to a working local model:
|
||||
|
||||
| Step | What happens |
|
||||
|---|---|
|
||||
| **Language** | 7 languages; changeable later from the header |
|
||||
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
|
||||
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
|
||||
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, `routing.default` points at it, and you land in the Playground |
|
||||
|
||||
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
|
||||
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
|
||||
both still work, and changes hot-reload without a restart.
|
||||
|
||||
```bash
|
||||
# explicit config / port / no browser — all optional
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
For smart routing across a pool, point `[router]` at the hosted allaigate router or a local
|
||||
`cortiq-router` (see [docs/ROUTER.md](docs/ROUTER.md)). Without a router the gateway degrades
|
||||
gracefully to the default model, so a local-only setup works out of the box.
|
||||
|
||||
Now any OpenAI client works through the gateway:
|
||||
|
||||
```python
|
||||
@@ -144,9 +175,9 @@ print(resp.choices[0].message.content)
|
||||
`model: "cortiq-auto"` enables routing. Any **real** model name from the config
|
||||
(`"gpt-4o-mini"`, `"local-qwen"`) is a direct passthrough, without routing.
|
||||
|
||||
> **Using the hosted allaigate router?** Set `url = "https://138.226.222.209"`,
|
||||
> `verify_tls = false`, `taxonomy_id = "data-assistant"`, and a `cortiq_…` key in
|
||||
> `CORTIQ_ROUTER_KEY`. On hard prompts the router escalates to an oracle (~10 s), so
|
||||
> **Using the hosted allaigate router?** It is the default: `url =
|
||||
> "https://router.allaigate.com"` with `verify_tls = true`. Set `taxonomy_id =
|
||||
> "data-assistant"` and a `cortiq_…` key in `CORTIQ_ROUTER_KEY`. On hard prompts the router escalates to an oracle (~10 s), so
|
||||
> set `timeout_ms = 12000+` — otherwise the gateway gracefully degrades to the default model.
|
||||
|
||||
---
|
||||
|
||||
+45
-18
@@ -36,9 +36,11 @@
|
||||
|
||||
## ✨ Ключевое
|
||||
|
||||
- **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели.
|
||||
- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
|
||||
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
|
||||
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен. Плюс **правила по задачам** (`[routing.tasks]`): переводы — в локальную модель, код — сильнейшему кодеру; цепочка по сложности остаётся резервом ([доки](docs/ROUTING.ru.md)).
|
||||
- **Генерация картинок и видео.** Песочница запускает локальные text-to-image (Lumina) и text-to-video (MiniMax-H3) `.cmf` модели через `cortiq imagine`/`animate`: живой прогресс с ETA, галерея генераций, экспорт в PNG, проигрывание AVI прямо в браузере — без ffmpeg вообще.
|
||||
- **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
|
||||
- **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
|
||||
- **Playground.** Прогон промпта через живой пайплайн с разбором решения роутинга.
|
||||
@@ -57,6 +59,7 @@ docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
|
||||
# с crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # без флагов, без конфига, без токена — см. Быстрый старт
|
||||
|
||||
# или сборка из исходников
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -79,6 +82,8 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
||||
См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
|
||||
`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
|
||||
|
||||
Развёртывание для команды/офиса (сервис systemd/launchd/Windows, TLS-прокси, API-ключи с лимитами и месячными бюджетами, бэкапы): [docs/DEPLOY.ru.md](docs/DEPLOY.ru.md).
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Панель управления
|
||||
@@ -94,32 +99,54 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
||||
|  |  |
|
||||
| **Playground** — тест живого пайплайна с решением роутинга | **Dashboard** — трафик, стоимость, латентность |
|
||||
|
||||
Панель **по умолчанию открыта** — токен не нужен. Чтобы закрыть её, задайте любое из
|
||||
`--admin-token <ТОКЕН>`, `[admin].token` или `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <ВАШ_ТОКЕН>
|
||||
cortiq-gateway --admin-token <ВАШ_ТОКЕН>
|
||||
# откройте http://localhost:9000/admin?token=<ВАШ_ТОКЕН>
|
||||
```
|
||||
|
||||
Если `--admin-token` / `[admin].token_env` не заданы — токен генерируется при старте и
|
||||
печатается в лог. Все эндпоинты `/admin/api/*` требуют Bearer admin-токен; значения
|
||||
секретов API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
Когда токен задан, все эндпоинты `/admin/api/*` требуют его как Bearer. Значения секретов
|
||||
API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Быстрый старт
|
||||
|
||||
```bash
|
||||
# 1. поднимите роутер (хостовый allaigate или локальный cortiq-router)
|
||||
# он слушает, например, http://localhost:8080 (или https://api.allaigate.com)
|
||||
|
||||
# 2. опишите свой пул моделей
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. запустите шлюз
|
||||
cargo run --release -- --config config/gateway.toml
|
||||
# Шлюз слушает 0.0.0.0:9000 и отдаёт панель на /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
Это вся команда. При первом запуске шлюз:
|
||||
|
||||
1. **создаёт конфиг** — `config/gateway.toml`, если запущен из репозитория, иначе
|
||||
`~/.cortiq-gateway/gateway.toml` (переопределяется через `$CORTIQ_GATEWAY_HOME` или `--config`);
|
||||
2. **подбирает свободный порт**, если 9000 занят, и записывает выбор в конфиг;
|
||||
3. **открывает браузер** на мастере настройки `/admin#/onboarding`.
|
||||
|
||||
Мастер доводит от нуля до работающей локальной модели:
|
||||
|
||||
| Шаг | Что происходит |
|
||||
|---|---|
|
||||
| **Язык** | 7 языков; потом меняется в шапке |
|
||||
| **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки |
|
||||
| **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` |
|
||||
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё — и вы попадаете в Песочницу |
|
||||
|
||||
Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели.
|
||||
Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг
|
||||
руками — оба пути работают, изменения применяются на лету без перезапуска.
|
||||
|
||||
```bash
|
||||
# явный конфиг / порт / без браузера — всё опционально
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
Для умного роутинга по пулу направьте `[router]` на хостовый allaigate или локальный
|
||||
`cortiq-router` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)). Без роутера шлюз мягко
|
||||
деградирует до модели по умолчанию, поэтому чисто локальная установка работает сразу.
|
||||
|
||||
Теперь любой OpenAI-клиент работает через шлюз:
|
||||
|
||||
```python
|
||||
@@ -140,9 +167,9 @@ print(resp.choices[0].message.content)
|
||||
`model: "cortiq-auto"` включает роутинг. Любое **реальное** имя модели из конфига
|
||||
(`"gpt-4o-mini"`, `"local-qwen"`) — это passthrough напрямую, без роутинга.
|
||||
|
||||
> **Используете хостовый allaigate-роутер?** Укажите `url = "https://138.226.222.209"`,
|
||||
> `verify_tls = false`, `taxonomy_id = "data-assistant"` и ключ `cortiq_…` в
|
||||
> `CORTIQ_ROUTER_KEY`. На сложных запросах роутер эскалирует к oracle (~10 с), поэтому
|
||||
> **Используете хостовый allaigate-роутер?** Он стоит по умолчанию: `url =
|
||||
> "https://router.allaigate.com"`, `verify_tls = true`. Задайте `taxonomy_id =
|
||||
> "data-assistant"` и ключ `cortiq_…` в `CORTIQ_ROUTER_KEY`. На сложных запросах роутер эскалирует к oracle (~10 с), поэтому
|
||||
> ставьте `timeout_ms = 12000+` — иначе шлюз мягко деградирует на default-модель.
|
||||
|
||||
---
|
||||
|
||||
+2
-2
@@ -4,12 +4,12 @@ heuristic (what you get without a semantic classifier, e.g. with LiteLLM/Portkey
|
||||
|
||||
CORTIQ_ROUTER_KEY=cortiq_... python3 bench/accuracy.py
|
||||
|
||||
Env: CORTIQ_ROUTER_KEY (required), ROUTER_URL (default https://138.226.222.209),
|
||||
Env: CORTIQ_ROUTER_KEY (required), ROUTER_URL (default https://router.allaigate.com),
|
||||
TAXONOMY (default data-assistant).
|
||||
"""
|
||||
import json, os, ssl, sys, urllib.request
|
||||
|
||||
ROUTER_URL = os.environ.get("ROUTER_URL", "https://138.226.222.209").rstrip("/")
|
||||
ROUTER_URL = os.environ.get("ROUTER_URL", "https://router.allaigate.com").rstrip("/")
|
||||
TAXONOMY = os.environ.get("TAXONOMY", "data-assistant")
|
||||
KEY = os.environ.get("CORTIQ_ROUTER_KEY", "")
|
||||
CTX = ssl._create_unverified_context()
|
||||
|
||||
@@ -8,17 +8,17 @@
|
||||
listen = "0.0.0.0:9000"
|
||||
|
||||
# --- connection to cortiq-router (decides task type / complexity) -----------
|
||||
# Hosted allaigate router: url = "https://138.226.222.209", verify_tls = false
|
||||
# Hosted allaigate router: url = "https://router.allaigate.com" (the default)
|
||||
# (cert is bound to an IP), taxonomy_id = "data-assistant", a cortiq_… key in
|
||||
# CORTIQ_ROUTER_KEY.
|
||||
# NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the
|
||||
# hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default.
|
||||
[router]
|
||||
url = "http://localhost:8080"
|
||||
url = "https://router.allaigate.com" # hosted allaigate router (or your own cortiq-router URL)
|
||||
api_key_env = "CORTIQ_ROUTER_KEY" # name of the env var holding the router key
|
||||
timeout_ms = 800 # timeout for the /v1/route call (12000+ for oracle)
|
||||
taxonomy_id = "general.task-type" # for allaigate: "data-assistant"
|
||||
verify_tls = true # false if the router uses an IP / self-signed cert
|
||||
timeout_ms = 12000 # hard prompts escalate to an oracle (~10 s)
|
||||
taxonomy_id = "data-assistant" # the hosted router's task taxonomy
|
||||
verify_tls = false # the hosted endpoint is IP-bound; true for your own domain
|
||||
|
||||
# --- how to pick the text to classify from the conversation -----------------
|
||||
[route]
|
||||
@@ -71,7 +71,7 @@ openai_chat = true
|
||||
openai_completions = true
|
||||
openai_embeddings = true
|
||||
openai_models = true
|
||||
anthropic_messages = false
|
||||
anthropic_messages = true
|
||||
mcp = false
|
||||
native_passthrough = true
|
||||
|
||||
@@ -119,6 +119,10 @@ enabled = true
|
||||
file = "config/stats.jsonl" # empty = in-memory only
|
||||
ring_size = 500 # how many recent requests to keep in the feed
|
||||
retention = "7d" # aggregate/series depth (replayed on startup)
|
||||
# stats.jsonl is compacted once it exceeds this size: old records fold into
|
||||
# cumulative per-account totals (sidecar file), so per-key accounting survives
|
||||
# while the disk stays bounded. 0 disables.
|
||||
max_file_mb = 16
|
||||
|
||||
# ============================================================================
|
||||
# Semantic cache — returns a cached answer for prompts semantically near a
|
||||
|
||||
@@ -17,6 +17,10 @@ services:
|
||||
CORTIQ_ROUTER_KEY: ${CORTIQ_ROUTER_KEY:-}
|
||||
volumes:
|
||||
- ./config/gateway.compose.toml:/app/config/gateway.toml:ro
|
||||
# downloaded .cmf models + stats persist here across recreates; swap the
|
||||
# named volume for a host folder (e.g. ./models:/app/data/models) if you
|
||||
# want the files visible next to the compose file
|
||||
- cortiq-data:/app/data
|
||||
depends_on:
|
||||
- ollama
|
||||
restart: unless-stopped
|
||||
@@ -30,4 +34,5 @@ services:
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
cortiq-data:
|
||||
ollama:
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
# Deploying cortiq-gateway for a team / office
|
||||
|
||||
One gateway on a small server, API keys per department, local models for the
|
||||
cheap tiers, cloud models for the hard ones. This page covers the ops side.
|
||||
|
||||
## 1. Run as a service
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Task Scheduler → Create Task → "Run whether user is logged on or not",
|
||||
trigger At startup, action `cortiq-gateway.exe --no-port-search`. Or use
|
||||
[NSSM](https://nssm.cc): `nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS and the outside world
|
||||
|
||||
The gateway itself speaks plain HTTP — put a reverse proxy with TLS in front
|
||||
and bind the gateway to localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE streaming
|
||||
proxy_read_timeout 3600s; # long generations
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` does the
|
||||
same with automatic certificates.)
|
||||
|
||||
## 3. Security checklist
|
||||
|
||||
- **Admin token**: set `[admin] token` (or `CORTIQ_ADMIN_TOKEN`) — without it
|
||||
the console warns that it is open to the network.
|
||||
- **API keys**: create one per department/developer on the keys page. With at
|
||||
least one key configured the model APIs require Bearer / x-api-key —
|
||||
requests without a valid key get 401.
|
||||
- **Rate limits**: per-key requests/min → 429 with Retry-After.
|
||||
- **Monthly budgets**: per-key $ ceiling → 429 insufficient_quota when the
|
||||
account's month-to-date spend crosses it.
|
||||
- Bind to 127.0.0.1 behind the proxy; never expose the admin console raw.
|
||||
|
||||
## 4. Backups
|
||||
|
||||
Everything lives under the data directory (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — full config incl. keys (or use the console's TOML export);
|
||||
- `secrets.toml` — provider API keys (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — request accounting incl.
|
||||
cumulative per-key totals and month-to-date budget spend.
|
||||
|
||||
Copy those three and you can rebuild the instance anywhere.
|
||||
|
||||
## 5. Observability
|
||||
|
||||
- Dashboard: KPIs, per-key spend, recent requests with complexity scores,
|
||||
OpenRouter balance (red under $5).
|
||||
- `GET /metrics` (enable `[telemetry] metrics = true`) for Prometheus.
|
||||
- Logs view in the console; `stats.max_file_mb` keeps the disk bounded.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Развёртывание cortiq-gateway для команды / офиса
|
||||
|
||||
Один шлюз на небольшом сервере, API-ключи по отделам, локальные модели для
|
||||
дешёвых уровней, облачные — для сложных. Эта страница про эксплуатацию.
|
||||
|
||||
## 1. Запуск как сервис
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/cortiq-gateway.service
|
||||
[Unit]
|
||||
Description=Cortiq Gateway
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
User=cortiq
|
||||
Environment=CORTIQ_GATEWAY_HOME=/var/lib/cortiq-gateway
|
||||
ExecStart=/usr/local/bin/cortiq-gateway --no-port-search
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now cortiq-gateway
|
||||
```
|
||||
|
||||
### macOS (launchd)
|
||||
|
||||
```xml
|
||||
<!-- ~/Library/LaunchAgents/com.cortiq.gateway.plist -->
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>Label</key><string>com.cortiq.gateway</string>
|
||||
<key>ProgramArguments</key><array>
|
||||
<string>/usr/local/bin/cortiq-gateway</string>
|
||||
<string>--no-port-search</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
</dict></plist>
|
||||
```
|
||||
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.cortiq.gateway.plist
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Планировщик заданий → Создать задачу → «Выполнять вне зависимости от входа
|
||||
пользователя», триггер «При запуске», действие `cortiq-gateway.exe
|
||||
--no-port-search`. Либо [NSSM](https://nssm.cc):
|
||||
`nssm install cortiq-gateway C:\bin\cortiq-gateway.exe`.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run -d --restart unless-stopped -p 9000:9000 \
|
||||
-v cortiq-data:/data ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
```
|
||||
|
||||
## 2. TLS и внешний мир
|
||||
|
||||
Сам шлюз говорит по HTTP — поставьте перед ним reverse-proxy с TLS, а шлюз
|
||||
привяжите к localhost:
|
||||
|
||||
```toml
|
||||
# gateway.toml
|
||||
listen = "127.0.0.1:9000"
|
||||
```
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name ai.example.com;
|
||||
ssl_certificate /etc/letsencrypt/live/ai.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:9000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off; # SSE-стриминг
|
||||
proxy_read_timeout 3600s; # долгие генерации
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(`caddy reverse-proxy --from ai.example.com --to 127.0.0.1:9000` делает то же
|
||||
с автоматическими сертификатами.)
|
||||
|
||||
## 3. Чек-лист безопасности
|
||||
|
||||
- **Токен админки**: задайте `[admin] token` (или `CORTIQ_ADMIN_TOKEN`) — без
|
||||
него консоль предупреждает, что открыта сети.
|
||||
- **API-ключи**: по ключу на отдел/разработчика на странице ключей. Как только
|
||||
настроен хотя бы один ключ, модельные API требуют Bearer / x-api-key —
|
||||
запросы без валидного ключа получают 401.
|
||||
- **Rate-limit**: запросов/мин на ключ → 429 с Retry-After.
|
||||
- **Месячные бюджеты**: потолок $ на ключ → 429 insufficient_quota, когда
|
||||
траты аккаунта за месяц превысят лимит.
|
||||
- Привязывайте к 127.0.0.1 за прокси; не выставляйте админку наружу голой.
|
||||
|
||||
## 4. Бэкапы
|
||||
|
||||
Всё живёт в data-директории (`CORTIQ_GATEWAY_HOME`):
|
||||
|
||||
- `gateway.toml` — полный конфиг вкл. ключи (или TOML-экспорт из консоли);
|
||||
- `secrets.toml` — ключи провайдеров (chmod 600);
|
||||
- `stats.jsonl` + `stats.jsonl.totals.json` — учёт запросов вкл. вечные
|
||||
тоталы по ключам и месячные траты для бюджетов.
|
||||
|
||||
Скопируйте эти три файла — и инстанс поднимается где угодно.
|
||||
|
||||
## 5. Наблюдаемость
|
||||
|
||||
- Рабочий стол: KPI, расходы по ключам, последние запросы с коэффициентами
|
||||
сложности, баланс OpenRouter (красный ниже $5).
|
||||
- `GET /metrics` (включите `[telemetry] metrics = true`) для Prometheus.
|
||||
- Просмотр логов в консоли; `stats.max_file_mb` ограничивает диск.
|
||||
+61
-9
@@ -53,7 +53,24 @@ Routing rule common to all protocols:
|
||||
```
|
||||
|
||||
Standard OpenAI fields are supported; unknown fields are proxied to the provider unchanged
|
||||
(forward-compatible).
|
||||
(forward-compatible), minus the bookkeeping keys chat UIs attach to the payload
|
||||
(`chat_id`, `session_id`, `metadata`, `features`, …), which are dropped so strict upstreams
|
||||
do not reject the call.
|
||||
|
||||
The parser is deliberately lenient about the shapes real clients send:
|
||||
`content` may be a string, `null`, or an array of content parts (text parts are
|
||||
concatenated, images and other non-text parts are skipped); `tools` may be `null`;
|
||||
`reasoning_effort` may be a string (`low` | `medium` | `high`) or a token budget;
|
||||
`max_completion_tokens` is accepted as an alias for `max_tokens`. A body that still
|
||||
cannot be read comes back as an OpenAI error envelope with HTTP 400 — never a bare 422,
|
||||
which OpenAI clients cannot parse (they show no answer at all).
|
||||
|
||||
`tools` is treated as an **offer**, not a requirement. The gateway prefers a tool-capable
|
||||
model, but when none is available it answers without the tools rather than failing, and says
|
||||
so via `X-Cortiq-Tools-Dropped: true` (chat UIs attach their built-in tools to every message,
|
||||
so the strict reading made ordinary chat impossible on a tools-less local pool).
|
||||
`tool_choice: "required"` — or a named function — is a real requirement and still fails with
|
||||
`502 no_healthy_model` when no model can honour it.
|
||||
|
||||
### Response (non-streaming)
|
||||
|
||||
@@ -62,12 +79,12 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
|
||||
"id": "chatcmpl-9f...",
|
||||
"object": "chat.completion",
|
||||
"created": 1718800000,
|
||||
"model": "local-qwen", // the ACTUALLY selected model
|
||||
"model": "cortiq-auto", // the model the CLIENT asked for, echoed back
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": { "role": "assistant", "content": "x = 2 or x = 3" },
|
||||
"finish_reason": "stop"
|
||||
"finish_reason": "stop" // always one of stop|length|tool_calls|content_filter
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
@@ -86,10 +103,38 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
|
||||
}
|
||||
```
|
||||
|
||||
> `model` echoes the id the client requested, exactly as OpenAI does — clients match the
|
||||
> reply against the id they sent. The model that actually answered is in
|
||||
> `cortiq.selected_model` and in the `X-Cortiq-Selected-Model` header.
|
||||
|
||||
> Optional fields are omitted rather than sent as `null`: absent tool calls mean no
|
||||
> `tool_calls` key at all. `"tool_calls": null` is not in the OpenAI schema and trips
|
||||
> strict clients.
|
||||
|
||||
> The `cortiq` field is an optional extension. OpenAI SDKs ignore it. If the client does
|
||||
> not want extra fields, disable `cortiq.echo` — the metadata will remain in the response
|
||||
> headers.
|
||||
|
||||
Streaming (`stream: true`) emits strict `chat.completion.chunk` events: the stream always
|
||||
opens with a `role` delta, carries exactly one terminal `finish_reason`, and ends with
|
||||
`data: [DONE]` — even when the upstream stops early or dies mid-flight, in which case the
|
||||
failure arrives as an SSE `error` event. A client can therefore never be left waiting on a
|
||||
half-formed stream.
|
||||
|
||||
### Tool calls
|
||||
|
||||
`tools` and `tool_calls` are proxied through, with the wire kept clean:
|
||||
|
||||
* raw `<tool_call>…</tool_call>` markup (how Qwen-family templates express a call in the
|
||||
generated text) is stripped from `content`, so a client never receives the same call
|
||||
twice — once as prose and once structured. Markers split across SSE chunks are handled;
|
||||
ordinary text containing `<` is not touched;
|
||||
* when the upstream never parsed that markup, it is promoted into a real `tool_calls`
|
||||
entry with a generated `call_…` id, so the call is executable rather than decorative;
|
||||
* streamed entries carry `index` always, `id`/`type` once per call, and
|
||||
`function.arguments` as a string even if the runtime sent an object;
|
||||
* a reply carrying tool calls always ends with `finish_reason: "tool_calls"`.
|
||||
|
||||
### Metadata in response headers (always present)
|
||||
|
||||
```
|
||||
@@ -164,20 +209,27 @@ single user message and follows the same pipeline. Response `object` is `"text_c
|
||||
## 4. OpenAI Models
|
||||
|
||||
`GET /v1/models` — auto-discovery for clients. Returns the gateway's virtual models
|
||||
(`cortiq-auto`, `cortiq-embed`) and all real models in the pool.
|
||||
(`cortiq-auto`, `cortiq-embed`), every model in the `[[models]]` pool, and every managed
|
||||
local `[[cmf.servers]]` model, so a client can pin a local `.cmf` directly.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
|
||||
{ "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
|
||||
{ "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
|
||||
{ "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
|
||||
"cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Managed CMF models are listed as soon as they are configured, with their live state in the
|
||||
non-standard `cortiq` field (clients ignore it). They are not filtered by health: a large
|
||||
`.cmf` takes minutes to load, and a model that disappears from the list while it warms up
|
||||
is worse than one marked "not ready yet".
|
||||
|
||||
---
|
||||
|
||||
## 5. Anthropic Messages
|
||||
@@ -198,7 +250,7 @@ single user message and follows the same pipeline. Response `object` is `"text_c
|
||||
"id": "msg_...",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "local-qwen",
|
||||
"model": "cortiq-auto",
|
||||
"content": [ { "type": "text", "text": "x = 2 or x = 3" } ],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": { "input_tokens": 24, "output_tokens": 11 }
|
||||
|
||||
+60
-8
@@ -53,7 +53,24 @@
|
||||
```
|
||||
|
||||
Поддерживаются стандартные поля OpenAI; неизвестные поля проксируются к провайдеру
|
||||
без изменений (forward-compatible).
|
||||
без изменений (forward-compatible) — кроме служебных ключей, которые чат-интерфейсы
|
||||
добавляют в payload для собственных нужд (`chat_id`, `session_id`, `metadata`,
|
||||
`features`, …): они отбрасываются, чтобы строгий upstream не отклонил вызов.
|
||||
|
||||
Парсер намеренно терпим к тем формам, которые реально шлют клиенты: `content` может быть
|
||||
строкой, `null` или массивом content-частей (текстовые части склеиваются, изображения и
|
||||
прочие нетекстовые пропускаются); `tools` может быть `null`; `reasoning_effort` — строкой
|
||||
(`low` | `medium` | `high`) или бюджетом токенов; `max_completion_tokens` принимается как
|
||||
синоним `max_tokens`. Если тело всё-таки не читается, возвращается OpenAI-конверт ошибки с
|
||||
HTTP 400 — но не «голый» 422, который OpenAI-клиенты не умеют разбирать (для пользователя
|
||||
это выглядит как отсутствие ответа).
|
||||
|
||||
`tools` трактуется как **предложение**, а не требование. Шлюз предпочитает модель с
|
||||
поддержкой tools, но если такой нет — отвечает без них, а не падает, и сообщает об этом
|
||||
заголовком `X-Cortiq-Tools-Dropped: true` (чат-интерфейсы добавляют свои встроенные tools в
|
||||
каждое сообщение, поэтому строгое прочтение делало обычный чат на локальном пуле без tools
|
||||
невозможным). `tool_choice: "required"` — или именованная функция — это уже настоящее
|
||||
требование, и оно по-прежнему завершается `502 no_healthy_model`, если его некому выполнить.
|
||||
|
||||
### Ответ (без стриминга)
|
||||
|
||||
@@ -62,12 +79,12 @@
|
||||
"id": "chatcmpl-9f...",
|
||||
"object": "chat.completion",
|
||||
"created": 1718800000,
|
||||
"model": "local-qwen", // РЕАЛЬНО выбранная модель
|
||||
"model": "cortiq-auto", // модель, которую ЗАПРОСИЛ клиент — возвращается как есть
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": { "role": "assistant", "content": "x = 2 или x = 3" },
|
||||
"finish_reason": "stop"
|
||||
"finish_reason": "stop" // всегда одно из stop|length|tool_calls|content_filter
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
@@ -86,9 +103,37 @@
|
||||
}
|
||||
```
|
||||
|
||||
> `model` возвращает тот id, который запросил клиент, — ровно как это делает OpenAI:
|
||||
> клиенты сопоставляют ответ с отправленным id. Модель, которая реально ответила, — в
|
||||
> `cortiq.selected_model` и в заголовке `X-Cortiq-Selected-Model`.
|
||||
|
||||
> Необязательные поля не сериализуются вместо того, чтобы приходить как `null`: если
|
||||
> tool-вызовов нет, ключа `tool_calls` в ответе нет вообще. `"tool_calls": null` не входит
|
||||
> в схему OpenAI и ломает строгие клиенты.
|
||||
|
||||
> Поле `cortiq` — необязательное расширение. OpenAI-SDK его игнорируют. Если клиент
|
||||
> не хочет «лишних» полей — выключите `cortiq.echo`, метаданные останутся в заголовках.
|
||||
|
||||
Стриминг (`stream: true`) отдаёт строгие события `chat.completion.chunk`: поток всегда
|
||||
открывается дельтой с `role`, содержит ровно один финальный `finish_reason` и завершается
|
||||
`data: [DONE]` — даже если upstream оборвался или упал на середине (тогда ошибка приходит
|
||||
SSE-событием `error`). Клиент никогда не остаётся ждать недосказанный поток.
|
||||
|
||||
### Tool calls
|
||||
|
||||
`tools` и `tool_calls` проксируются, но провод держится чистым:
|
||||
|
||||
* сырая разметка `<tool_call>…</tool_call>` (так Qwen-подобные шаблоны выражают вызов
|
||||
прямо в тексте) вырезается из `content` — клиент никогда не получает один и тот же
|
||||
вызов дважды, прозой и структурой. Маркер, разорванный между SSE-чанками, обрабатывается
|
||||
корректно; обычный текст с символом `<` не трогается;
|
||||
* если апстрим эту разметку не распарсил, она повышается до настоящего элемента
|
||||
`tool_calls` со сгенерированным `call_…` id — вызов становится исполнимым, а не
|
||||
декоративным;
|
||||
* в стриме у каждого элемента всегда есть `index`, `id`/`type` — ровно один раз на вызов,
|
||||
а `function.arguments` — всегда строка, даже если рантайм прислал объект;
|
||||
* ответ с tool-вызовами всегда завершается `finish_reason: "tool_calls"`.
|
||||
|
||||
### Метаданные в заголовках (всегда)
|
||||
|
||||
```
|
||||
@@ -163,20 +208,27 @@ data: [DONE]
|
||||
## 4. OpenAI Models
|
||||
|
||||
`GET /v1/models` — автодискавери для клиентов. Возвращает виртуальные модели шлюза
|
||||
(`cortiq-auto`, `cortiq-embed`) и все реальные модели пула.
|
||||
(`cortiq-auto`, `cortiq-embed`), все модели пула `[[models]]` и все управляемые локальные
|
||||
модели `[[cmf.servers]]` — чтобы клиент мог напрямую закрепить локальный `.cmf`.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{ "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "local-qwen", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
|
||||
{ "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
|
||||
{ "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
|
||||
{ "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
|
||||
{ "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
|
||||
{ "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
|
||||
"cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Управляемые CMF-модели попадают в список сразу, как только настроены; их живое состояние —
|
||||
в нестандартном поле `cortiq` (клиенты его игнорируют). По здоровью они не фильтруются:
|
||||
крупный `.cmf` грузится минутами, и модель, исчезающая из списка на время прогрева, хуже
|
||||
модели, помеченной «ещё не готова».
|
||||
|
||||
---
|
||||
|
||||
## 5. Anthropic Messages
|
||||
|
||||
+5
-5
@@ -41,16 +41,16 @@ minted and shown to you.
|
||||
|
||||
```toml
|
||||
[router]
|
||||
url = "https://138.226.222.209" # allaigate routing API
|
||||
verify_tls = false # cert is bound to an IP
|
||||
url = "https://router.allaigate.com" # allaigate routing API (the default)
|
||||
verify_tls = true # valid certificate — keep verification on
|
||||
taxonomy_id = "data-assistant"
|
||||
api_key_env = "CORTIQ_ROUTER_KEY" # your cortiq_… key
|
||||
timeout_ms = 15000 # allow for oracle escalation on hard prompts
|
||||
api_key_env = "CORTIQ_ROUTER_KEY" # your cortiq_… key
|
||||
timeout_ms = 15000 # allow for oracle escalation on hard prompts
|
||||
```
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <path> only if you keep the config elsewhere
|
||||
```
|
||||
|
||||
## API contract
|
||||
|
||||
+5
-5
@@ -40,16 +40,16 @@ Cortiq Gateway — тонкий и быстрый прокси; **интелле
|
||||
|
||||
```toml
|
||||
[router]
|
||||
url = "https://138.226.222.209" # routing API allaigate
|
||||
verify_tls = false # серт привязан к IP
|
||||
url = "https://router.allaigate.com" # routing API allaigate (значение по умолчанию)
|
||||
verify_tls = true # валидный сертификат — проверку не выключаем
|
||||
taxonomy_id = "data-assistant"
|
||||
api_key_env = "CORTIQ_ROUTER_KEY" # ваш ключ cortiq_…
|
||||
timeout_ms = 15000 # запас под oracle-эскалацию на сложных запросах
|
||||
api_key_env = "CORTIQ_ROUTER_KEY" # ваш ключ cortiq_…
|
||||
timeout_ms = 15000 # запас под oracle-эскалацию на сложных запросах
|
||||
```
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <путь> нужен, только если конфиг лежит в другом месте
|
||||
```
|
||||
|
||||
## Контракт API
|
||||
|
||||
@@ -162,3 +162,44 @@ fine-tune the task mask. Over time this makes routing fit your specific pool mor
|
||||
enabled = false
|
||||
# the ground-truth source is set by the integrator (e.g. the X-Cortiq-Correct-Label header)
|
||||
```
|
||||
|
||||
## Task-based routing
|
||||
|
||||
The router classifies every request twice: a **task label** (what kind of work it
|
||||
is) and a **complexity tier** (how hard it is). Tiers pick *how much model* you
|
||||
need; task rules pick *which* model is best at the job:
|
||||
|
||||
```toml
|
||||
[routing.tasks]
|
||||
translation = ["local-qwen"] # translations are cheap — keep them local
|
||||
code = ["claude-sonnet", "gpt"] # code goes to the strongest coder first
|
||||
```
|
||||
|
||||
A matching rule's models go **first** in the candidate chain; the complexity
|
||||
chain follows as fallback, so a task pin never weakens failover. Tasks without
|
||||
a rule route by complexity alone.
|
||||
|
||||
**Per-complexity models for a task.** A task rule also accepts a per-tier
|
||||
map — "which model serves this task at each complexity":
|
||||
|
||||
```toml
|
||||
[routing.tasks.code]
|
||||
low = ["cheap-coder"] # routine code stays cheap
|
||||
high = ["best-coder"] # hard code goes to the strongest coder
|
||||
any = ["generic-coder"] # leads at every tier after the tier-specific pick
|
||||
```
|
||||
|
||||
The tier-specific list leads its own tier; `any` (and the legacy plain-array
|
||||
form) leads at every tier; the global tier chain and `default` always remain
|
||||
as fallback, so a task rule never weakens failover. The admin console's
|
||||
Routing page edits this as rows of “model + complexity” pairs per task.
|
||||
|
||||
**Exception — high complexity escalates.** When the router rates a request
|
||||
`high` (and a `high` chain is configured), the tier chain leads and the task
|
||||
rule demotes to fallback. Otherwise a rule like `code → cheap-model` would pin
|
||||
even the hardest prompts to the cheap model and the tier system would never
|
||||
get a say. Low/medium keep task-rule-first semantics. The available labels come from the router
|
||||
(`GET /v1/taxonomies`) — the admin console's Routing page builds the editor
|
||||
from your account's live taxonomy and shows each task's share of recent
|
||||
traffic. When a rule supplied the head of the chain the response reports
|
||||
`route_source: "router+task-rule"`.
|
||||
|
||||
@@ -160,3 +160,43 @@ cooldown = "30s"
|
||||
enabled = false
|
||||
# источник истины задаёт интегратор (например, заголовок X-Cortiq-Correct-Label)
|
||||
```
|
||||
|
||||
## Распределение по задачам
|
||||
|
||||
Роутер классифицирует каждый запрос дважды: **тип задачи** (что это за работа)
|
||||
и **уровень сложности** (насколько она трудна). Уровни выбирают, *сколько
|
||||
модели* нужно; правила по задачам — *какая* модель лучше в этом деле:
|
||||
|
||||
```toml
|
||||
[routing.tasks]
|
||||
translation = ["local-qwen"] # переводы дёшевы — оставляем локально
|
||||
code = ["claude-sonnet", "gpt"] # код — сильнейшему кодеру
|
||||
```
|
||||
|
||||
Модели подходящего правила идут **первыми** в цепочке кандидатов; цепочка по
|
||||
сложности следует за ними как резерв — правило по задаче не ослабляет failover.
|
||||
Задачи без правила маршрутизируются только по сложности.
|
||||
|
||||
**Модели по сложности внутри задачи.** Правило задачи принимает и карту по
|
||||
уровням — «какая модель обслуживает задачу на какой сложности»:
|
||||
|
||||
```toml
|
||||
[routing.tasks.code]
|
||||
low = ["cheap-coder"] # рутинный код — дёшево
|
||||
high = ["best-coder"] # сложный код — сильнейшему кодеру
|
||||
any = ["generic-coder"] # ведёт на всех уровнях после уровневого выбора
|
||||
```
|
||||
|
||||
Список конкретного уровня ведёт на своём уровне; `any` (и старая форма-массив)
|
||||
ведёт на всех; глобальная цепочка уровня и `default` всегда остаются резервом —
|
||||
правило задачи не ослабляет failover. На вкладке Роутинг это редактируется
|
||||
строками пар «модель + сложность» у каждой задачи.
|
||||
|
||||
**Исключение — высокая сложность эскалирует.** Когда роутер оценил запрос как
|
||||
`high` (и цепочка `high` настроена), цепочка уровня идёт первой, а правило
|
||||
задачи становится резервом. Иначе правило вида `code → дешёвая-модель`
|
||||
приковало бы к дешёвой модели даже самые сложные запросы, и система уровней
|
||||
не имела бы права голоса. Для low/medium правило задачи ведёт, как и раньше. Список меток приходит
|
||||
с роутера (`GET /v1/taxonomies`) — вкладка Роутинг строит редактор по живой
|
||||
таксономии аккаунта и показывает долю каждой задачи в трафике. Когда цепочку
|
||||
возглавило правило задачи, ответ сообщает `route_source: "router+task-rule"`.
|
||||
|
||||
+1185
-44
File diff suppressed because it is too large
Load Diff
+225
@@ -0,0 +1,225 @@
|
||||
//! One-click local model benchmark: run `cortiq bench <model> --json` and keep
|
||||
//! the parsed result per model, so the Models page can show real tok/s numbers
|
||||
//! measured on this machine.
|
||||
|
||||
use serde::Serialize;
|
||||
use std::collections::HashMap;
|
||||
use std::process::Stdio;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[derive(Clone, Serialize)]
|
||||
pub struct BenchRun {
|
||||
pub model: String, // file stem under models_dir
|
||||
pub state: String, // running | done | error
|
||||
pub log: Vec<String>,
|
||||
pub tok_s: Option<f64>,
|
||||
/// The full JSON object the benchmark printed, passed through untouched.
|
||||
pub raw: Option<serde_json::Value>,
|
||||
pub started: u64,
|
||||
pub finished: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct BenchStore {
|
||||
runs: Mutex<HashMap<String, BenchRun>>,
|
||||
busy: Mutex<bool>,
|
||||
}
|
||||
|
||||
fn now() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// tok/s out of the benchmark's JSON: known field names first (steady decode
|
||||
/// speed is the honest number), then any `*tok_s*` field as a fallback. The
|
||||
/// old loose "contains tok + per" heuristic could grab `allocs_per_token`.
|
||||
fn extract_tok_s(v: &serde_json::Value) -> Option<f64> {
|
||||
let obj = v.as_object()?;
|
||||
for want in [
|
||||
"decode_tok_s_steady",
|
||||
"decode_tok_s_incl_prefill",
|
||||
"decode_tok_s",
|
||||
"tok_s",
|
||||
"tok_per_s",
|
||||
"tokens_per_sec",
|
||||
] {
|
||||
if let Some(n) = obj.get(want).and_then(|x| x.as_f64()) {
|
||||
return Some(n);
|
||||
}
|
||||
}
|
||||
obj.iter()
|
||||
.find(|(k, val)| {
|
||||
let k = k.to_ascii_lowercase();
|
||||
(k.contains("tok_s") || k.contains("tokens_per")) && val.as_f64().is_some()
|
||||
})
|
||||
.and_then(|(_, val)| val.as_f64())
|
||||
}
|
||||
|
||||
/// Last parseable JSON object in mixed CLI output. Newer cortiq releases
|
||||
/// pretty-print the `--json` result over many lines, older ones emit a single
|
||||
/// line — this handles both, skipping any log noise around it.
|
||||
fn last_json_object(text: &str) -> Option<serde_json::Value> {
|
||||
let mut end = text.len();
|
||||
while let Some(close) = text[..end].rfind('}') {
|
||||
let mut depth = 0i32;
|
||||
let mut start = None;
|
||||
for (i, c) in text[..=close].char_indices().rev() {
|
||||
match c {
|
||||
'}' => depth += 1,
|
||||
'{' => {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
start = Some(i);
|
||||
break;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if let Some(st) = start {
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(&text[st..=close]) {
|
||||
if v.is_object() {
|
||||
return Some(v);
|
||||
}
|
||||
}
|
||||
}
|
||||
end = close;
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
impl BenchStore {
|
||||
pub fn new() -> Arc<Self> {
|
||||
Arc::new(Self::default())
|
||||
}
|
||||
pub fn get(&self, model: &str) -> Option<BenchRun> {
|
||||
self.runs.lock().unwrap().get(model).cloned()
|
||||
}
|
||||
pub fn list(&self) -> Vec<BenchRun> {
|
||||
self.runs.lock().unwrap().values().cloned().collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// Kick off a benchmark for `<models_dir>/<model>.cmf`. One at a time — a
|
||||
/// benchmark loads the whole model and owns the machine while it runs.
|
||||
pub fn start(
|
||||
store: Arc<BenchStore>,
|
||||
cmf: &crate::config::CmfCfg,
|
||||
model: String,
|
||||
) -> Result<(), String> {
|
||||
let path = std::path::Path::new(&cmf.models_dir).join(format!("{model}.cmf"));
|
||||
if !path.exists() {
|
||||
return Err(format!("model file not found: {}", path.display()));
|
||||
}
|
||||
{
|
||||
let mut busy = store.busy.lock().unwrap();
|
||||
if *busy {
|
||||
return Err("a benchmark is already running — wait for it to finish".into());
|
||||
}
|
||||
*busy = true;
|
||||
}
|
||||
store.runs.lock().unwrap().insert(
|
||||
model.clone(),
|
||||
BenchRun {
|
||||
model: model.clone(),
|
||||
state: "running".into(),
|
||||
log: Vec::new(),
|
||||
tok_s: None,
|
||||
raw: None,
|
||||
started: now(),
|
||||
finished: None,
|
||||
},
|
||||
);
|
||||
let bin = cmf.cortiq_bin.clone();
|
||||
let gpu = cmf.gpu;
|
||||
tokio::spawn(async move {
|
||||
let out = tokio::process::Command::new(&bin)
|
||||
.arg("bench")
|
||||
.arg(&path)
|
||||
.arg("--json")
|
||||
.env("CMF_GPU", if gpu { "1" } else { "0" })
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.output()
|
||||
.await;
|
||||
let mut g = store.runs.lock().unwrap();
|
||||
let run = g.get_mut(&model).unwrap();
|
||||
run.finished = Some(now());
|
||||
match out {
|
||||
Ok(o) => {
|
||||
let text = format!(
|
||||
"{}\n{}",
|
||||
String::from_utf8_lossy(&o.stdout),
|
||||
String::from_utf8_lossy(&o.stderr)
|
||||
);
|
||||
// keep a readable tail; the JSON line is parsed separately
|
||||
run.log = text
|
||||
.lines()
|
||||
.map(|l| l.trim())
|
||||
.filter(|l| !l.is_empty() && !l.contains("Metal GPU path"))
|
||||
.map(String::from)
|
||||
.collect::<Vec<_>>();
|
||||
let n = run.log.len();
|
||||
if n > 30 {
|
||||
run.log.drain(0..n - 30);
|
||||
}
|
||||
let parsed = last_json_object(&text);
|
||||
match parsed {
|
||||
Some(v) if o.status.success() => {
|
||||
run.tok_s = extract_tok_s(&v);
|
||||
run.raw = Some(v);
|
||||
run.state = "done".into();
|
||||
}
|
||||
_ => run.state = "error".into(),
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
run.log.push(format!("✗ failed to start cortiq bench: {e}"));
|
||||
run.state = "error".into();
|
||||
}
|
||||
}
|
||||
drop(g);
|
||||
*store.busy.lock().unwrap() = false;
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn tok_s_from_common_keys() {
|
||||
let v = serde_json::json!({"tok_s": 42.5, "allocs_per_token": 0});
|
||||
assert_eq!(extract_tok_s(&v), Some(42.5));
|
||||
let v = serde_json::json!({"decode_tokens_per_sec": 31.0});
|
||||
assert_eq!(extract_tok_s(&v), Some(31.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn steady_decode_wins_and_allocs_never_matches() {
|
||||
let v = serde_json::json!({
|
||||
"allocs_per_token": 769.4,
|
||||
"decode_tok_s_incl_prefill": 9.98,
|
||||
"decode_tok_s_steady": 11.86,
|
||||
"prefill_tok_s": 25.5,
|
||||
});
|
||||
assert_eq!(extract_tok_s(&v), Some(11.86));
|
||||
let v = serde_json::json!({"allocs_per_token": 769.4, "seq_len": 140});
|
||||
assert_eq!(extract_tok_s(&v), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_found_single_line_and_pretty() {
|
||||
let single = "noise\n{\"tok_s\": 5.0}\nmore noise";
|
||||
assert_eq!(last_json_object(single).unwrap()["tok_s"], 5.0);
|
||||
let pretty = "INFO loading\n{\n \"decode_tok_s_steady\": 11.86,\n \"seq_len\": 140\n}\nINFO done {not json";
|
||||
assert_eq!(
|
||||
last_json_object(pretty).unwrap()["decode_tok_s_steady"],
|
||||
11.86
|
||||
);
|
||||
assert!(last_json_object("no braces here").is_none());
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -55,7 +55,7 @@ impl SemanticCache {
|
||||
pub fn new(cfg: &CacheCfg) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
enabled: cfg.enabled,
|
||||
threshold: cfg.threshold,
|
||||
threshold: cfg.threshold as f32,
|
||||
ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
|
||||
max_entries: cfg.max_entries.max(1),
|
||||
embed_model: cfg.embed_model.clone(),
|
||||
|
||||
+63
-11
@@ -56,6 +56,9 @@ struct Managed {
|
||||
pub struct CmfRuntime {
|
||||
status: Mutex<CmfStatus>,
|
||||
children: Mutex<Vec<Managed>>,
|
||||
/// Server ids whose healthz handshake advertised tool support — lets the
|
||||
/// pipeline route tools traffic to local models with no manual caps.
|
||||
pub detected_tools: Mutex<std::collections::HashSet<String>>,
|
||||
}
|
||||
|
||||
impl CmfRuntime {
|
||||
@@ -63,6 +66,7 @@ impl CmfRuntime {
|
||||
Arc::new(Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -193,18 +197,19 @@ impl Default for CmfRuntime {
|
||||
Self {
|
||||
status: Mutex::new(CmfStatus::default()),
|
||||
children: Mutex::new(Vec::new()),
|
||||
detected_tools: Mutex::new(std::collections::HashSet::new()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// crates.io `newest_version` for `cortiq-cli`.
|
||||
async fn latest_crates_version() -> Option<String> {
|
||||
/// crates.io `newest_version` for any crate.
|
||||
async fn latest_version_of(krate: &str) -> Option<String> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(Duration::from_secs(10))
|
||||
.build()
|
||||
.ok()?;
|
||||
let body = client
|
||||
.get("https://crates.io/api/v1/crates/cortiq-cli")
|
||||
.get(format!("https://crates.io/api/v1/crates/{krate}"))
|
||||
.header("User-Agent", "cortiq-gateway")
|
||||
.send()
|
||||
.await
|
||||
@@ -216,6 +221,16 @@ async fn latest_crates_version() -> Option<String> {
|
||||
v["crate"]["newest_version"].as_str().map(|s| s.to_string())
|
||||
}
|
||||
|
||||
/// crates.io `newest_version` for `cortiq-cli`.
|
||||
async fn latest_crates_version() -> Option<String> {
|
||||
latest_version_of("cortiq-cli").await
|
||||
}
|
||||
|
||||
/// Newest published gateway version — the "update available" footer check.
|
||||
pub(crate) async fn latest_gateway_version() -> Option<String> {
|
||||
latest_version_of("cortiq-gateway").await
|
||||
}
|
||||
|
||||
/// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2").
|
||||
pub(crate) fn installed_version(bin: &str) -> Option<String> {
|
||||
let out = std::process::Command::new(bin)
|
||||
@@ -238,7 +253,7 @@ fn parse_ver(s: &str) -> (u32, u32, u32) {
|
||||
)
|
||||
}
|
||||
|
||||
fn version_lt(a: &str, b: &str) -> bool {
|
||||
pub(crate) fn version_lt(a: &str, b: &str) -> bool {
|
||||
parse_ver(a) < parse_ver(b)
|
||||
}
|
||||
|
||||
@@ -290,11 +305,31 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
if server.threads > 1 {
|
||||
cmd.env("CMF_THREADS", server.threads.to_string());
|
||||
}
|
||||
// Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
|
||||
// on unified memory, so mostly a prefill win; opt-in.
|
||||
if server.gpu {
|
||||
cmd.env("CMF_GPU", "1");
|
||||
}
|
||||
// O(1) Nyström attention: layers spec + sub-parameters.
|
||||
if let Some(o1) = &server.o1 {
|
||||
let s = o1.trim();
|
||||
if !s.is_empty() {
|
||||
// Pass even "off" so it explicitly overrides a file converter hint.
|
||||
cmd.arg("--o1").arg(s);
|
||||
}
|
||||
}
|
||||
if let Some(m) = server.o1_m {
|
||||
cmd.arg("--o1-m").arg(m.to_string());
|
||||
}
|
||||
if let Some(w) = server.o1_window {
|
||||
cmd.arg("--o1-window").arg(w.to_string());
|
||||
}
|
||||
if let Some(sink) = server.o1_sink {
|
||||
cmd.arg("--o1-sink").arg(sink.to_string());
|
||||
}
|
||||
// MTP (Multi-Token Prediction) speculative decoding: the engine reads
|
||||
// CMF_MTP env; "0" disables. There is no CLI flag for this.
|
||||
if server.skip_mtp {
|
||||
cmd.env("CMF_MTP", "0");
|
||||
}
|
||||
cmd.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.kill_on_drop(true)
|
||||
@@ -302,8 +337,11 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
/// Poll `/healthz` until the local server answers 200 (or the deadline
|
||||
/// passes). Newer `cortiq serve` advertises the loaded model's capabilities
|
||||
/// in the healthz body ({"capabilities":{"tools":true}}) — returned so the
|
||||
/// caller can register them; older servers return `None` there.
|
||||
async fn wait_healthy(host: &str, port: u16, tries: u32) -> (bool, Option<bool>) {
|
||||
let url = format!("http://{host}:{port}/healthz");
|
||||
let client = reqwest::Client::new();
|
||||
for _ in 0..tries {
|
||||
@@ -314,12 +352,17 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
.await
|
||||
{
|
||||
if r.status().is_success() {
|
||||
return true;
|
||||
let tools = r
|
||||
.json::<serde_json::Value>()
|
||||
.await
|
||||
.ok()
|
||||
.and_then(|v| v["capabilities"]["tools"].as_bool());
|
||||
return (true, tools);
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
false
|
||||
(false, None)
|
||||
}
|
||||
|
||||
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
||||
@@ -402,8 +445,17 @@ pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
|
||||
if !rt.is_running(&server.id) {
|
||||
continue; // failed to spawn / file missing
|
||||
}
|
||||
if wait_healthy(&cfg.local_host, server.port, 240).await {
|
||||
let (ok, tools) = wait_healthy(&cfg.local_host, server.port, 240).await;
|
||||
if ok {
|
||||
rt.set_healthy(&server.id, true);
|
||||
// capability handshake: the server said whether its model's chat
|
||||
// template can drive tool calling — no manual caps needed
|
||||
if tools == Some(true) {
|
||||
rt.detected_tools.lock().unwrap().insert(server.id.clone());
|
||||
tracing::info!(id = %server.id, "local server advertises tool support");
|
||||
} else {
|
||||
rt.detected_tools.lock().unwrap().remove(&server.id);
|
||||
}
|
||||
rt.log(format!(
|
||||
"local CMF server ready — '{}' at http://{}:{}/v1",
|
||||
server.id, cfg.local_host, server.port
|
||||
|
||||
+431
-7
@@ -9,6 +9,9 @@ use std::path::PathBuf;
|
||||
/// `$CORTIQ_GATEWAY_HOME`, then `$XDG_DATA_HOME/cortiq-gateway`, then
|
||||
/// `~/.cortiq-gateway` — so defaults work regardless of the process CWD
|
||||
/// (important once the gateway is installed rather than run from its repo).
|
||||
/// On Windows also checks `%APPDATA%`, `%LOCALAPPDATA%` and `%USERPROFILE%`
|
||||
/// so double-clicking the exe from `Downloads` does not try to write into
|
||||
/// `Program Files`.
|
||||
pub fn data_dir() -> PathBuf {
|
||||
for (var, sub) in [
|
||||
("CORTIQ_GATEWAY_HOME", ""),
|
||||
@@ -25,6 +28,20 @@ pub fn data_dir() -> PathBuf {
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(windows)]
|
||||
{
|
||||
for (var, sub) in [
|
||||
("APPDATA", "cortiq-gateway"),
|
||||
("LOCALAPPDATA", "cortiq-gateway"),
|
||||
("USERPROFILE", ".cortiq-gateway"),
|
||||
] {
|
||||
if let Ok(v) = std::env::var(var) {
|
||||
if !v.is_empty() {
|
||||
return PathBuf::from(v).join(sub);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
PathBuf::from("data")
|
||||
}
|
||||
|
||||
@@ -83,6 +100,40 @@ pub struct CmfServer {
|
||||
/// Offload matvecs to the GPU (CMF_GPU=1).
|
||||
#[serde(default)]
|
||||
pub gpu: bool,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) Nyström attention layer spec (all | deepN | i,j,k | off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// O(1) landmark budget (default 32; ≥4).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_m: Option<u32>,
|
||||
/// O(1) exact-window width (default 128).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_window: Option<u32>,
|
||||
/// O(1) permanent exact sink keys (default 4, StreamingLLM discipline).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_sink: Option<u32>,
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
|
||||
/// gateway auto-detects tool support from the server's healthz handshake.
|
||||
#[serde(default)]
|
||||
pub caps: Vec<String>,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -172,6 +223,17 @@ impl CmfCfg {
|
||||
port: self.local_port,
|
||||
threads: self.threads,
|
||||
gpu: self.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
@@ -248,12 +310,42 @@ fn default_router_enabled() -> bool {
|
||||
true
|
||||
}
|
||||
fn default_router_timeout() -> u64 {
|
||||
800
|
||||
// the hosted router escalates hard prompts to an oracle (~10 s);
|
||||
// an 800 ms budget silently degraded every one of those to the fallback
|
||||
15_000
|
||||
}
|
||||
fn default_verify_tls() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
/// The hosted allaigate routing API.
|
||||
pub const DEFAULT_ROUTER_URL: &str = "https://router.allaigate.com";
|
||||
|
||||
/// The bare IP the hosted router used to be addressed by. It served a
|
||||
/// certificate for `router.allaigate.com`, so every config pointing at it had to
|
||||
/// turn TLS verification off. The hostname now resolves, so configs written
|
||||
/// before it existed are moved over on load — the endpoint is the same machine,
|
||||
/// and this is the difference between a verified connection and a blind one.
|
||||
const LEGACY_ROUTER_HOSTS: &[&str] = &[
|
||||
"https://138.226.222.209",
|
||||
"http://138.226.222.209",
|
||||
"138.226.222.209",
|
||||
];
|
||||
|
||||
impl RouterCfg {
|
||||
/// Point a legacy IP-addressed router config at the hostname and restore TLS
|
||||
/// verification. Returns true when something changed.
|
||||
fn migrate_hosted_endpoint(&mut self) -> bool {
|
||||
let url = self.url.trim().trim_end_matches('/');
|
||||
if !LEGACY_ROUTER_HOSTS.contains(&url) {
|
||||
return false;
|
||||
}
|
||||
self.url = DEFAULT_ROUTER_URL.to_string();
|
||||
self.verify_tls = true;
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct RouteCfg {
|
||||
#[serde(default = "default_text_strategy")]
|
||||
@@ -306,6 +398,27 @@ pub struct ModelCfg {
|
||||
pub api_key_env: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub caps: Vec<String>,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) constant-memory streaming attention hint (all, deep, custom, off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// Skip Multi-Token Prediction (MTP) heads.
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
fn default_kind() -> String {
|
||||
"chat".into()
|
||||
@@ -318,17 +431,149 @@ pub struct RoutingCfg {
|
||||
pub default: String,
|
||||
#[serde(default)]
|
||||
pub policy: RoutingPolicy,
|
||||
/// Per-task rules: `task_label → ordered model list`. A matching rule's
|
||||
/// models go FIRST; the complexity-tier chain follows as fallback, so a
|
||||
/// task pin never weakens failover. Named field, so a tier cannot be
|
||||
/// called "tasks" — serde extracts it before the flattened tier map.
|
||||
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
|
||||
pub tasks: HashMap<String, TierTargets>,
|
||||
/// Local complexity bands: re-map the router's continuous score into
|
||||
/// low/medium/high with YOUR OWN boundaries. The hosted router's bands
|
||||
/// are global for all customers; these are per-gateway. `None` = keep
|
||||
/// the router's tier verdict as delivered.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub bands: Option<TierBands>,
|
||||
}
|
||||
|
||||
/// Score boundaries: `score ≤ low` → low, `≤ medium` → medium, else high.
|
||||
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
|
||||
pub struct TierBands {
|
||||
pub low: f32,
|
||||
pub medium: f32,
|
||||
}
|
||||
|
||||
impl TierBands {
|
||||
pub fn tier_of(&self, score: f32) -> &'static str {
|
||||
if score <= self.low {
|
||||
"low"
|
||||
} else if score <= self.medium {
|
||||
"medium"
|
||||
} else {
|
||||
"high"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A routing tier is either a list of model_ids or the string `default`.
|
||||
/// (`default` is handled by a separate field; only tier lists appear here.)
|
||||
///
|
||||
/// TASK rules additionally accept a per-complexity map — "which model serves
|
||||
/// this task at each tier":
|
||||
///
|
||||
/// ```toml
|
||||
/// [routing.tasks.code]
|
||||
/// low = ["cheap-coder"]
|
||||
/// high = ["big-model"]
|
||||
/// any = ["fallback"] # leads at every tier after the tier-specific list
|
||||
/// ```
|
||||
///
|
||||
/// The plain-array form stays valid and means "leads at any tier". The
|
||||
/// transitional `{ models = [...], tier = "high" }` form is normalized into
|
||||
/// the map on load. Untagged: map is tried first, arrays keep parsing as is.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum TierTargets {
|
||||
Map(HashMap<String, TaskTierVal>),
|
||||
List(Vec<String>),
|
||||
One(String),
|
||||
}
|
||||
|
||||
/// A value inside a task's per-tier map: a model list, a single model, or the
|
||||
/// transitional string value of the legacy `tier = "high"` key.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum TaskTierVal {
|
||||
List(Vec<String>),
|
||||
Str(String),
|
||||
}
|
||||
|
||||
impl TaskTierVal {
|
||||
fn list(&self) -> Vec<String> {
|
||||
match self {
|
||||
TaskTierVal::List(v) => v.clone(),
|
||||
TaskTierVal::Str(s) => vec![s.clone()],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub const TASK_TIER_KEYS: [&str; 4] = ["any", "low", "medium", "high"];
|
||||
|
||||
impl TierTargets {
|
||||
/// Every referenced model id, whatever the syntactic form (validation).
|
||||
pub fn models(&self) -> Vec<String> {
|
||||
match self {
|
||||
TierTargets::Map(m) => {
|
||||
let mut out = Vec::new();
|
||||
for (k, v) in m {
|
||||
if k == "tier" {
|
||||
continue; // legacy marker, not a model list
|
||||
}
|
||||
out.extend(v.list());
|
||||
}
|
||||
out
|
||||
}
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
}
|
||||
}
|
||||
|
||||
/// Normalized task rule: `(any_tier_models, per_tier_models)`.
|
||||
/// Legacy `{models, tier}` becomes `per[tier] = models`; a plain array
|
||||
/// becomes `any`.
|
||||
pub fn task_rule(&self) -> (Vec<String>, HashMap<String, Vec<String>>) {
|
||||
match self {
|
||||
TierTargets::Map(m) => {
|
||||
let mut any: Vec<String> = Vec::new();
|
||||
let mut per: HashMap<String, Vec<String>> = HashMap::new();
|
||||
let legacy_tier = m.get("tier").and_then(|v| match v {
|
||||
TaskTierVal::Str(s) => Some(s.clone()),
|
||||
_ => None,
|
||||
});
|
||||
for (k, v) in m {
|
||||
match k.as_str() {
|
||||
"tier" => {}
|
||||
"any" | "models" => any.extend(v.list()),
|
||||
"low" | "medium" | "high" => {
|
||||
per.entry(k.clone()).or_default().extend(v.list())
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if let Some(tier) = legacy_tier {
|
||||
// legacy pin: the models were meant for that tier
|
||||
let moved = std::mem::take(&mut any);
|
||||
per.entry(tier).or_default().extend(moved);
|
||||
}
|
||||
(any, per)
|
||||
}
|
||||
TierTargets::List(v) => (v.clone(), HashMap::new()),
|
||||
TierTargets::One(s) => (vec![s.clone()], HashMap::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Unknown keys inside a task map (validation).
|
||||
pub fn invalid_task_keys(&self) -> Vec<String> {
|
||||
match self {
|
||||
TierTargets::Map(m) => m
|
||||
.keys()
|
||||
.filter(|k| !TASK_TIER_KEYS.contains(&k.as_str()) && *k != "models" && *k != "tier")
|
||||
.cloned()
|
||||
.collect(),
|
||||
_ => Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct RoutingPolicy {
|
||||
#[serde(default = "default_mode")]
|
||||
@@ -408,6 +653,9 @@ pub struct ApiKeyCfg {
|
||||
pub account: String,
|
||||
#[serde(default)]
|
||||
pub rate_per_min: u32,
|
||||
/// Monthly spend ceiling in USD for this key's account; 0/absent = none.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub monthly_budget_usd: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub allow_models: Vec<String>,
|
||||
}
|
||||
@@ -461,10 +709,12 @@ pub struct AdminCfg {
|
||||
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
|
||||
#[serde(default = "default_admin_enabled")]
|
||||
pub enabled: bool,
|
||||
/// Name of the env variable holding the admin token. If unset (or empty),
|
||||
/// a token is generated at startup and printed to the log.
|
||||
/// Name of the env variable holding the admin token.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token_env: Option<String>,
|
||||
/// Persistent admin token configured directly in toml.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token: Option<String>,
|
||||
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
|
||||
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
@@ -475,6 +725,7 @@ impl Default for AdminCfg {
|
||||
Self {
|
||||
enabled: default_admin_enabled(),
|
||||
token_env: None,
|
||||
token: None,
|
||||
listen: None,
|
||||
}
|
||||
}
|
||||
@@ -498,6 +749,12 @@ pub struct StatsCfg {
|
||||
/// Retention window for aggregates/time-series replayed on startup, e.g. `7d`.
|
||||
#[serde(default = "default_stats_retention")]
|
||||
pub retention: String,
|
||||
/// Compact the JSONL file once it exceeds this size: records older than
|
||||
/// `retention` fold into cumulative per-account totals (a small sidecar
|
||||
/// file) and are removed, so per-key accounting survives while the disk
|
||||
/// stays bounded. 0 disables compaction.
|
||||
#[serde(default = "default_stats_max_file_mb")]
|
||||
pub max_file_mb: u64,
|
||||
}
|
||||
impl Default for StatsCfg {
|
||||
fn default() -> Self {
|
||||
@@ -506,6 +763,7 @@ impl Default for StatsCfg {
|
||||
file: default_stats_file(),
|
||||
ring_size: default_stats_ring(),
|
||||
retention: default_stats_retention(),
|
||||
max_file_mb: default_stats_max_file_mb(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -521,6 +779,9 @@ fn default_stats_ring() -> usize {
|
||||
fn default_stats_retention() -> String {
|
||||
"7d".into()
|
||||
}
|
||||
fn default_stats_max_file_mb() -> u64 {
|
||||
16
|
||||
}
|
||||
|
||||
/// Self-warming shadow loop (docs/SELF_WARMING_GATEWAY.md). While a task
|
||||
/// label is not yet promoted, the client is served the CLOUD answer; in
|
||||
@@ -604,9 +865,10 @@ fn default_shadow_soak() -> usize {
|
||||
pub struct CacheCfg {
|
||||
#[serde(default)]
|
||||
pub enabled: bool,
|
||||
/// Cosine-similarity threshold for a hit (0..1).
|
||||
/// Cosine-similarity threshold for a hit (0..1). f64 so the TOML holds
|
||||
/// the number you typed (an f32 round-trips as 0.9200000166893005).
|
||||
#[serde(default = "default_semcache_threshold")]
|
||||
pub threshold: f32,
|
||||
pub threshold: f64,
|
||||
/// Entry time-to-live, e.g. `1h`.
|
||||
#[serde(default = "default_semcache_ttl")]
|
||||
pub ttl: String,
|
||||
@@ -628,7 +890,7 @@ impl Default for CacheCfg {
|
||||
}
|
||||
}
|
||||
}
|
||||
fn default_semcache_threshold() -> f32 {
|
||||
fn default_semcache_threshold() -> f64 {
|
||||
0.92
|
||||
}
|
||||
fn default_semcache_ttl() -> String {
|
||||
@@ -638,16 +900,116 @@ fn default_semcache_max() -> usize {
|
||||
1000
|
||||
}
|
||||
|
||||
/// Sensible defaults for a fresh install (empty pool — user fills via wizard/UI).
|
||||
impl Default for Config {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
listen: "0.0.0.0:9000".to_string(),
|
||||
// The hosted allaigate router — the thing this gateway is built
|
||||
// around. Without a key the pipeline degrades gracefully to local
|
||||
// models and the console asks for one; nothing breaks.
|
||||
router: RouterCfg {
|
||||
enabled: true,
|
||||
url: DEFAULT_ROUTER_URL.to_string(),
|
||||
timeout_ms: 15_000, // hard prompts escalate to an oracle (~10 s)
|
||||
verify_tls: true,
|
||||
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
|
||||
taxonomy_id: Some("data-assistant".to_string()),
|
||||
},
|
||||
route: RouteCfg::default(),
|
||||
models: Vec::new(),
|
||||
routing: RoutingCfg {
|
||||
tiers: HashMap::new(),
|
||||
default: String::new(),
|
||||
policy: RoutingPolicy::default(),
|
||||
tasks: HashMap::new(),
|
||||
bands: None,
|
||||
},
|
||||
breaker: BreakerCfg::default(),
|
||||
protocols: ProtocolsCfg {
|
||||
openai_chat: true,
|
||||
openai_completions: true,
|
||||
openai_embeddings: true,
|
||||
openai_models: true,
|
||||
// on by default: the API-keys view ships a Claude Code
|
||||
// auto-setup snippet that targets /v1/messages
|
||||
anthropic_messages: true,
|
||||
mcp: false,
|
||||
native_passthrough: true,
|
||||
},
|
||||
cortiq: CortiqCfg::default(),
|
||||
api_keys: Vec::new(),
|
||||
idempotency: IdempotencyCfg::default(),
|
||||
log: LogCfg::default(),
|
||||
telemetry: TelemetryCfg::default(),
|
||||
admin: AdminCfg::default(),
|
||||
stats: StatsCfg::default(),
|
||||
cache: CacheCfg::default(),
|
||||
shadow: ShadowCfg::default(),
|
||||
cmf: CmfCfg::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Where the config lives when `--config` is not given.
|
||||
/// A readable `./config/gateway.toml` (a real gateway checkout / install dir)
|
||||
/// wins; otherwise the per-user home (`$CORTIQ_GATEWAY_HOME` or
|
||||
/// `~/.cortiq-gateway`) — so `cortiq-gateway` typed in any directory picks up
|
||||
/// the same configuration it was set up with. A foreign project that merely
|
||||
/// has `Cargo.toml` and a `config/` folder must NOT capture the config
|
||||
/// (that used to shadow the user's real config with a broken path).
|
||||
pub fn default_config_path() -> String {
|
||||
let repo_path = "config/gateway.toml";
|
||||
// the file itself, readable — not just an entry that exists()
|
||||
if std::fs::read_to_string(repo_path).is_ok() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
// our own repo checkout before first run: config/ exists, file not yet
|
||||
let ours = std::fs::read_to_string("Cargo.toml")
|
||||
.map(|t| t.contains("name = \"cortiq-gateway\""))
|
||||
.unwrap_or(false);
|
||||
if ours && std::path::Path::new("config").is_dir() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
data_dir()
|
||||
.join("gateway.toml")
|
||||
.to_string_lossy()
|
||||
.into_owned()
|
||||
}
|
||||
|
||||
impl Config {
|
||||
/// Load and validate the config from a file.
|
||||
pub fn load(path: &str) -> anyhow::Result<Self> {
|
||||
let raw = std::fs::read_to_string(path)
|
||||
.map_err(|e| anyhow::anyhow!("cannot read config {path}: {e}"))?;
|
||||
let cfg: Config = toml::from_str(&raw)?;
|
||||
let mut cfg: Config = toml::from_str(&raw)?;
|
||||
if cfg.router.migrate_hosted_endpoint() {
|
||||
tracing::info!(
|
||||
url = DEFAULT_ROUTER_URL,
|
||||
"router endpoint moved from the bare IP to the hostname; TLS verification is on"
|
||||
);
|
||||
let _ = cfg.save(path);
|
||||
}
|
||||
cfg.validate()?;
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Load if exists, otherwise create a fresh default config at `path` and return it.
|
||||
/// Never requires the user to create or copy a file by hand.
|
||||
pub fn load_or_create(path: &str) -> anyhow::Result<Self> {
|
||||
if std::path::Path::new(path).exists() {
|
||||
return Self::load(path);
|
||||
}
|
||||
let cfg = Self::default();
|
||||
// ensure parent dir exists
|
||||
if let Some(dir) = std::path::Path::new(path).parent() {
|
||||
let _ = std::fs::create_dir_all(dir);
|
||||
}
|
||||
cfg.save(path)?;
|
||||
eprintln!("created default config at {path} — open /admin to finish setup");
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Integrity validation: non-empty model pool, valid routing targets.
|
||||
pub fn validate(&self) -> anyhow::Result<()> {
|
||||
// A managed local CMF model counts as an available model even though it
|
||||
@@ -691,6 +1053,7 @@ impl Config {
|
||||
}
|
||||
for (tier, targets) in &self.routing.tiers {
|
||||
let list = match targets {
|
||||
TierTargets::Map(_) => targets.models(),
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
@@ -704,6 +1067,33 @@ impl Config {
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(b) = &self.routing.bands {
|
||||
if !(b.low > 0.0 && b.low < b.medium && b.medium < 1.0) {
|
||||
anyhow::bail!(
|
||||
"config: routing.bands must satisfy 0 < low < medium < 1 (got {} / {})",
|
||||
b.low,
|
||||
b.medium
|
||||
);
|
||||
}
|
||||
}
|
||||
for (task, targets) in &self.routing.tasks {
|
||||
for id in &targets.models() {
|
||||
if !ids.contains(id.as_str()) {
|
||||
anyhow::bail!(
|
||||
"config: routing.tasks.{} references unknown model id '{}'",
|
||||
task,
|
||||
id
|
||||
);
|
||||
}
|
||||
}
|
||||
if let Some(bad) = targets.invalid_task_keys().first() {
|
||||
anyhow::bail!(
|
||||
"config: routing.tasks.{}.{} is not a tier (use any/low/medium/high)",
|
||||
task,
|
||||
bad
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -736,6 +1126,39 @@ impl Config {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn the_hosted_router_defaults_to_the_hostname_with_tls_on() {
|
||||
let cfg = Config::default();
|
||||
assert_eq!(cfg.router.url, "https://router.allaigate.com");
|
||||
assert!(cfg.router.verify_tls);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_legacy_ip_router_config_moves_to_the_hostname() {
|
||||
for legacy in [
|
||||
"https://138.226.222.209",
|
||||
"https://138.226.222.209/",
|
||||
"138.226.222.209",
|
||||
] {
|
||||
let mut r = Config::default().router;
|
||||
r.url = legacy.to_string();
|
||||
r.verify_tls = false;
|
||||
assert!(r.migrate_hosted_endpoint(), "{legacy} must migrate");
|
||||
assert_eq!(r.url, DEFAULT_ROUTER_URL);
|
||||
assert!(r.verify_tls, "TLS verification must be restored");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_self_hosted_router_url_is_left_alone() {
|
||||
let mut r = Config::default().router;
|
||||
r.url = "http://127.0.0.1:8081".to_string();
|
||||
r.verify_tls = false;
|
||||
assert!(!r.migrate_hosted_endpoint());
|
||||
assert_eq!(r.url, "http://127.0.0.1:8081");
|
||||
assert!(!r.verify_tls);
|
||||
}
|
||||
|
||||
const SAMPLE: &str = r#"
|
||||
listen = "0.0.0.0:9000"
|
||||
|
||||
@@ -794,6 +1217,7 @@ min_class = { low = "cheap", medium = "mid" }
|
||||
|
||||
let low = back.routing.tiers.get("low").expect("low tier present");
|
||||
match low {
|
||||
TierTargets::Map(_) => assert_eq!(low.models(), vec!["local-qwen".to_string()]),
|
||||
TierTargets::List(v) => assert_eq!(v, &vec!["local-qwen".to_string()]),
|
||||
TierTargets::One(s) => assert_eq!(s, "local-qwen"),
|
||||
}
|
||||
|
||||
@@ -13,6 +13,8 @@ pub enum GatewayError {
|
||||
Unauthorized(String),
|
||||
#[error("rate limit exceeded")]
|
||||
RateLimited { retry_after_secs: u64 },
|
||||
#[error("monthly budget exhausted (limit ${budget_usd})")]
|
||||
BudgetExhausted { budget_usd: f64 },
|
||||
#[error("upstream unavailable: {0}")]
|
||||
UpstreamUnavailable(String),
|
||||
#[error("internal error: {0}")]
|
||||
@@ -37,6 +39,11 @@ impl GatewayError {
|
||||
"rate_limit_error",
|
||||
"rate_limited",
|
||||
),
|
||||
Self::BudgetExhausted { .. } => (
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"insufficient_quota",
|
||||
"insufficient_quota",
|
||||
),
|
||||
Self::UpstreamUnavailable(_) => (
|
||||
StatusCode::BAD_GATEWAY,
|
||||
"upstream_unavailable",
|
||||
|
||||
+862
-18
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,78 @@
|
||||
//! In-memory ring of recent log lines, so the admin console can show logs to
|
||||
//! users who never see the terminal (Docker, autostart, double-clicked exe).
|
||||
//! Fed by a second `tracing` layer writing plain text (no ANSI) into the ring
|
||||
//! while the normal stdout layer stays colored.
|
||||
|
||||
use std::collections::VecDeque;
|
||||
use std::io;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
|
||||
const CAP: usize = 500;
|
||||
|
||||
fn ring() -> &'static Arc<Mutex<VecDeque<String>>> {
|
||||
static RING: OnceLock<Arc<Mutex<VecDeque<String>>>> = OnceLock::new();
|
||||
RING.get_or_init(|| Arc::new(Mutex::new(VecDeque::with_capacity(CAP))))
|
||||
}
|
||||
|
||||
/// Last `limit` lines, oldest first.
|
||||
pub fn snapshot(limit: usize) -> Vec<String> {
|
||||
let g = ring().lock().unwrap();
|
||||
g.iter()
|
||||
.skip(g.len().saturating_sub(limit))
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Drop everything in the ring (the admin console's "clear" button).
|
||||
pub fn clear() {
|
||||
ring().lock().unwrap().clear();
|
||||
}
|
||||
|
||||
pub fn push_line(line: &str) {
|
||||
let line = line.trim_end();
|
||||
if line.is_empty() {
|
||||
return;
|
||||
}
|
||||
let mut g = ring().lock().unwrap();
|
||||
if g.len() >= CAP {
|
||||
g.pop_front();
|
||||
}
|
||||
g.push_back(line.to_string());
|
||||
}
|
||||
|
||||
/// `MakeWriter` for the ring layer: buffers partial writes until a newline.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct RingMakeWriter;
|
||||
|
||||
pub struct RingWriter {
|
||||
buf: String,
|
||||
}
|
||||
|
||||
impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for RingMakeWriter {
|
||||
type Writer = RingWriter;
|
||||
fn make_writer(&'a self) -> Self::Writer {
|
||||
RingWriter { buf: String::new() }
|
||||
}
|
||||
}
|
||||
|
||||
impl io::Write for RingWriter {
|
||||
fn write(&mut self, data: &[u8]) -> io::Result<usize> {
|
||||
self.buf.push_str(&String::from_utf8_lossy(data));
|
||||
while let Some(idx) = self.buf.find('\n') {
|
||||
let line: String = self.buf.drain(..=idx).collect();
|
||||
push_line(&line);
|
||||
}
|
||||
Ok(data.len())
|
||||
}
|
||||
fn flush(&mut self) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for RingWriter {
|
||||
fn drop(&mut self) {
|
||||
if !self.buf.is_empty() {
|
||||
push_line(&self.buf.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
+264
-27
@@ -9,11 +9,14 @@
|
||||
#![allow(dead_code)]
|
||||
|
||||
mod admin;
|
||||
mod bench;
|
||||
mod cache;
|
||||
mod cmf_runtime;
|
||||
mod config;
|
||||
mod error;
|
||||
mod import;
|
||||
mod logbuf;
|
||||
mod media;
|
||||
mod model;
|
||||
mod pipeline;
|
||||
mod promotion;
|
||||
@@ -25,6 +28,8 @@ mod routing;
|
||||
mod secrets;
|
||||
mod state;
|
||||
mod stats;
|
||||
mod sysmon;
|
||||
mod toolcalls;
|
||||
|
||||
use axum::{routing::get, Router};
|
||||
use clap::Parser;
|
||||
@@ -38,49 +43,206 @@ use state::SharedState;
|
||||
about = "Universal LLM gateway with intelligent routing"
|
||||
)]
|
||||
struct Args {
|
||||
/// Path to the TOML config file.
|
||||
#[arg(long, default_value = "config/gateway.toml")]
|
||||
config: String,
|
||||
/// Path to the TOML config file. If omitted, uses `config/gateway.toml` when running
|
||||
/// inside the repo, otherwise `~/.cortiq-gateway/gateway.toml` (or `$CORTIQ_GATEWAY_HOME`).
|
||||
/// Missing file is created automatically — no manual copying needed.
|
||||
#[arg(long)]
|
||||
config: Option<String>,
|
||||
/// Admin token for the web panel (overrides `[admin].token_env`).
|
||||
#[arg(long)]
|
||||
admin_token: Option<String>,
|
||||
/// Do not automatically open the browser on first run.
|
||||
#[arg(long)]
|
||||
no_browser: bool,
|
||||
/// Do not auto-select a free port if the configured one is busy.
|
||||
#[arg(long)]
|
||||
no_port_search: bool,
|
||||
}
|
||||
|
||||
/// Resolve the admin token: env from `[admin].token_env` → `--admin-token` →
|
||||
/// generate a random one. Returns `(token, generated)`.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
|
||||
///
|
||||
/// By default no token is required (open mode). User sets it explicitly via
|
||||
/// `--admin-token`, `[admin].token` or `[admin].token_env` if they want auth.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> String {
|
||||
if let Some(env) = &cfg.admin.token_env {
|
||||
if let Ok(v) = std::env::var(env) {
|
||||
if !v.is_empty() {
|
||||
return (v, false);
|
||||
return v;
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(t) = &args.admin_token {
|
||||
if !t.is_empty() {
|
||||
return (t.clone(), false);
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
(admin::random_token(24), true)
|
||||
if let Some(t) = &cfg.admin.token {
|
||||
if !t.is_empty() {
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
String::new()
|
||||
}
|
||||
|
||||
fn is_docker() -> bool {
|
||||
std::path::Path::new("/.dockerenv").exists()
|
||||
|| std::fs::read_to_string("/proc/1/cgroup")
|
||||
.map(|s| s.contains("docker") || s.contains("kubepods"))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn should_open_browser(args: &Args) -> bool {
|
||||
if args.no_browser {
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CORTIQ_NO_BROWSER")
|
||||
.map(|v| !v.is_empty() && v != "0")
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CI").is_ok() || std::env::var("NO_BROWSER").is_ok() {
|
||||
return false;
|
||||
}
|
||||
if is_docker() {
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn parse_listen(listen: &str) -> (String, u16) {
|
||||
if let Some((host, port_s)) = listen.rsplit_once(':') {
|
||||
if let Ok(p) = port_s.parse::<u16>() {
|
||||
return (host.to_string(), p);
|
||||
}
|
||||
}
|
||||
(listen.to_string(), 9000)
|
||||
}
|
||||
|
||||
fn is_port_free(host: &str, port: u16) -> bool {
|
||||
let addr = format!("{host}:{port}");
|
||||
// bind test + connect test (docker port forwarding can make bind succeed while port is still forwarded)
|
||||
if std::net::TcpListener::bind(&addr).is_err() {
|
||||
return false;
|
||||
}
|
||||
use std::net::ToSocketAddrs;
|
||||
let probe_host = if host == "0.0.0.0" { "127.0.0.1" } else { host };
|
||||
if let Ok(mut addrs) = (probe_host, port).to_socket_addrs() {
|
||||
if let Some(a) = addrs.next() {
|
||||
if std::net::TcpStream::connect_timeout(&a, std::time::Duration::from_millis(200))
|
||||
.is_ok()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn resolve_listen(cfg: &mut Config, no_search: bool) -> String {
|
||||
if no_search {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
let (host, port) = parse_listen(&cfg.listen);
|
||||
let clean_host = if host.is_empty() {
|
||||
"0.0.0.0".to_string()
|
||||
} else {
|
||||
host
|
||||
};
|
||||
if is_port_free(&clean_host, port) {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
for p in (port + 1)..=port.saturating_add(50) {
|
||||
if is_port_free(&clean_host, p) {
|
||||
let new_listen = format!("{clean_host}:{p}");
|
||||
eprintln!("port {port} busy — switching to free port {p} ({new_listen})");
|
||||
cfg.listen = new_listen.clone();
|
||||
return new_listen;
|
||||
}
|
||||
}
|
||||
if !is_port_free(&clean_host, port) {
|
||||
eprintln!(
|
||||
"warning: port {port} busy and no free port found in next 50, will try to bind anyway"
|
||||
);
|
||||
}
|
||||
cfg.listen.clone()
|
||||
}
|
||||
|
||||
fn setup_marker_path(config_path: &str) -> std::path::PathBuf {
|
||||
// marker lives next to config file: gateway.toml -> .setup_done
|
||||
std::path::Path::new(config_path)
|
||||
.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.join(".setup_done")
|
||||
}
|
||||
|
||||
fn needs_first_run_wizard(cfg: &Config, config_path: &str) -> bool {
|
||||
let marker = setup_marker_path(config_path);
|
||||
if marker.exists() {
|
||||
return false;
|
||||
}
|
||||
// also consider "needs setup" if model pool is empty and no managed cmf
|
||||
if !cfg.models.is_empty() || !cfg.cmf.effective_servers().is_empty() {
|
||||
// still show wizard once if marker missing, but not if user already has models
|
||||
// we treat missing marker + existing models as "already configured" to avoid nagging
|
||||
// persist marker lazily
|
||||
let _ = std::fs::write(&marker, "done");
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let args = Args::parse();
|
||||
let config_path = args
|
||||
.config
|
||||
.clone()
|
||||
.unwrap_or_else(config::default_config_path);
|
||||
|
||||
let cfg = Config::load(&args.config)?;
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| cfg.log.level.clone().into()),
|
||||
)
|
||||
.init();
|
||||
let mut cfg = Config::load_or_create(&config_path)?;
|
||||
// auto-pick a free port if the configured one is busy (unless disabled)
|
||||
let original_listen = cfg.listen.clone();
|
||||
let listen = resolve_listen(&mut cfg, args.no_port_search);
|
||||
// persist the resolved port if it changed
|
||||
if listen != original_listen {
|
||||
let _ = cfg.save(&config_path);
|
||||
}
|
||||
{
|
||||
use tracing_subscriber::layer::SubscriberExt;
|
||||
use tracing_subscriber::util::SubscriberInitExt;
|
||||
let filter = tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| cfg.log.level.clone().into());
|
||||
tracing_subscriber::registry()
|
||||
.with(filter)
|
||||
.with(tracing_subscriber::fmt::layer())
|
||||
// second copy of every line, without ANSI, into the in-memory ring
|
||||
// that backs the admin console's Логи view
|
||||
.with(
|
||||
tracing_subscriber::fmt::layer()
|
||||
.with_ansi(false)
|
||||
.with_writer(logbuf::RingMakeWriter),
|
||||
)
|
||||
.init();
|
||||
}
|
||||
|
||||
let listen = cfg.listen.clone();
|
||||
let admin_enabled = cfg.admin.enabled;
|
||||
let (admin_token, generated) = resolve_admin_token(&cfg, &args);
|
||||
let admin_token = resolve_admin_token(&cfg, &args);
|
||||
let first_run = needs_first_run_wizard(&cfg, &config_path);
|
||||
|
||||
let state = SharedState::build(cfg, args.config.clone())?;
|
||||
let state = SharedState::build(cfg, config_path.clone())?;
|
||||
|
||||
// Open admin console reachable from other machines: fine on a laptop,
|
||||
// a footgun on a VPS. Flag it for the UI banner and say it in the log.
|
||||
let host_public = listen.starts_with("0.0.0.0") || listen.starts_with("[::]");
|
||||
if admin_enabled && admin_token.is_empty() && host_public && !is_docker() {
|
||||
state
|
||||
.admin_open_public
|
||||
.store(true, std::sync::atomic::Ordering::Relaxed);
|
||||
tracing::warn!(
|
||||
"admin console is OPEN (no token) and listening on {listen} — anyone on your network can manage this gateway; set a token via --admin-token or [admin].token if this machine is reachable from outside"
|
||||
);
|
||||
}
|
||||
|
||||
// Close the loop with the CMF format (github.com/infosave2007/cmf): install
|
||||
// / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
|
||||
@@ -94,9 +256,10 @@ async fn main() -> anyhow::Result<()> {
|
||||
|
||||
let mut app = Router::new()
|
||||
.route("/healthz", get(|| async { "ok" }))
|
||||
.route("/health", get(|| async { "ok" }))
|
||||
.route("/readyz", get(|| async { "ready" }))
|
||||
.route("/metrics", get(admin::metrics))
|
||||
.merge(protocols::build_router());
|
||||
.merge(protocols::build_router(state.clone()));
|
||||
|
||||
if admin_enabled {
|
||||
app = app
|
||||
@@ -104,18 +267,92 @@ async fn main() -> anyhow::Result<()> {
|
||||
.fallback(admin::assets::fallback);
|
||||
}
|
||||
|
||||
let app = app.with_state(state);
|
||||
let cors = tower_http::cors::CorsLayer::new()
|
||||
.allow_origin(tower_http::cors::Any)
|
||||
.allow_methods(tower_http::cors::Any)
|
||||
.allow_headers(tower_http::cors::Any);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(&listen).await?;
|
||||
// 32 MB request ceiling: enough for base64 keyframes and TOML imports,
|
||||
// small enough that a hostile client can't balloon memory
|
||||
let app =
|
||||
app.with_state(state)
|
||||
.layer(cors)
|
||||
.layer(tower_http::limit::RequestBodyLimitLayer::new(
|
||||
32 * 1024 * 1024,
|
||||
));
|
||||
|
||||
let listener = {
|
||||
let mut attempt = 0u32;
|
||||
loop {
|
||||
match tokio::net::TcpListener::bind(&listen).await {
|
||||
Ok(l) => break l,
|
||||
Err(e) => {
|
||||
// during self-restart the old process is still letting go of
|
||||
// the port — retry for a few seconds before giving up
|
||||
let retrying = std::env::var("CORTIQ_BIND_RETRY").is_ok();
|
||||
if retrying && attempt < 50 {
|
||||
attempt += 1;
|
||||
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
|
||||
continue;
|
||||
}
|
||||
return Err(e.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
tracing::info!("cortiq-gateway listening on {listen}");
|
||||
if admin_enabled {
|
||||
tracing::info!("admin console: http://{listen}/admin");
|
||||
if generated {
|
||||
tracing::warn!(
|
||||
"admin token (auto-generated, set [admin].token_env to persist): {admin_token}"
|
||||
);
|
||||
let host_for_url = if listen.starts_with("0.0.0.0:") {
|
||||
listen.replacen("0.0.0.0", "127.0.0.1", 1)
|
||||
} else {
|
||||
listen.clone()
|
||||
};
|
||||
// On first run open the onboarding wizard, otherwise the normal console.
|
||||
let admin_path = if first_run {
|
||||
"/admin#/onboarding"
|
||||
} else {
|
||||
"/admin"
|
||||
};
|
||||
let url = if admin_token.is_empty() {
|
||||
format!("http://{host_for_url}{admin_path}")
|
||||
} else {
|
||||
format!("http://{host_for_url}{admin_path}?token={admin_token}")
|
||||
};
|
||||
if admin_token.is_empty() {
|
||||
tracing::info!("admin console (open, no token): http://{host_for_url}/admin");
|
||||
} else {
|
||||
tracing::info!("admin console: http://{host_for_url}/admin");
|
||||
}
|
||||
if first_run {
|
||||
tracing::info!("first run — opening onboarding wizard at {url}");
|
||||
}
|
||||
if admin_enabled && should_open_browser(&args) {
|
||||
let browser_url = url.clone();
|
||||
tokio::spawn(async move {
|
||||
// give server a moment to be ready
|
||||
tokio::time::sleep(std::time::Duration::from_millis(600)).await;
|
||||
#[cfg(feature = "open-browser")]
|
||||
{
|
||||
if let Err(e) = open::that(&browser_url) {
|
||||
tracing::warn!("failed to open browser: {e}");
|
||||
} else {
|
||||
tracing::info!("opened browser at {browser_url}");
|
||||
}
|
||||
}
|
||||
#[cfg(not(feature = "open-browser"))]
|
||||
{
|
||||
tracing::info!("open browser: {browser_url} (feature disabled)");
|
||||
}
|
||||
});
|
||||
} else if first_run {
|
||||
tracing::info!("open this URL to finish setup: {url}");
|
||||
}
|
||||
}
|
||||
axum::serve(listener, app).await?;
|
||||
axum::serve(listener, app)
|
||||
.with_graceful_shutdown(async {
|
||||
let _ = tokio::signal::ctrl_c().await;
|
||||
tracing::info!("shutdown signal received — draining connections");
|
||||
})
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
+861
@@ -0,0 +1,861 @@
|
||||
//! Media generation: run `cortiq imagine` (text→image, Lumina) and
|
||||
//! `cortiq animate` (text→video+audio, MiniMax-H3) over local `.cmf` files as
|
||||
//! tracked background jobs, the same shape as import jobs so the admin UI can
|
||||
//! poll progress and fetch the result.
|
||||
//!
|
||||
//! These models are NOT served by `cortiq serve` — each generation is one CLI
|
||||
//! run. The gateway owns the process, parses its progress, and keeps the output
|
||||
//! under `<data>/media/<job>/`.
|
||||
|
||||
use serde::Serialize;
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Stdio;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
use tokio::io::{AsyncBufReadExt, BufReader};
|
||||
use tokio::sync::Notify;
|
||||
|
||||
fn now() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Random job id — media jobs are never deduplicated (every run is a new work).
|
||||
fn gen_media_id() -> String {
|
||||
let mut b = [0u8; 8];
|
||||
let _ = getrandom::getrandom(&mut b);
|
||||
b.iter().map(|x| format!("{x:02x}")).collect()
|
||||
}
|
||||
|
||||
/// What a `.cmf` under models_dir can do, decided by `cortiq info`.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, Debug, Serialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum Modality {
|
||||
Chat,
|
||||
Image,
|
||||
Video,
|
||||
}
|
||||
|
||||
/// `cortiq info` output → modality. The arch line names the family; the media
|
||||
/// packs also carry their pipeline components (DiT/VAE/vocoder) in the text.
|
||||
fn classify_info(info: &str) -> Modality {
|
||||
let lower = info.to_ascii_lowercase();
|
||||
let arch = lower
|
||||
.lines()
|
||||
.find_map(|l| l.trim().strip_prefix("arch:"))
|
||||
.unwrap_or("")
|
||||
.trim()
|
||||
.to_string();
|
||||
// generic arch suffixes first (e.g. "lumina2-image", "mmh3-video"), then
|
||||
// known family names as a fallback for packs without the suffix
|
||||
if arch.contains("video") || arch.contains("mmh3") || arch.contains("minimax") {
|
||||
return Modality::Video;
|
||||
}
|
||||
if arch.contains("image") || arch.contains("lumina") || arch.contains("diffusion") {
|
||||
return Modality::Image;
|
||||
}
|
||||
if lower.contains("text-to-video") {
|
||||
return Modality::Video;
|
||||
}
|
||||
if lower.contains("text-to-image") {
|
||||
return Modality::Image;
|
||||
}
|
||||
Modality::Chat
|
||||
}
|
||||
|
||||
/// One local `.cmf` with its detected modality.
|
||||
#[derive(Clone, Serialize)]
|
||||
pub struct MediaModel {
|
||||
pub name: String, // file stem
|
||||
pub path: String,
|
||||
pub size_bytes: u64,
|
||||
pub modality: Modality,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct InfoCache(Mutex<HashMap<String, (u64, u64, Modality)>>); // path → (mtime, size, modality)
|
||||
|
||||
/// Modality of one `.cmf`, via `cortiq info` — cached by (path, mtime, size),
|
||||
/// so the process runs once per file version.
|
||||
pub async fn classify_file(path: &Path, cortiq_bin: &str) -> Modality {
|
||||
static CACHE: std::sync::OnceLock<InfoCache> = std::sync::OnceLock::new();
|
||||
let cache = CACHE.get_or_init(InfoCache::default);
|
||||
|
||||
let size = crate::import::path_size(path);
|
||||
let mtime = std::fs::metadata(path)
|
||||
.ok()
|
||||
.and_then(|m| m.modified().ok())
|
||||
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0);
|
||||
let key = path.to_string_lossy().to_string();
|
||||
if let Some(k) = cache
|
||||
.0
|
||||
.lock()
|
||||
.unwrap()
|
||||
.get(&key)
|
||||
.filter(|(m, s, _)| *m == mtime && *s == size)
|
||||
.map(|(_, _, k)| *k)
|
||||
{
|
||||
return k;
|
||||
}
|
||||
let output = tokio::process::Command::new(cortiq_bin)
|
||||
.arg("info")
|
||||
.arg(path)
|
||||
.output()
|
||||
.await;
|
||||
let text = output
|
||||
.map(|o| {
|
||||
format!(
|
||||
"{}{}",
|
||||
String::from_utf8_lossy(&o.stdout),
|
||||
String::from_utf8_lossy(&o.stderr)
|
||||
)
|
||||
})
|
||||
.unwrap_or_default();
|
||||
let k = classify_info(&text);
|
||||
cache.0.lock().unwrap().insert(key, (mtime, size, k));
|
||||
k
|
||||
}
|
||||
|
||||
/// Scan models_dir and classify every `.cmf` (file or sharded dir).
|
||||
pub async fn list_media_models(models_dir: &str, cortiq_bin: &str) -> Vec<MediaModel> {
|
||||
let mut out = Vec::new();
|
||||
let Ok(rd) = std::fs::read_dir(models_dir) else {
|
||||
return out;
|
||||
};
|
||||
for entry in rd.filter_map(|e| e.ok()) {
|
||||
let path = entry.path();
|
||||
let name = path
|
||||
.file_name()
|
||||
.unwrap_or_default()
|
||||
.to_string_lossy()
|
||||
.to_string();
|
||||
if !name.to_ascii_lowercase().ends_with(".cmf") {
|
||||
continue;
|
||||
}
|
||||
let size = crate::import::path_size(&path);
|
||||
if size == 0 {
|
||||
continue; // empty or still being written
|
||||
}
|
||||
let modality = classify_file(&path, cortiq_bin).await;
|
||||
out.push(MediaModel {
|
||||
name: name
|
||||
.trim_end_matches(".cmf")
|
||||
.trim_end_matches(".CMF")
|
||||
.to_string(),
|
||||
path: path.to_string_lossy().to_string(),
|
||||
size_bytes: size,
|
||||
modality,
|
||||
});
|
||||
}
|
||||
out.sort_by(|a, b| a.name.cmp(&b.name));
|
||||
out
|
||||
}
|
||||
|
||||
/// Parameters for one generation, straight from the playground form.
|
||||
#[derive(Clone, Debug, serde::Deserialize)]
|
||||
pub struct MediaParams {
|
||||
pub model: String, // file stem under models_dir
|
||||
pub kind: String, // image | video
|
||||
pub prompt: String,
|
||||
#[serde(default)]
|
||||
pub width: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub height: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub steps: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub cfg: Option<f32>, // image only
|
||||
#[serde(default)]
|
||||
pub seed: Option<u64>,
|
||||
#[serde(default)]
|
||||
pub frames: Option<u32>, // video only
|
||||
#[serde(default)]
|
||||
pub quality: Option<u32>, // video only: AVI JPEG quality
|
||||
/// Optional first/last keyframe for video, base64-encoded binary P6 PPM.
|
||||
#[serde(default)]
|
||||
pub first_frame_b64: Option<String>,
|
||||
#[serde(default)]
|
||||
pub last_frame_b64: Option<String>,
|
||||
}
|
||||
|
||||
/// Live state of one generation.
|
||||
#[derive(Clone, Serialize, serde::Deserialize)]
|
||||
pub struct MediaJob {
|
||||
pub id: String,
|
||||
pub kind: String, // image | video
|
||||
pub model: String,
|
||||
pub prompt: String,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub steps: u32,
|
||||
pub seed: u64,
|
||||
pub frames: Option<u32>,
|
||||
pub state: String, // queued | running | done | error | cancelled
|
||||
pub progress: Option<f32>,
|
||||
pub phase: Option<String>,
|
||||
pub log: Vec<String>,
|
||||
pub started: u64,
|
||||
pub finished: Option<u64>,
|
||||
/// Files this job produced: name → relative kind ("image", "video", "audio", "mp4").
|
||||
pub outputs: Vec<String>,
|
||||
pub out_bytes: Option<u64>,
|
||||
}
|
||||
|
||||
/// Everything needed to actually launch a queued job when its turn comes.
|
||||
struct Prepared {
|
||||
bin: String,
|
||||
args: Vec<String>,
|
||||
gpu: bool,
|
||||
dir: PathBuf,
|
||||
out_path: PathBuf,
|
||||
kind: String,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct MediaStore {
|
||||
jobs: Mutex<HashMap<String, MediaJob>>,
|
||||
cancels: Mutex<HashMap<String, Arc<Notify>>>,
|
||||
/// One generation runs at a time — these models take the whole GPU/RAM
|
||||
/// budget — but new requests queue up instead of being rejected.
|
||||
queue: Mutex<std::collections::VecDeque<String>>,
|
||||
current: Mutex<Option<String>>,
|
||||
prepared: Mutex<HashMap<String, Prepared>>,
|
||||
persist: Mutex<Option<PathBuf>>,
|
||||
}
|
||||
|
||||
impl MediaStore {
|
||||
pub fn new() -> Arc<Self> {
|
||||
Arc::new(Self::default())
|
||||
}
|
||||
/// Load saved generations (their files are still under media_dir) and start
|
||||
/// persisting. Jobs that were queued/running when the gateway died settle
|
||||
/// as errors — the process is gone.
|
||||
pub fn attach_persistence(&self, path: PathBuf) {
|
||||
if let Ok(text) = std::fs::read_to_string(&path) {
|
||||
if let Ok(mut jobs) = serde_json::from_str::<Vec<MediaJob>>(&text) {
|
||||
let mut g = self.jobs.lock().unwrap();
|
||||
for mut j in jobs.drain(..) {
|
||||
if j.state == "running" || j.state == "queued" {
|
||||
j.state = "error".into();
|
||||
j.log.push("✗ interrupted by a gateway restart".into());
|
||||
j.finished = Some(now());
|
||||
}
|
||||
g.insert(j.id.clone(), j);
|
||||
}
|
||||
}
|
||||
}
|
||||
*self.persist.lock().unwrap() = Some(path);
|
||||
}
|
||||
fn save(&self) {
|
||||
let Some(path) = self.persist.lock().unwrap().clone() else {
|
||||
return;
|
||||
};
|
||||
let jobs: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
|
||||
if let Ok(json) = serde_json::to_vec(&jobs) {
|
||||
let tmp = path.with_extension("json.tmp");
|
||||
if std::fs::write(&tmp, json).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &path);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// How many jobs sit in the queue ahead of `id` (0 = next up).
|
||||
pub fn queue_position(&self, id: &str) -> Option<usize> {
|
||||
self.queue.lock().unwrap().iter().position(|x| x == id)
|
||||
}
|
||||
pub fn list(&self) -> Vec<MediaJob> {
|
||||
let mut v: Vec<MediaJob> = self.jobs.lock().unwrap().values().cloned().collect();
|
||||
v.sort_by_key(|j| std::cmp::Reverse(j.started));
|
||||
v.truncate(20);
|
||||
v
|
||||
}
|
||||
pub fn get(&self, id: &str) -> Option<MediaJob> {
|
||||
self.jobs.lock().unwrap().get(id).cloned()
|
||||
}
|
||||
pub fn cancel(self: &Arc<Self>, id: &str, media_dir: &Path) -> bool {
|
||||
let state = self
|
||||
.jobs
|
||||
.lock()
|
||||
.unwrap()
|
||||
.get(id)
|
||||
.map(|j| j.state.clone())
|
||||
.unwrap_or_default();
|
||||
match state.as_str() {
|
||||
"queued" => {
|
||||
self.queue.lock().unwrap().retain(|x| x != id);
|
||||
self.prepared.lock().unwrap().remove(id);
|
||||
let _ = std::fs::remove_dir_all(media_dir.join(id));
|
||||
self.set_cancelled(id);
|
||||
true
|
||||
}
|
||||
"running" => {
|
||||
if let Some(n) = self.cancels.lock().unwrap().get(id) {
|
||||
n.notify_one();
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
/// Remove a settled job and its output directory. Err = still running.
|
||||
pub fn delete(&self, id: &str, media_dir: &Path) -> Result<bool, String> {
|
||||
let job = self.jobs.lock().unwrap().get(id).cloned();
|
||||
let Some(job) = job else { return Ok(false) };
|
||||
if job.state == "running" || job.state == "queued" {
|
||||
return Err("cancel the running generation first".into());
|
||||
}
|
||||
let _ = std::fs::remove_dir_all(media_dir.join(id));
|
||||
self.jobs.lock().unwrap().remove(id);
|
||||
self.save();
|
||||
Ok(true)
|
||||
}
|
||||
fn push_line(&self, id: &str, line: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.log.push(line);
|
||||
let n = j.log.len();
|
||||
if n > 120 {
|
||||
j.log.drain(0..n - 120);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn set_progress(&self, id: &str, frac: f32, phase: Option<String>) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
let f = frac.clamp(0.0, 1.0);
|
||||
if j.progress.is_none() || f > j.progress.unwrap() {
|
||||
j.progress = Some(f);
|
||||
}
|
||||
if let Some(p) = phase {
|
||||
j.phase = Some(p);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn finish(self: &Arc<Self>, id: &str, ok: bool, outputs: Vec<String>, out_bytes: u64) {
|
||||
let mut g = self.jobs.lock().unwrap();
|
||||
if let Some(j) = g.get_mut(id) {
|
||||
if j.state == "cancelled" {
|
||||
drop(g);
|
||||
self.after_slot_freed(id);
|
||||
return;
|
||||
}
|
||||
j.finished = Some(now());
|
||||
j.state = if ok { "done" } else { "error" }.into();
|
||||
if ok {
|
||||
j.progress = Some(1.0);
|
||||
j.outputs = outputs;
|
||||
j.out_bytes = Some(out_bytes);
|
||||
}
|
||||
}
|
||||
drop(g);
|
||||
self.after_slot_freed(id);
|
||||
}
|
||||
fn set_cancelled(self: &Arc<Self>, id: &str) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.state = "cancelled".into();
|
||||
j.finished = Some(now());
|
||||
}
|
||||
self.after_slot_freed(id);
|
||||
}
|
||||
fn after_slot_freed(self: &Arc<Self>, id: &str) {
|
||||
{
|
||||
let mut cur = self.current.lock().unwrap();
|
||||
if cur.as_deref() == Some(id) {
|
||||
*cur = None;
|
||||
}
|
||||
}
|
||||
self.cancels.lock().unwrap().remove(id);
|
||||
self.save();
|
||||
promote_next(self.clone());
|
||||
}
|
||||
}
|
||||
|
||||
/// Start the next queued generation, if the slot is free.
|
||||
fn promote_next(store: Arc<MediaStore>) {
|
||||
let next = {
|
||||
let mut cur = store.current.lock().unwrap();
|
||||
if cur.is_some() {
|
||||
return;
|
||||
}
|
||||
let Some(id) = store.queue.lock().unwrap().pop_front() else {
|
||||
return;
|
||||
};
|
||||
*cur = Some(id.clone());
|
||||
id
|
||||
};
|
||||
let Some(p) = store.prepared.lock().unwrap().remove(&next) else {
|
||||
// job was cancelled while queued and slipped through — free the slot
|
||||
*store.current.lock().unwrap() = None;
|
||||
return;
|
||||
};
|
||||
if let Some(j) = store.jobs.lock().unwrap().get_mut(&next) {
|
||||
j.state = "running".into();
|
||||
j.phase = Some("starting".into());
|
||||
j.started = now();
|
||||
}
|
||||
store.save();
|
||||
let cancel = Arc::new(Notify::new());
|
||||
store
|
||||
.cancels
|
||||
.lock()
|
||||
.unwrap()
|
||||
.insert(next.clone(), cancel.clone());
|
||||
let s = store.clone();
|
||||
tokio::spawn(async move {
|
||||
run_generation(
|
||||
s, next, p.bin, p.args, p.gpu, p.dir, p.out_path, p.kind, cancel,
|
||||
)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
|
||||
/// Progress markers in `cortiq imagine`/`animate` stdout. The exact wording has
|
||||
/// varied between releases, so accept anything that carries an X/Y pair near a
|
||||
/// step/frame word, a bare percentage, or the converter-style `@PROGRESS`.
|
||||
fn parse_media_progress(line: &str) -> Option<(f32, Option<String>)> {
|
||||
let l = line.trim();
|
||||
if let Some(rest) = l.strip_prefix("@PROGRESS ") {
|
||||
let mut it = rest.splitn(2, ' ');
|
||||
let frac: f32 = it.next()?.trim().parse().ok()?;
|
||||
let phase = it
|
||||
.next()
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty());
|
||||
return Some((frac.clamp(0.0, 1.0), phase));
|
||||
}
|
||||
let lower = l.to_ascii_lowercase();
|
||||
// "step 12/30", "denoise 3/4", "frame 7/39" — any word followed by x/y
|
||||
for word in ["step", "frame", "sigma", "denois"] {
|
||||
if let Some(pos) = lower.find(word) {
|
||||
let tail = &lower[pos..];
|
||||
if let Some((x, y)) = extract_ratio(tail) {
|
||||
return Some((x / y, Some(l.to_string())));
|
||||
}
|
||||
}
|
||||
}
|
||||
// bare "42%" (avoid matching sizes like "42%faster")
|
||||
if let Some(idx) = lower.find('%') {
|
||||
let head: String = lower[..idx]
|
||||
.chars()
|
||||
.rev()
|
||||
.take_while(|c| c.is_ascii_digit() || *c == '.')
|
||||
.collect();
|
||||
let num: String = head.chars().rev().collect();
|
||||
if let Ok(p) = num.parse::<f32>() {
|
||||
if (0.0..=100.0).contains(&p) {
|
||||
return Some((p / 100.0, None));
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// First "N/M" pair in the string with M > 0.
|
||||
fn extract_ratio(s: &str) -> Option<(f32, f32)> {
|
||||
let bytes = s.as_bytes();
|
||||
let mut i = 0;
|
||||
while i < bytes.len() {
|
||||
if bytes[i].is_ascii_digit() {
|
||||
let start = i;
|
||||
while i < bytes.len() && bytes[i].is_ascii_digit() {
|
||||
i += 1;
|
||||
}
|
||||
if i < bytes.len() && bytes[i] == b'/' {
|
||||
let x: f32 = s[start..i].parse().ok()?;
|
||||
let ystart = i + 1;
|
||||
let mut j = ystart;
|
||||
while j < bytes.len() && bytes[j].is_ascii_digit() {
|
||||
j += 1;
|
||||
}
|
||||
if j > ystart {
|
||||
if let Ok(y) = s[ystart..j].parse::<f32>() {
|
||||
if y > 0.0 && x <= y {
|
||||
return Some((x, y));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Root for generated files: `<data>/media`.
|
||||
pub fn media_dir() -> PathBuf {
|
||||
crate::config::data_dir().join("media")
|
||||
}
|
||||
|
||||
const B64: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
/// Minimal base64 decode (standard alphabet, padding optional) — the keyframes
|
||||
/// are the only base64 input the gateway takes; not worth a dependency.
|
||||
fn b64_decode(s: &str) -> Option<Vec<u8>> {
|
||||
let mut lut = [255u8; 256];
|
||||
for (i, c) in B64.iter().enumerate() {
|
||||
lut[*c as usize] = i as u8;
|
||||
}
|
||||
let mut out = Vec::with_capacity(s.len() * 3 / 4);
|
||||
let mut buf = 0u32;
|
||||
let mut bits = 0u32;
|
||||
for c in s.bytes() {
|
||||
if c == b'=' || c == b'\n' || c == b'\r' {
|
||||
continue;
|
||||
}
|
||||
let v = lut[c as usize];
|
||||
if v == 255 {
|
||||
return None;
|
||||
}
|
||||
buf = (buf << 6) | v as u32;
|
||||
bits += 6;
|
||||
if bits >= 8 {
|
||||
bits -= 8;
|
||||
out.push((buf >> bits) as u8);
|
||||
}
|
||||
}
|
||||
Some(out)
|
||||
}
|
||||
|
||||
/// Kick off a generation. Returns the job id, or an error when another one is
|
||||
/// already running / the model file is missing.
|
||||
pub fn start_generation(
|
||||
store: Arc<MediaStore>,
|
||||
cmf: &crate::config::CmfCfg,
|
||||
p: MediaParams,
|
||||
) -> Result<String, String> {
|
||||
if p.prompt.trim().is_empty() {
|
||||
return Err("empty prompt".into());
|
||||
}
|
||||
let kind = p.kind.as_str();
|
||||
if kind != "image" && kind != "video" {
|
||||
return Err(format!("unknown kind '{kind}'"));
|
||||
}
|
||||
let model_path = Path::new(&cmf.models_dir).join(format!("{}.cmf", p.model));
|
||||
if !model_path.exists() {
|
||||
return Err(format!("model file not found: {}", model_path.display()));
|
||||
}
|
||||
|
||||
let id = gen_media_id();
|
||||
let dir = media_dir().join(&id);
|
||||
if let Err(e) = std::fs::create_dir_all(&dir) {
|
||||
return Err(format!("cannot create media dir {}: {e}", dir.display()));
|
||||
}
|
||||
|
||||
// Optional keyframes (video): decode to PPM files next to the output.
|
||||
let mut first_frame: Option<PathBuf> = None;
|
||||
let mut last_frame: Option<PathBuf> = None;
|
||||
for (b64, slot, name) in [
|
||||
(&p.first_frame_b64, &mut first_frame, "first.ppm"),
|
||||
(&p.last_frame_b64, &mut last_frame, "last.ppm"),
|
||||
] {
|
||||
if let Some(data) = b64.as_deref().filter(|s| !s.is_empty()) {
|
||||
let bytes = b64_decode(data).ok_or("invalid keyframe base64")?;
|
||||
if !bytes.starts_with(b"P6") {
|
||||
return Err("keyframe must be a binary P6 PPM".into());
|
||||
}
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, bytes).map_err(|e| format!("write keyframe: {e}"))?;
|
||||
*slot = Some(path);
|
||||
}
|
||||
}
|
||||
|
||||
let width = p.width.unwrap_or(512).clamp(64, 2048);
|
||||
let height = p
|
||||
.height
|
||||
.unwrap_or(if kind == "video" { 288 } else { 512 })
|
||||
.clamp(64, 2048);
|
||||
let steps = p
|
||||
.steps
|
||||
.unwrap_or(if kind == "video" { 4 } else { 30 })
|
||||
.clamp(1, 200);
|
||||
let seed = p.seed.unwrap_or(42);
|
||||
let frames = p.frames.unwrap_or(39).clamp(1, 1000);
|
||||
|
||||
let out_name = if kind == "image" {
|
||||
"out.ppm"
|
||||
} else {
|
||||
"out.avi"
|
||||
};
|
||||
let out_path = dir.join(out_name);
|
||||
|
||||
let mut args: Vec<String> = if kind == "image" {
|
||||
vec![
|
||||
"imagine".into(),
|
||||
model_path.to_string_lossy().to_string(),
|
||||
"--prompt".into(),
|
||||
p.prompt.clone(),
|
||||
"--width".into(),
|
||||
width.to_string(),
|
||||
"--height".into(),
|
||||
height.to_string(),
|
||||
"--steps".into(),
|
||||
steps.to_string(),
|
||||
"--cfg".into(),
|
||||
format!("{}", p.cfg.unwrap_or(4.0)),
|
||||
"--seed".into(),
|
||||
seed.to_string(),
|
||||
"--out".into(),
|
||||
out_path.to_string_lossy().to_string(),
|
||||
]
|
||||
} else {
|
||||
let mut a = vec![
|
||||
"animate".into(),
|
||||
model_path.to_string_lossy().to_string(),
|
||||
"--prompt".into(),
|
||||
p.prompt.clone(),
|
||||
"--width".into(),
|
||||
width.to_string(),
|
||||
"--height".into(),
|
||||
height.to_string(),
|
||||
"--frames".into(),
|
||||
frames.to_string(),
|
||||
"--steps".into(),
|
||||
steps.to_string(),
|
||||
"--seed".into(),
|
||||
seed.to_string(),
|
||||
"--quality".into(),
|
||||
p.quality.unwrap_or(92).clamp(10, 100).to_string(),
|
||||
"--out".into(),
|
||||
out_path.to_string_lossy().to_string(),
|
||||
];
|
||||
if let Some(f) = &first_frame {
|
||||
a.push("--first-frame".into());
|
||||
a.push(f.to_string_lossy().to_string());
|
||||
}
|
||||
if let Some(f) = &last_frame {
|
||||
a.push("--last-frame".into());
|
||||
a.push(f.to_string_lossy().to_string());
|
||||
}
|
||||
a
|
||||
};
|
||||
// keep the model path first for readability in logs
|
||||
let _ = &mut args;
|
||||
|
||||
store.jobs.lock().unwrap().insert(
|
||||
id.clone(),
|
||||
MediaJob {
|
||||
id: id.clone(),
|
||||
kind: kind.to_string(),
|
||||
model: p.model.clone(),
|
||||
prompt: p.prompt.clone(),
|
||||
width,
|
||||
height,
|
||||
steps,
|
||||
seed,
|
||||
frames: (kind == "video").then_some(frames),
|
||||
state: "queued".into(),
|
||||
progress: None,
|
||||
phase: None,
|
||||
log: vec![format!("→ cortiq {}", args.join(" "))],
|
||||
started: now(),
|
||||
finished: None,
|
||||
outputs: Vec::new(),
|
||||
out_bytes: None,
|
||||
},
|
||||
);
|
||||
store.prepared.lock().unwrap().insert(
|
||||
id.clone(),
|
||||
Prepared {
|
||||
bin: cmf.cortiq_bin.clone(),
|
||||
args,
|
||||
gpu: cmf.gpu,
|
||||
dir,
|
||||
out_path,
|
||||
kind: kind.to_string(),
|
||||
},
|
||||
);
|
||||
store.queue.lock().unwrap().push_back(id.clone());
|
||||
store.save();
|
||||
promote_next(store.clone());
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn run_generation(
|
||||
store: Arc<MediaStore>,
|
||||
id: String,
|
||||
bin: String,
|
||||
args: Vec<String>,
|
||||
gpu: bool,
|
||||
dir: PathBuf,
|
||||
out_path: PathBuf,
|
||||
kind: String,
|
||||
cancel: Arc<Notify>,
|
||||
) {
|
||||
let mut cmd = tokio::process::Command::new(&bin);
|
||||
cmd.args(&args)
|
||||
.env("CMF_GPU", if gpu { "1" } else { "0" })
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped());
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed to start cortiq: {e}"));
|
||||
store.finish(&id, false, Vec::new(), 0);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let mut readers = Vec::new();
|
||||
if let Some(o) = child.stdout.take() {
|
||||
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), o)));
|
||||
}
|
||||
if let Some(e) = child.stderr.take() {
|
||||
readers.push(tokio::spawn(read_lines(store.clone(), id.clone(), e)));
|
||||
}
|
||||
|
||||
let status = tokio::select! {
|
||||
s = child.wait() => s,
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
for r in readers { r.abort(); }
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
store.push_line(&id, "✗ cancelled".into());
|
||||
store.set_cancelled(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
for r in readers {
|
||||
let _ = r.await;
|
||||
}
|
||||
|
||||
let ok = status.map(|s| s.success()).unwrap_or(false);
|
||||
if !ok || !out_path.exists() {
|
||||
store.push_line(&id, "✗ generation failed — see the log above".into());
|
||||
store.finish(&id, false, Vec::new(), 0);
|
||||
return;
|
||||
}
|
||||
|
||||
let mut outputs = vec![if kind == "image" { "image" } else { "video" }.to_string()];
|
||||
if kind == "video" {
|
||||
// `animate` writes a .wav next to the .avi
|
||||
if dir.join("out.wav").exists() {
|
||||
outputs.push("audio".into());
|
||||
}
|
||||
// opportunistic browser-friendly remux when ffmpeg is present
|
||||
if let Ok(p) = which_ffmpeg() {
|
||||
store.push_line(&id, "→ remuxing to mp4 (ffmpeg found)".into());
|
||||
let mp4 = dir.join("out.mp4");
|
||||
let mut f = tokio::process::Command::new(p);
|
||||
f.arg("-y").arg("-i").arg(dir.join("out.avi"));
|
||||
if dir.join("out.wav").exists() {
|
||||
f.arg("-i")
|
||||
.arg(dir.join("out.wav"))
|
||||
.args(["-map", "0:v", "-map", "1:a", "-c:a", "aac"]);
|
||||
}
|
||||
f.args(["-c:v", "libx264", "-pix_fmt", "yuv420p"]).arg(&mp4);
|
||||
f.stdout(Stdio::null()).stderr(Stdio::null());
|
||||
if matches!(f.status().await, Ok(s) if s.success()) && mp4.exists() {
|
||||
outputs.push("mp4".into());
|
||||
}
|
||||
}
|
||||
}
|
||||
let bytes = crate::import::path_size(&out_path);
|
||||
store.push_line(
|
||||
&id,
|
||||
format!("✓ done — {}", crate::import::format_bytes(bytes)),
|
||||
);
|
||||
store.finish(&id, true, outputs, bytes);
|
||||
}
|
||||
|
||||
/// Strip ANSI color escapes that `tracing` writes even into a pipe.
|
||||
fn strip_ansi(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
let mut chars = s.chars();
|
||||
while let Some(c) = chars.next() {
|
||||
if c == '\u{1b}' {
|
||||
for d in chars.by_ref() {
|
||||
if d.is_ascii_alphabetic() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
out.push(c);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
async fn read_lines(store: Arc<MediaStore>, id: String, stream: impl tokio::io::AsyncRead + Unpin) {
|
||||
let mut lines = BufReader::new(stream).lines();
|
||||
while let Ok(Some(l)) = lines.next_line().await {
|
||||
let l = strip_ansi(l.trim());
|
||||
if l.is_empty() {
|
||||
continue;
|
||||
}
|
||||
// the Metal backend logs one line per pipeline — dozens per run, all identical
|
||||
if l.contains("Metal GPU path") {
|
||||
continue;
|
||||
}
|
||||
if let Some((frac, phase)) = parse_media_progress(&l) {
|
||||
store.set_progress(&id, frac, phase);
|
||||
} else {
|
||||
store.push_line(&id, l);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn which_ffmpeg() -> Result<String, ()> {
|
||||
for p in [
|
||||
"ffmpeg",
|
||||
"/usr/local/bin/ffmpeg",
|
||||
"/opt/homebrew/bin/ffmpeg",
|
||||
] {
|
||||
if std::process::Command::new(p)
|
||||
.arg("-version")
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.status()
|
||||
.map(|s| s.success())
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return Ok(p.to_string());
|
||||
}
|
||||
}
|
||||
Err(())
|
||||
}
|
||||
|
||||
/// (relative file, content-type) for an output kind of a finished job.
|
||||
pub fn output_file(kind: &str) -> Option<(&'static str, &'static str)> {
|
||||
match kind {
|
||||
"image" => Some(("out.ppm", "image/x-portable-pixmap")),
|
||||
"video" => Some(("out.avi", "video/x-msvideo")),
|
||||
"audio" => Some(("out.wav", "audio/wav")),
|
||||
"mp4" => Some(("out.mp4", "video/mp4")),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn progress_parses_common_shapes() {
|
||||
assert_eq!(parse_media_progress("step 15/30").unwrap().0, 0.5);
|
||||
assert_eq!(
|
||||
parse_media_progress("Denoising step 3/4 ...").unwrap().0,
|
||||
0.75
|
||||
);
|
||||
assert_eq!(parse_media_progress("frame 39/39 decoded").unwrap().0, 1.0);
|
||||
assert_eq!(parse_media_progress("@PROGRESS 0.25 vae").unwrap().0, 0.25);
|
||||
assert!(parse_media_progress("loaded 2361 tensors").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_by_arch() {
|
||||
assert_eq!(classify_info(" Arch: lumina2\n"), Modality::Image);
|
||||
assert_eq!(classify_info(" Arch: mmh3\n"), Modality::Video);
|
||||
assert_eq!(classify_info(" Arch: qwen3\n"), Modality::Chat);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn b64_roundtrip() {
|
||||
assert_eq!(b64_decode("UDYgd2g=").unwrap(), b"P6 wh");
|
||||
assert!(b64_decode("!!").is_none());
|
||||
}
|
||||
}
|
||||
+48
-2
@@ -15,13 +15,53 @@ pub enum RoutingDirective {
|
||||
Pinned { model_id: String },
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub role: String, // system | user | assistant | tool
|
||||
#[serde(default)]
|
||||
/// Always a plain string internally. On the wire OpenAI clients legitimately
|
||||
/// send `null` (assistant turns that only carry tool calls) or an array of
|
||||
/// content parts (Open WebUI and every other multimodal UI), so
|
||||
/// [`de_content`] flattens both shapes instead of rejecting the request.
|
||||
#[serde(default, deserialize_with = "de_content")]
|
||||
pub content: String,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub tool_calls: Vec<serde_json::Value>,
|
||||
/// Speaker name (`messages[].name`) — proxied through when the client sends it.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
/// Set on `role = "tool"` replies; dropping it breaks multi-turn tool calling.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tool_call_id: Option<String>,
|
||||
}
|
||||
|
||||
/// Deserialize `messages[].content` from any shape an OpenAI-compatible client
|
||||
/// may send: a string, `null`, or an array of content parts. Text parts are
|
||||
/// concatenated; non-text parts (images, audio, files) are skipped — this
|
||||
/// gateway routes text, and a picture must not turn into a 422.
|
||||
fn de_content<'de, D>(d: D) -> std::result::Result<String, D::Error>
|
||||
where
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
use serde::Deserialize as _;
|
||||
Ok(match serde_json::Value::deserialize(d)? {
|
||||
serde_json::Value::Null => String::new(),
|
||||
serde_json::Value::String(s) => s,
|
||||
serde_json::Value::Array(parts) => {
|
||||
let mut out = String::new();
|
||||
for part in parts {
|
||||
// {"type":"text","text":"..."} / {"type":"input_text","text":"..."}
|
||||
let text = part
|
||||
.get("text")
|
||||
.and_then(|t| t.as_str())
|
||||
.or_else(|| part.as_str());
|
||||
if let Some(t) = text {
|
||||
out.push_str(t);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
other => other.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default)]
|
||||
@@ -29,6 +69,7 @@ pub struct GenParams {
|
||||
pub temperature: Option<f32>,
|
||||
pub max_tokens: Option<u32>,
|
||||
pub top_p: Option<f32>,
|
||||
pub think_budget: Option<u32>,
|
||||
pub stop: Vec<String>,
|
||||
/// Fields we do not interpret but must proxy through to the provider.
|
||||
pub passthrough: serde_json::Map<String, serde_json::Value>,
|
||||
@@ -72,6 +113,11 @@ pub struct RouteInfo {
|
||||
pub cost_usd: f64,
|
||||
#[serde(default)]
|
||||
pub failover: bool,
|
||||
/// The client offered `tools` but the model that answered cannot use them,
|
||||
/// so they were dropped from the upstream call (see `Pipeline::resolve`).
|
||||
/// Surfaced as `X-Cortiq-Tools-Dropped` so a silent downgrade is debuggable.
|
||||
#[serde(default)]
|
||||
pub tools_dropped: bool,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize)]
|
||||
|
||||
+388
-20
@@ -183,7 +183,10 @@ impl Pipeline {
|
||||
&self,
|
||||
req: &ChatRequest,
|
||||
live: &crate::state::Live,
|
||||
) -> Result<(RouteDecision, String, Vec<String>)> {
|
||||
// Tool support detected via the local servers' healthz handshake —
|
||||
// treated as equivalent to declared caps when filtering candidates.
|
||||
detected_tools: &std::collections::HashSet<String>,
|
||||
) -> Result<(RouteDecision, String, Vec<String>, bool)> {
|
||||
// 1. Extract text for classification
|
||||
let text = self.extract_text(
|
||||
&req.messages,
|
||||
@@ -231,9 +234,30 @@ impl Pipeline {
|
||||
let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
|
||||
match live.router.route(&text, prof).await {
|
||||
Ok(Some(dec)) => {
|
||||
let mut dec = dec;
|
||||
// per-gateway complexity bands: re-map the
|
||||
// router's continuous score with the user's
|
||||
// own boundaries (sliders on the Routing page)
|
||||
if let Some(b) = &live.cfg.routing.bands {
|
||||
if matches!(
|
||||
dec.complexity_tier.as_str(),
|
||||
"low" | "medium" | "high"
|
||||
) {
|
||||
dec.complexity_tier =
|
||||
b.tier_of(dec.complexity_score).to_string();
|
||||
}
|
||||
}
|
||||
let src = dec.source.clone();
|
||||
let tier = dec.complexity_tier.clone();
|
||||
(dec, src, live.routing.candidates(&tier))
|
||||
let (cands, task_rule) = live
|
||||
.routing
|
||||
.candidates_for(&dec.task_label, &dec.complexity_tier);
|
||||
// surface which rule chose the head of the chain
|
||||
let src = if task_rule {
|
||||
format!("{src}+task-rule")
|
||||
} else {
|
||||
src
|
||||
};
|
||||
(dec, src, cands)
|
||||
}
|
||||
Ok(None) | Err(_) => {
|
||||
tracing::warn!(
|
||||
@@ -253,10 +277,14 @@ impl Pipeline {
|
||||
|
||||
// Filter candidates by capabilities (e.g. tools support)
|
||||
let needs_tools = !req.tools.is_empty();
|
||||
let tools_mandatory = needs_tools && tool_choice_is_mandatory(&req.params.passthrough);
|
||||
let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
|
||||
prov.caps().tools || detected_tools.contains(id)
|
||||
};
|
||||
let mut final_candidates = Vec::new();
|
||||
for model_id in candidates {
|
||||
if let Some(prov) = live.registry.get(&model_id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&model_id, &prov) {
|
||||
tracing::debug!(
|
||||
"Candidate '{}' skipped: lacks support for requested tools",
|
||||
model_id
|
||||
@@ -267,6 +295,31 @@ impl Pipeline {
|
||||
}
|
||||
}
|
||||
|
||||
// `tools` is an OFFER in the OpenAI contract, not a demand: only
|
||||
// `tool_choice: "required"` (or a named function) makes the call
|
||||
// impossible without them. Chat UIs lean on that — Open WebUI attaches
|
||||
// its ~26 built-in tools to every single message — so treating the offer
|
||||
// as a hard requirement made an ordinary "hello" unanswerable on a pool
|
||||
// of tools-less local models. Serve the request without the tools and
|
||||
// flag the downgrade (`X-Cortiq-Tools-Dropped`) instead of failing.
|
||||
let mut tools_dropped = false;
|
||||
|
||||
// A pin stays a pin: when the pinned model exists and merely lacks tool
|
||||
// support, honour the pin and drop the offer rather than reporting the
|
||||
// model as missing.
|
||||
if final_candidates.is_empty() && needs_tools && !tools_mandatory {
|
||||
if let RoutingDirective::Pinned { model_id } = &req.routing {
|
||||
if live.registry.get(model_id).is_some() {
|
||||
final_candidates.push(model_id.clone());
|
||||
tools_dropped = true;
|
||||
tracing::info!(
|
||||
model = %model_id,
|
||||
"pinned model does not support the offered tools — serving without them"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An explicit pin must never silently answer from a different model: if
|
||||
// the pinned id is unknown or was filtered out, fail loudly so the caller
|
||||
// knows their choice was not honoured (instead of getting a stray reply
|
||||
@@ -285,25 +338,86 @@ impl Pipeline {
|
||||
if final_candidates.is_empty() && live.cfg.cmf.manage_server {
|
||||
for id in live.local_candidates() {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Capability widening: a tier chain built of tools-less local models
|
||||
// must not kill an agent request. When the filtered chain is empty on
|
||||
// an AUTO route, widen to any pool model that satisfies the required
|
||||
// capabilities — preferring the routing table's own medium→high order
|
||||
// (an agentic request is worth a capable model), then the rest of the
|
||||
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
|
||||
// this every low-tier verdict was a hard failure.
|
||||
let widened_order = || -> Vec<String> {
|
||||
let mut widened: Vec<String> = Vec::new();
|
||||
for tier in ["medium", "high"] {
|
||||
for id in live.routing.candidates(tier) {
|
||||
if !widened.contains(&id) {
|
||||
widened.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in live.registry.ids() {
|
||||
if !widened.contains(id) {
|
||||
widened.push(id.clone());
|
||||
}
|
||||
}
|
||||
widened
|
||||
};
|
||||
if final_candidates.is_empty() {
|
||||
for id in widened_order() {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !has_tools(&id, &prov) {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
if !final_candidates.is_empty() {
|
||||
tracing::info!(
|
||||
chosen = %final_candidates[0],
|
||||
"capability widening: tier chain had no model with the required capabilities"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Same relaxation for an AUTO route once every narrower pass came up empty.
|
||||
if final_candidates.is_empty() && needs_tools && !tools_mandatory {
|
||||
for id in widened_order() {
|
||||
if live.registry.get(&id).is_some() {
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
if !final_candidates.is_empty() {
|
||||
tools_dropped = true;
|
||||
tracing::info!(
|
||||
chosen = %final_candidates[0],
|
||||
"no model in the pool supports the offered tools — serving without them"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if final_candidates.is_empty() {
|
||||
return Err(GatewayError::UpstreamUnavailable(
|
||||
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
Ok((decision, route_source, final_candidates))
|
||||
Ok((decision, route_source, final_candidates, tools_dropped))
|
||||
}
|
||||
|
||||
async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
||||
async fn run_inner(&self, mut req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
|
||||
let live = state.live();
|
||||
let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, mut final_candidates, tools_dropped) =
|
||||
self.resolve(&req, &live, &detected_tools).await?;
|
||||
if tools_dropped {
|
||||
strip_tools(&mut req);
|
||||
}
|
||||
|
||||
// Serving-when-promoted (gated): a promoted task-label is served by the
|
||||
// local model FIRST (failover to cloud on error). Per-request VETO uses
|
||||
@@ -341,7 +455,66 @@ impl Pipeline {
|
||||
route_source
|
||||
);
|
||||
|
||||
match provider.chat(req.clone()).await {
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat(req_call).await {
|
||||
Ok(mut resp) => {
|
||||
let cost =
|
||||
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
|
||||
@@ -354,6 +527,7 @@ impl Pipeline {
|
||||
router_request_id: decision.router_request_id.clone(),
|
||||
cost_usd: cost,
|
||||
failover: attempt > 0,
|
||||
tools_dropped,
|
||||
};
|
||||
return Ok(resp);
|
||||
}
|
||||
@@ -380,7 +554,7 @@ impl Pipeline {
|
||||
/// recorded when the stream completes (tokens taken from the final usage chunk).
|
||||
pub async fn run_stream(
|
||||
&self,
|
||||
req: ChatRequest,
|
||||
mut req: ChatRequest,
|
||||
state: &SharedState,
|
||||
) -> Result<(RouteInfo, crate::providers::ChatStream)> {
|
||||
let started = Instant::now();
|
||||
@@ -393,13 +567,18 @@ impl Pipeline {
|
||||
.to_string();
|
||||
|
||||
let live = state.live();
|
||||
let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
|
||||
let (decision, route_source, final_candidates, tools_dropped) =
|
||||
match self.resolve(&req, &live, &detected_tools).await {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
record_error(state, &account, &protocol, &directive, started, &e);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
if tools_dropped {
|
||||
strip_tools(&mut req);
|
||||
}
|
||||
|
||||
let mut last_err = None;
|
||||
let mut attempt = 0usize;
|
||||
@@ -415,7 +594,67 @@ impl Pipeline {
|
||||
model_id,
|
||||
route_source
|
||||
);
|
||||
match provider.chat_stream(req.clone()).await {
|
||||
// Merge per-model/server config (same as run_inner).
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat_stream(req_call).await {
|
||||
Ok(stream) => {
|
||||
let info = RouteInfo {
|
||||
task_label: decision.task_label.clone(),
|
||||
@@ -426,7 +665,13 @@ impl Pipeline {
|
||||
router_request_id: decision.router_request_id.clone(),
|
||||
cost_usd: 0.0,
|
||||
failover: attempt > 0,
|
||||
tools_dropped,
|
||||
};
|
||||
// for the client-facing usage estimate when the provider
|
||||
// sends none: the prompt size is known here, the reply is
|
||||
// measured inside the tap
|
||||
let prompt_chars: usize =
|
||||
req.messages.iter().map(|m| m.content.chars().count()).sum();
|
||||
let tapped = tap_stream(
|
||||
stream,
|
||||
state.stats.clone(),
|
||||
@@ -436,6 +681,7 @@ impl Pipeline {
|
||||
protocol,
|
||||
directive,
|
||||
started,
|
||||
prompt_chars,
|
||||
);
|
||||
return Ok((info, Box::pin(tapped)));
|
||||
}
|
||||
@@ -497,6 +743,26 @@ impl Pipeline {
|
||||
}
|
||||
}
|
||||
|
||||
/// Remove the tool offer from a request the chosen model cannot honour. The
|
||||
/// companion fields must go too: an upstream given `tool_choice` with no
|
||||
/// `tools` answers 400.
|
||||
fn strip_tools(req: &mut ChatRequest) {
|
||||
req.tools.clear();
|
||||
req.params.passthrough.remove("tool_choice");
|
||||
req.params.passthrough.remove("parallel_tool_calls");
|
||||
}
|
||||
|
||||
/// Is the client's `tool_choice` a demand rather than a preference?
|
||||
/// `"required"` and `{"type":"function", …}` mean the caller cannot be served
|
||||
/// without tools; `"auto"`, `"none"` and an absent field are just an offer.
|
||||
fn tool_choice_is_mandatory(passthrough: &serde_json::Map<String, serde_json::Value>) -> bool {
|
||||
match passthrough.get("tool_choice") {
|
||||
Some(serde_json::Value::String(s)) => s.eq_ignore_ascii_case("required"),
|
||||
Some(serde_json::Value::Object(_)) => true,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn now_secs() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
@@ -546,6 +812,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
|
||||
role: "assistant".into(),
|
||||
content: hit.content.clone(),
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
},
|
||||
finish_reason: "stop".into(),
|
||||
}],
|
||||
@@ -563,6 +830,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
|
||||
router_request_id: None,
|
||||
cost_usd: 0.0,
|
||||
failover: false,
|
||||
tools_dropped: false,
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -607,6 +875,7 @@ fn tap_stream(
|
||||
protocol: String,
|
||||
directive: String,
|
||||
started: Instant,
|
||||
prompt_chars: usize,
|
||||
) -> impl futures::Stream<Item = Result<bytes::Bytes>> {
|
||||
use futures::StreamExt;
|
||||
async_stream::stream! {
|
||||
@@ -616,12 +885,15 @@ fn tap_stream(
|
||||
let mut completion_tokens = 0u32;
|
||||
let mut got_usage = false;
|
||||
let mut delta_chars = 0usize;
|
||||
let mut delta_chunks = 0u32;
|
||||
let mut errored: Option<String> = None;
|
||||
|
||||
while let Some(item) = inner.next().await {
|
||||
let mut done_marker = false;
|
||||
match &item {
|
||||
Ok(bytes) => {
|
||||
if let Ok(s) = std::str::from_utf8(bytes) {
|
||||
done_marker = s.contains("data: [DONE]");
|
||||
buf.push_str(s);
|
||||
while let Some(idx) = buf.find("\n\n") {
|
||||
let evt: String = buf.drain(..idx + 2).collect();
|
||||
@@ -650,6 +922,7 @@ fn tap_stream(
|
||||
v["choices"][0]["delta"]["content"].as_str()
|
||||
{
|
||||
delta_chars += c.chars().count();
|
||||
delta_chunks += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -659,12 +932,52 @@ fn tap_stream(
|
||||
}
|
||||
Err(e) => errored = Some(e.to_string()),
|
||||
}
|
||||
// The local cortiq server (and some providers) never send a usage
|
||||
// chunk. Clients still deserve token counts, so ahead of [DONE] we
|
||||
// inject one final OpenAI-shaped usage event with the gateway's
|
||||
// estimate (~4 chars/token), flagged `cortiq_estimated` so nobody
|
||||
// mistakes it for the tokenizer's truth.
|
||||
if done_marker {
|
||||
// The initial X-Cortiq-Cost-USD header is always 0 — cost is
|
||||
// only known after generation. Ship the final number (and the
|
||||
// token estimate when the provider sent none) ahead of [DONE].
|
||||
if !got_usage {
|
||||
prompt_tokens = (prompt_chars / 4) as u32;
|
||||
completion_tokens = if delta_chunks > 0 {
|
||||
delta_chunks
|
||||
} else {
|
||||
(delta_chars / 4) as u32
|
||||
};
|
||||
}
|
||||
let cost = provider.price(prompt_tokens, completion_tokens);
|
||||
let mut evt = serde_json::json!({
|
||||
"object": "chat.completion.chunk",
|
||||
"model": info.selected_model,
|
||||
"choices": [],
|
||||
"cortiq_cost_usd": cost,
|
||||
});
|
||||
if !got_usage {
|
||||
evt["usage"] = serde_json::json!({
|
||||
"prompt_tokens": prompt_tokens,
|
||||
"completion_tokens": completion_tokens,
|
||||
"total_tokens": prompt_tokens + completion_tokens,
|
||||
});
|
||||
evt["cortiq_estimated"] = serde_json::json!(true);
|
||||
got_usage = true;
|
||||
}
|
||||
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
|
||||
}
|
||||
yield item;
|
||||
}
|
||||
|
||||
if !got_usage && completion_tokens == 0 {
|
||||
// estimate when the server didn't return usage (~4 chars per token)
|
||||
completion_tokens = (delta_chars / 4) as u32;
|
||||
// stream ended without a [DONE] marker — estimate for the stats
|
||||
completion_tokens = if delta_chunks > 0 {
|
||||
delta_chunks
|
||||
} else {
|
||||
(delta_chars / 4) as u32
|
||||
};
|
||||
prompt_tokens = (prompt_chars / 4) as u32;
|
||||
}
|
||||
let cost = provider.price(prompt_tokens, completion_tokens);
|
||||
stats.record(RequestRecord {
|
||||
@@ -723,6 +1036,7 @@ fn shadow_req(model_id: &str, content: String, max_tokens: u32) -> ChatRequest {
|
||||
role: "user".to_string(),
|
||||
content,
|
||||
tool_calls: Vec::new(),
|
||||
..Default::default()
|
||||
}],
|
||||
tools: Vec::new(),
|
||||
params: GenParams {
|
||||
@@ -828,3 +1142,57 @@ async fn shadow_capture(
|
||||
judge_tokens,
|
||||
});
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn map(json: serde_json::Value) -> serde_json::Map<String, serde_json::Value> {
|
||||
json.as_object().cloned().unwrap_or_default()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_offered_tool_list_is_not_a_demand() {
|
||||
// Open WebUI attaches its built-in tools with no tool_choice at all.
|
||||
assert!(!tool_choice_is_mandatory(&map(serde_json::json!({}))));
|
||||
assert!(!tool_choice_is_mandatory(&map(
|
||||
serde_json::json!({"tool_choice": "auto"})
|
||||
)));
|
||||
assert!(!tool_choice_is_mandatory(&map(
|
||||
serde_json::json!({"tool_choice": "none"})
|
||||
)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn required_and_named_tool_choice_are_demands() {
|
||||
assert!(tool_choice_is_mandatory(&map(
|
||||
serde_json::json!({"tool_choice": "required"})
|
||||
)));
|
||||
assert!(tool_choice_is_mandatory(&map(
|
||||
serde_json::json!({"tool_choice": {"type": "function", "function": {"name": "f"}}})
|
||||
)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stripping_tools_also_drops_their_companion_fields() {
|
||||
let mut req = ChatRequest {
|
||||
routing: RoutingDirective::Auto { profile: None },
|
||||
messages: vec![],
|
||||
tools: vec![serde_json::json!({"type": "function"})],
|
||||
params: GenParams {
|
||||
passthrough: map(
|
||||
serde_json::json!({"tool_choice": "auto", "parallel_tool_calls": true, "top_k": 40}),
|
||||
),
|
||||
..Default::default()
|
||||
},
|
||||
stream: false,
|
||||
meta: RequestMeta::default(),
|
||||
};
|
||||
strip_tools(&mut req);
|
||||
assert!(req.tools.is_empty());
|
||||
assert!(!req.params.passthrough.contains_key("tool_choice"));
|
||||
assert!(!req.params.passthrough.contains_key("parallel_tool_calls"));
|
||||
// unrelated passthrough must survive
|
||||
assert!(req.params.passthrough.contains_key("top_k"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
|
||||
use crate::protocols::openai_chat::cortiq_headers;
|
||||
use crate::protocols::openai_chat::{cortiq_headers, sse_headers};
|
||||
use crate::state::SharedState;
|
||||
use axum::response::{IntoResponse, Response};
|
||||
use axum::{extract::State, routing::post, Json, Router};
|
||||
@@ -63,6 +63,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<MessagesRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.anthropic_messages {
|
||||
@@ -83,6 +84,7 @@ async fn handler(
|
||||
role: "system".into(),
|
||||
content: sys,
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
});
|
||||
}
|
||||
for m in &req.messages {
|
||||
@@ -90,9 +92,12 @@ async fn handler(
|
||||
role: m["role"].as_str().unwrap_or("user").to_string(),
|
||||
content: content_to_string(&m["content"]),
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
});
|
||||
}
|
||||
|
||||
// Echoed back as-is: clients match the reply against the id they asked for.
|
||||
let requested_model = req.model.clone();
|
||||
let canonical = ChatRequest {
|
||||
routing: parse_routing(&req.model),
|
||||
messages,
|
||||
@@ -101,25 +106,26 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "anthropic_messages".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
if req.stream {
|
||||
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let mut headers = cortiq_headers(&info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
axum::http::HeaderValue::from_static("text/event-stream"),
|
||||
);
|
||||
let translated = openai_to_anthropic_sse(stream);
|
||||
return Ok((headers, axum::body::Body::from_stream(translated)).into_response());
|
||||
let translated = openai_to_anthropic_sse(stream, requested_model);
|
||||
return Ok((
|
||||
sse_headers(&info),
|
||||
axum::body::Body::from_stream(translated),
|
||||
)
|
||||
.into_response());
|
||||
}
|
||||
|
||||
let resp = state.pipeline.run(canonical, &state).await?;
|
||||
@@ -133,7 +139,7 @@ async fn handler(
|
||||
"id": resp.id,
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": resp.model_used,
|
||||
"model": requested_model,
|
||||
"content": [{ "type": "text", "text": answer }],
|
||||
"stop_reason": "end_turn",
|
||||
"stop_sequence": serde_json::Value::Null,
|
||||
@@ -149,6 +155,7 @@ async fn handler(
|
||||
/// events, so Anthropic-native clients can stream regardless of the upstream provider.
|
||||
fn openai_to_anthropic_sse(
|
||||
stream: crate::providers::ChatStream,
|
||||
model: String,
|
||||
) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
|
||||
use futures::StreamExt;
|
||||
async_stream::stream! {
|
||||
@@ -159,7 +166,7 @@ fn openai_to_anthropic_sse(
|
||||
yield Ok(evt("message_start", serde_json::json!({
|
||||
"type": "message_start",
|
||||
"message": {"id": "msg_gw", "type": "message", "role": "assistant",
|
||||
"content": [], "model": "cortiq-auto", "stop_reason": null,
|
||||
"content": [], "model": model, "stop_reason": null,
|
||||
"usage": {"input_tokens": 0, "output_tokens": 0}}
|
||||
})));
|
||||
yield Ok(evt("content_block_start", serde_json::json!({
|
||||
|
||||
@@ -94,6 +94,7 @@ async fn handle_tool_call(
|
||||
role: "user".into(),
|
||||
content: prompt.to_string(),
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
}],
|
||||
tools: vec![],
|
||||
params: GenParams::default(),
|
||||
|
||||
+51
-1
@@ -12,10 +12,59 @@ pub mod openai_models;
|
||||
use crate::state::SharedState;
|
||||
use axum::Router;
|
||||
|
||||
/// Account resolved by the API-key middleware, consumed by protocol handlers
|
||||
/// for attribution (stats, budgets). Present only when keys are configured.
|
||||
#[derive(Clone)]
|
||||
pub struct AccountTag(pub String);
|
||||
|
||||
/// API-key gate for the model-facing surface. Open mode (no keys configured)
|
||||
/// passes everything through — the admin console already warns about it.
|
||||
/// With keys: Bearer/x-api-key must match (401), per-key rate_per_min is
|
||||
/// enforced with a fixed one-minute window (429 + Retry-After), and a
|
||||
/// monthly budget, when set, turns into 429 insufficient_quota once the
|
||||
/// account's month-to-date spend crosses it.
|
||||
async fn api_auth(
|
||||
axum::extract::State(state): axum::extract::State<SharedState>,
|
||||
mut req: axum::extract::Request,
|
||||
next: axum::middleware::Next,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
let live = state.live();
|
||||
if live.cfg.api_keys.is_empty() {
|
||||
return next.run(req).await;
|
||||
}
|
||||
let provided = req
|
||||
.headers()
|
||||
.get(axum::http::header::AUTHORIZATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
.or_else(|| req.headers().get("x-api-key").and_then(|v| v.to_str().ok()))
|
||||
.map(str::trim)
|
||||
.unwrap_or("");
|
||||
let Some(key) = live.cfg.api_keys.iter().find(|k| k.key == provided) else {
|
||||
return crate::error::GatewayError::Unauthorized("invalid or missing API key".into())
|
||||
.into_response();
|
||||
};
|
||||
if key.rate_per_min > 0 && !state.rate.allow(&key.key, key.rate_per_min) {
|
||||
return crate::error::GatewayError::RateLimited {
|
||||
retry_after_secs: state.rate.retry_after_secs(),
|
||||
}
|
||||
.into_response();
|
||||
}
|
||||
if let Some(budget) = key.monthly_budget_usd {
|
||||
if budget > 0.0 && state.stats.month_spend(&key.account) >= budget {
|
||||
return crate::error::GatewayError::BudgetExhausted { budget_usd: budget }
|
||||
.into_response();
|
||||
}
|
||||
}
|
||||
req.extensions_mut().insert(AccountTag(key.account.clone()));
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
/// Build the protocols router. Implemented adapters are **always** mounted;
|
||||
/// each handler checks the live `protocols.*` flag from config itself, so
|
||||
/// toggles in the admin panel take effect without a restart (hot switching).
|
||||
pub fn build_router() -> Router<SharedState> {
|
||||
pub fn build_router(state: SharedState) -> Router<SharedState> {
|
||||
Router::new()
|
||||
.merge(openai_chat::routes())
|
||||
.merge(openai_completions::routes())
|
||||
@@ -24,4 +73,5 @@ pub fn build_router() -> Router<SharedState> {
|
||||
.merge(anthropic_messages::routes())
|
||||
.merge(mcp::routes())
|
||||
.merge(native_passthrough::routes())
|
||||
.route_layer(axum::middleware::from_fn_with_state(state, api_auth))
|
||||
}
|
||||
|
||||
+573
-40
@@ -1,6 +1,17 @@
|
||||
//! Incoming adapter for OpenAI Chat Completions: `POST /v1/chat/completions`.
|
||||
//! Translates the OpenAI request body into a canonical [`ChatRequest`], passes it
|
||||
//! to the pipeline, then converts the canonical [`ChatResponse`] back to OpenAI format.
|
||||
//!
|
||||
//! Wire-compatibility rules this adapter enforces (see docs/PROTOCOLS.md §2):
|
||||
//! * the request body is parsed leniently — every shape a real OpenAI client
|
||||
//! sends (`content: null`, content-part arrays, `tools: null`, a string
|
||||
//! `reasoning_effort`, UI-private extras) is accepted, and a body we truly
|
||||
//! cannot read comes back as an OpenAI error envelope, never a bare 422;
|
||||
//! * the response is strict OpenAI JSON — no optional `null` fields, a
|
||||
//! spec-legal `finish_reason`, and the *requested* model id echoed back;
|
||||
//! * the SSE stream is re-normalised chunk by chunk and always terminates
|
||||
//! with `data: [DONE]`, so a client can never sit waiting on a half-formed
|
||||
//! stream from an upstream that improvises.
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RouteInfo, RoutingDirective};
|
||||
@@ -31,31 +42,119 @@ pub(crate) fn cortiq_headers(c: &RouteInfo) -> axum::http::HeaderMap {
|
||||
if let Some(id) = &c.router_request_id {
|
||||
put(&mut h, "X-Cortiq-Request-Id", id);
|
||||
}
|
||||
if c.tools_dropped {
|
||||
put(&mut h, "X-Cortiq-Tools-Dropped", "true");
|
||||
}
|
||||
h
|
||||
}
|
||||
|
||||
/// Headers every SSE response carries. `X-Accel-Buffering: no` matters when the
|
||||
/// gateway sits behind nginx/Traefik — without it the proxy buffers the whole
|
||||
/// stream and the client's chat window just spins.
|
||||
pub(crate) fn sse_headers(info: &RouteInfo) -> axum::http::HeaderMap {
|
||||
use axum::http::HeaderValue;
|
||||
let mut headers = cortiq_headers(info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
HeaderValue::from_static("text/event-stream; charset=utf-8"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::CACHE_CONTROL,
|
||||
HeaderValue::from_static("no-cache"),
|
||||
);
|
||||
headers.insert("X-Accel-Buffering", HeaderValue::from_static("no"));
|
||||
headers
|
||||
}
|
||||
|
||||
pub fn routes() -> Router<SharedState> {
|
||||
Router::new().route("/v1/chat/completions", post(handler))
|
||||
}
|
||||
|
||||
/// `reasoning_effort` is a string in the OpenAI spec (`low` | `medium` | `high`)
|
||||
/// but a token budget in several local runtimes. Accept both.
|
||||
#[derive(Deserialize)]
|
||||
#[serde(untagged)]
|
||||
enum Effort {
|
||||
Budget(u32),
|
||||
Level(String),
|
||||
}
|
||||
|
||||
impl Effort {
|
||||
fn to_budget(&self) -> Option<u32> {
|
||||
match self {
|
||||
Self::Budget(n) => Some(*n),
|
||||
Self::Level(s) => match s.trim().to_ascii_lowercase().as_str() {
|
||||
"none" | "off" | "minimal" => Some(0),
|
||||
"low" => Some(512),
|
||||
"medium" | "default" | "auto" => Some(2048),
|
||||
"high" | "max" => Some(8192),
|
||||
_ => None,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct OpenAiChatRequest {
|
||||
model: String,
|
||||
model: Option<String>,
|
||||
messages: Vec<Message>,
|
||||
#[serde(default)]
|
||||
temperature: Option<f32>,
|
||||
#[serde(default)]
|
||||
max_tokens: Option<u32>,
|
||||
/// The modern spelling of `max_tokens`; clients send one or the other.
|
||||
#[serde(default)]
|
||||
max_completion_tokens: Option<u32>,
|
||||
#[serde(default)]
|
||||
top_p: Option<f32>,
|
||||
#[serde(default)]
|
||||
stream: bool,
|
||||
think_budget: Option<u32>,
|
||||
#[serde(default)]
|
||||
tools: Vec<serde_json::Value>,
|
||||
reasoning_effort: Option<Effort>,
|
||||
/// `null` is a legal value here for several clients, hence `Option`.
|
||||
#[serde(default)]
|
||||
stream: Option<bool>,
|
||||
#[serde(default)]
|
||||
tools: Option<Vec<serde_json::Value>>,
|
||||
#[serde(flatten)]
|
||||
rest: serde_json::Map<String, serde_json::Value>,
|
||||
}
|
||||
|
||||
/// Keys chat UIs attach to the payload for their own bookkeeping. They are not
|
||||
/// part of the OpenAI request schema, and forwarding them to a strict upstream
|
||||
/// (llama.cpp, vLLM, `cortiq serve`) is a needless way to earn a 400.
|
||||
const CLIENT_PRIVATE_KEYS: &[&str] = &[
|
||||
"chat_id",
|
||||
"session_id",
|
||||
"id",
|
||||
"metadata",
|
||||
"background_tasks",
|
||||
"features",
|
||||
"variables",
|
||||
"model_item",
|
||||
"tool_ids",
|
||||
"filter_ids",
|
||||
"files",
|
||||
"citations",
|
||||
"params",
|
||||
"direct",
|
||||
];
|
||||
|
||||
/// Strip client-private bookkeeping from the passthrough map, and drop a
|
||||
/// non-string `user` (Open WebUI sends an object there for pipelines, which the
|
||||
/// OpenAI schema does not allow).
|
||||
fn sanitize_passthrough(
|
||||
mut rest: serde_json::Map<String, serde_json::Value>,
|
||||
) -> serde_json::Map<String, serde_json::Value> {
|
||||
for k in CLIENT_PRIVATE_KEYS {
|
||||
rest.remove(*k);
|
||||
}
|
||||
if rest.get("user").map(|u| !u.is_string()).unwrap_or(false) {
|
||||
rest.remove("user");
|
||||
}
|
||||
rest
|
||||
}
|
||||
|
||||
/// Parse the `model` field value into a routing directive.
|
||||
/// `cortiq-auto[:profile]` → Auto; otherwise → Pinned(real id).
|
||||
fn parse_routing(model: &str) -> RoutingDirective {
|
||||
@@ -69,9 +168,43 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
}
|
||||
}
|
||||
|
||||
fn now_secs() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Map whatever an upstream calls its stop reason onto the closed set the
|
||||
/// OpenAI schema defines. Clients that validate the field (LiteLLM, the
|
||||
/// official SDKs, several agent frameworks) reject anything else.
|
||||
pub(crate) fn normalize_finish_reason(raw: &str) -> &'static str {
|
||||
match raw.trim().to_ascii_lowercase().as_str() {
|
||||
"length" | "max_tokens" | "max_new_tokens" | "token_limit" => "length",
|
||||
"tool_calls" | "tool_use" | "function_call" => "tool_calls",
|
||||
"content_filter" | "safety" => "content_filter",
|
||||
_ => "stop",
|
||||
}
|
||||
}
|
||||
|
||||
/// A response id every client can key on. Upstreams that omit `id` (or send an
|
||||
/// empty one) would otherwise leave the field blank, which some clients treat
|
||||
/// as a malformed completion.
|
||||
fn response_id(upstream: &str) -> String {
|
||||
let trimmed = upstream.trim();
|
||||
if !trimmed.is_empty() {
|
||||
return trimmed.to_string();
|
||||
}
|
||||
format!("chatcmpl-{}", crate::admin::random_token(12))
|
||||
}
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
Json(req): Json<OpenAiChatRequest>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
// Taken as raw bytes rather than `Json<T>`: axum's extractor rejection is a
|
||||
// plain-text 422 that OpenAI clients cannot parse, and a client that cannot
|
||||
// parse the failure has no answer to show — it just hangs.
|
||||
body: axum::body::Bytes,
|
||||
) -> Result<Response> {
|
||||
// hot protocol toggle: if the adapter is disabled in config — return 404
|
||||
if !state.live().cfg.protocols.openai_chat {
|
||||
@@ -79,67 +212,95 @@ async fn handler(
|
||||
"openai_chat protocol is disabled".into(),
|
||||
));
|
||||
}
|
||||
let req: OpenAiChatRequest = serde_json::from_slice(&body)
|
||||
.map_err(|e| GatewayError::InvalidRequest(format!("invalid request body: {e}")))?;
|
||||
|
||||
if req.messages.is_empty() {
|
||||
return Err(GatewayError::InvalidRequest(
|
||||
"messages must not be empty".into(),
|
||||
));
|
||||
}
|
||||
|
||||
// Echoed back verbatim: OpenAI returns the model the caller asked for, and
|
||||
// clients (Open WebUI among them) match the reply against their own id. The
|
||||
// model that actually answered travels in `cortiq.selected_model` and the
|
||||
// `X-Cortiq-Selected-Model` header.
|
||||
let requested_model = req
|
||||
.model
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|m| !m.is_empty())
|
||||
.unwrap_or("cortiq-auto")
|
||||
.to_string();
|
||||
let stream = req.stream.unwrap_or(false);
|
||||
|
||||
let canonical = ChatRequest {
|
||||
routing: parse_routing(&req.model),
|
||||
routing: parse_routing(&requested_model),
|
||||
messages: req.messages,
|
||||
tools: req.tools,
|
||||
tools: req.tools.unwrap_or_default(),
|
||||
params: GenParams {
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
max_tokens: req.max_tokens.or(req.max_completion_tokens),
|
||||
top_p: req.top_p,
|
||||
think_budget: req
|
||||
.think_budget
|
||||
.or_else(|| req.reasoning_effort.as_ref().and_then(Effort::to_budget)),
|
||||
stop: Vec::new(),
|
||||
passthrough: req.rest,
|
||||
passthrough: sanitize_passthrough(req.rest),
|
||||
},
|
||||
stream: req.stream,
|
||||
stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_chat".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
// streaming: forward provider SSE chunks verbatim with X-Cortiq-* headers
|
||||
if req.stream {
|
||||
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let mut headers = cortiq_headers(&info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
axum::http::HeaderValue::from_static("text/event-stream"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::CACHE_CONTROL,
|
||||
axum::http::HeaderValue::from_static("no-cache"),
|
||||
);
|
||||
return Ok((headers, axum::body::Body::from_stream(stream)).into_response());
|
||||
// streaming: provider SSE is re-normalised into strict OpenAI chunks
|
||||
if stream {
|
||||
let (info, raw) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let headers = sse_headers(&info);
|
||||
let normalized = normalize_sse(raw, requested_model);
|
||||
return Ok((headers, axum::body::Body::from_stream(normalized)).into_response());
|
||||
}
|
||||
|
||||
let resp = state.pipeline.run(canonical, &state).await?;
|
||||
let headers = cortiq_headers(&resp.cortiq);
|
||||
|
||||
let mut body = serde_json::json!({
|
||||
"id": resp.id,
|
||||
"object": "chat.completion",
|
||||
"created": std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs(),
|
||||
"model": resp.model_used,
|
||||
"choices": resp.choices.iter().map(|c| {
|
||||
let choices: Vec<serde_json::Value> = resp
|
||||
.choices
|
||||
.iter()
|
||||
.map(|c| {
|
||||
let mut message = serde_json::Map::new();
|
||||
message.insert("role".into(), serde_json::json!(c.message.role));
|
||||
message.insert("content".into(), serde_json::json!(c.message.content));
|
||||
// Absent tool calls are omitted entirely — `"tool_calls": null` is
|
||||
// not in the schema and trips strict clients.
|
||||
if !c.message.tool_calls.is_empty() {
|
||||
message.insert(
|
||||
"tool_calls".into(),
|
||||
serde_json::Value::Array(c.message.tool_calls.clone()),
|
||||
);
|
||||
}
|
||||
let finish = if c.message.tool_calls.is_empty() {
|
||||
normalize_finish_reason(&c.finish_reason)
|
||||
} else {
|
||||
"tool_calls"
|
||||
};
|
||||
serde_json::json!({
|
||||
"index": c.index,
|
||||
"message": {
|
||||
"role": c.message.role,
|
||||
"content": c.message.content,
|
||||
"tool_calls": if c.message.tool_calls.is_empty() { serde_json::Value::Null } else { serde_json::Value::Array(c.message.tool_calls.clone()) },
|
||||
},
|
||||
"finish_reason": c.finish_reason,
|
||||
"message": serde_json::Value::Object(message),
|
||||
"finish_reason": finish,
|
||||
})
|
||||
}).collect::<Vec<_>>(),
|
||||
})
|
||||
.collect();
|
||||
|
||||
let mut body = serde_json::json!({
|
||||
"id": response_id(&resp.id),
|
||||
"object": "chat.completion",
|
||||
"created": now_secs(),
|
||||
"model": requested_model,
|
||||
"choices": choices,
|
||||
"usage": {
|
||||
"prompt_tokens": resp.usage.prompt_tokens,
|
||||
"completion_tokens": resp.usage.completion_tokens,
|
||||
@@ -148,7 +309,7 @@ async fn handler(
|
||||
});
|
||||
|
||||
if state.live().cfg.cortiq.echo {
|
||||
body["cortiq"] = serde_json::json!({
|
||||
let mut echo = serde_json::json!({
|
||||
"task_label": resp.cortiq.task_label,
|
||||
"complexity": {
|
||||
"score": resp.cortiq.complexity_score,
|
||||
@@ -156,10 +317,382 @@ async fn handler(
|
||||
},
|
||||
"selected_model": resp.cortiq.selected_model,
|
||||
"route_source": resp.cortiq.route_source,
|
||||
"router_request_id": resp.cortiq.router_request_id,
|
||||
"cost_usd": resp.cortiq.cost_usd,
|
||||
});
|
||||
if let Some(rid) = &resp.cortiq.router_request_id {
|
||||
echo["router_request_id"] = serde_json::json!(rid);
|
||||
}
|
||||
if resp.cortiq.tools_dropped {
|
||||
echo["tools_dropped"] = serde_json::json!(true);
|
||||
}
|
||||
body["cortiq"] = echo;
|
||||
}
|
||||
|
||||
Ok((headers, Json(body)).into_response())
|
||||
}
|
||||
|
||||
/// Rewrite an upstream SSE stream into strict OpenAI `chat.completion.chunk`
|
||||
/// events.
|
||||
///
|
||||
/// Every chunk is reissued with an id, `created`, and the *requested* model id;
|
||||
/// `message`-shaped chunks are converted to `delta`; stop reasons are mapped to
|
||||
/// the spec's closed set; empty deltas are dropped. The stream is guaranteed to
|
||||
/// open with a `role` delta, to carry exactly one terminal `finish_reason`, and
|
||||
/// to end with `data: [DONE]` — even when the upstream dies mid-flight, in
|
||||
/// which case the failure is delivered as an SSE `error` event so the client
|
||||
/// shows a message instead of waiting forever.
|
||||
///
|
||||
/// Tool calls are settled here too (see [`crate::toolcalls`]): raw
|
||||
/// `<tool_call>` markup never reaches `content`, structured deltas are brought
|
||||
/// to the OpenAI shape, markup is promoted to a real call when the upstream
|
||||
/// produced none, and a stream that emitted tool calls always finishes with
|
||||
/// `finish_reason: "tool_calls"`.
|
||||
fn normalize_sse(
|
||||
stream: crate::providers::ChatStream,
|
||||
model: String,
|
||||
) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
|
||||
use futures::StreamExt;
|
||||
async_stream::stream! {
|
||||
let fallback_id = response_id("");
|
||||
let created = now_secs();
|
||||
let mut buf = String::new();
|
||||
let mut id: Option<String> = None;
|
||||
// Which choice indices already carried a `role` delta (n > 1 gets one each).
|
||||
let mut roled: std::collections::HashSet<u64> = std::collections::HashSet::new();
|
||||
let mut sent_finish = false;
|
||||
let mut failure: Option<String> = None;
|
||||
// Tool-call hygiene: one markup filter per choice index, the set of
|
||||
// tool-call indices already opened, and whether anything structured
|
||||
// was emitted at all.
|
||||
let mut markup: std::collections::HashMap<u64, crate::toolcalls::ToolMarkupFilter> =
|
||||
std::collections::HashMap::new();
|
||||
let mut tool_indices: std::collections::HashSet<u64> = std::collections::HashSet::new();
|
||||
let mut emitted_tool_calls = false;
|
||||
|
||||
// Emit one normalised event; `None` means "nothing worth sending".
|
||||
macro_rules! chunk {
|
||||
($choices:expr, $extra:expr) => {{
|
||||
let mut evt = serde_json::json!({
|
||||
"id": id.clone().unwrap_or_else(|| fallback_id.clone()),
|
||||
"object": "chat.completion.chunk",
|
||||
"created": created,
|
||||
"model": model,
|
||||
"choices": $choices,
|
||||
});
|
||||
for (k, v) in $extra {
|
||||
evt[k] = v;
|
||||
}
|
||||
bytes::Bytes::from(format!("data: {evt}\n\n"))
|
||||
}};
|
||||
}
|
||||
|
||||
futures::pin_mut!(stream);
|
||||
'outer: while let Some(item) = stream.next().await {
|
||||
let bytes = match item {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
failure = Some(e.to_string());
|
||||
break 'outer;
|
||||
}
|
||||
};
|
||||
buf.push_str(&String::from_utf8_lossy(&bytes));
|
||||
|
||||
while let Some(idx) = buf.find("\n\n") {
|
||||
let event: String = buf.drain(..idx + 2).collect();
|
||||
for line in event.lines() {
|
||||
let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
|
||||
let data = data.trim();
|
||||
if data.is_empty() || data == "[DONE]" {
|
||||
continue;
|
||||
}
|
||||
let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
|
||||
|
||||
// An upstream error delivered inside the stream: pass it on
|
||||
// and stop — the client needs to see it, not time out.
|
||||
if let Some(err) = v.get("error").filter(|e| !e.is_null()) {
|
||||
failure = Some(err.to_string());
|
||||
break 'outer;
|
||||
}
|
||||
|
||||
if id.is_none() {
|
||||
id = v["id"].as_str().filter(|s| !s.is_empty()).map(str::to_string);
|
||||
}
|
||||
|
||||
let mut choices = Vec::new();
|
||||
if let Some(arr) = v["choices"].as_array() {
|
||||
for (i, c) in arr.iter().enumerate() {
|
||||
// Some upstreams send a full `message` per chunk
|
||||
// instead of a `delta`; both mean the same thing here.
|
||||
let src = if c["delta"].is_object() { &c["delta"] } else { &c["message"] };
|
||||
let index = c["index"].as_u64().unwrap_or(i as u64);
|
||||
let mut delta = serde_json::Map::new();
|
||||
if roled.insert(index) {
|
||||
delta.insert("role".into(), serde_json::json!("assistant"));
|
||||
}
|
||||
if let Some(text) = src["content"].as_str() {
|
||||
// Strip any `<tool_call>` markup before it can
|
||||
// reach the client as prose — the structured
|
||||
// call is the only form a tool loop may see.
|
||||
let visible = markup
|
||||
.entry(index)
|
||||
.or_default()
|
||||
.push(text);
|
||||
if !visible.is_empty() {
|
||||
delta.insert("content".into(), serde_json::json!(visible));
|
||||
}
|
||||
}
|
||||
// Reasoning traces and citations are not in the base
|
||||
// schema but every client that understands them
|
||||
// ignores them safely, so they are worth keeping.
|
||||
for key in ["reasoning_content", "reasoning", "thinking", "refusal", "annotations"] {
|
||||
if let Some(val) = src.get(key).filter(|v| !v.is_null()) {
|
||||
delta.insert(key.into(), val.clone());
|
||||
}
|
||||
}
|
||||
if let Some(tc) = src["tool_calls"].as_array().filter(|a| !a.is_empty()) {
|
||||
let normalized =
|
||||
crate::toolcalls::normalize_delta_tool_calls(tc, &mut tool_indices);
|
||||
if !normalized.is_empty() {
|
||||
emitted_tool_calls = true;
|
||||
delta.insert(
|
||||
"tool_calls".into(),
|
||||
serde_json::Value::Array(normalized),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let finish = c["finish_reason"].as_str().map(normalize_finish_reason);
|
||||
|
||||
// Last chance to promote `<tool_call>` markup into a
|
||||
// real call: an upstream that never parsed it would
|
||||
// otherwise hand the client prose no tool loop runs.
|
||||
if finish.is_some() && !emitted_tool_calls {
|
||||
if let Some(filter) = markup.get_mut(&index) {
|
||||
let leftover = filter.finish();
|
||||
if !leftover.is_empty() {
|
||||
let merged = match delta.get("content").and_then(|v| v.as_str()) {
|
||||
Some(prev) => format!("{prev}{leftover}"),
|
||||
None => leftover,
|
||||
};
|
||||
delta.insert("content".into(), serde_json::json!(merged));
|
||||
}
|
||||
let calls = crate::toolcalls::markup_to_tool_calls(filter.captured(), 0);
|
||||
if !calls.is_empty() {
|
||||
emitted_tool_calls = true;
|
||||
let tc = serde_json::json!({
|
||||
"index": index,
|
||||
"delta": { "tool_calls": calls },
|
||||
});
|
||||
let extra: Vec<(&str, serde_json::Value)> = Vec::new();
|
||||
yield Ok(chunk!(vec![tc], extra));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if finish.is_none() && delta.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let mut choice = serde_json::json!({
|
||||
"index": index,
|
||||
"delta": serde_json::Value::Object(delta),
|
||||
});
|
||||
if let Some(f) = finish {
|
||||
// A reply that carries tool calls stops for that
|
||||
// reason, whatever the upstream called it —
|
||||
// clients branch on this field to run the tool.
|
||||
let f = if emitted_tool_calls { "tool_calls" } else { f };
|
||||
choice["finish_reason"] = serde_json::json!(f);
|
||||
sent_finish = true;
|
||||
}
|
||||
choices.push(choice);
|
||||
}
|
||||
}
|
||||
|
||||
// Terminal metadata rides on a choice-less chunk, exactly
|
||||
// as OpenAI ships `usage` when `stream_options` asks for it.
|
||||
let mut extra: Vec<(&str, serde_json::Value)> = Vec::new();
|
||||
if let Some(u) = v.get("usage").filter(|u| !u.is_null()) {
|
||||
extra.push(("usage", u.clone()));
|
||||
}
|
||||
for key in ["cortiq_cost_usd", "cortiq_estimated"] {
|
||||
if let Some(val) = v.get(key).filter(|v| !v.is_null()) {
|
||||
extra.push((key, val.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
if !choices.is_empty() || !extra.is_empty() {
|
||||
yield Ok(chunk!(choices, extra));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(message) = failure {
|
||||
let evt = serde_json::json!({
|
||||
"error": { "message": message, "type": "upstream_unavailable", "code": "stream_failed" }
|
||||
});
|
||||
yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
|
||||
}
|
||||
|
||||
// Close the stream properly whatever the upstream did: a client that
|
||||
// never sees a stop reason and a [DONE] keeps its spinner turning.
|
||||
if !sent_finish {
|
||||
let mut delta = serde_json::Map::new();
|
||||
if roled.insert(0) {
|
||||
delta.insert("role".into(), serde_json::json!("assistant"));
|
||||
}
|
||||
// Held-back text and unparsed markup must not be lost with the stream.
|
||||
for (index, filter) in markup.iter_mut() {
|
||||
let leftover = filter.finish();
|
||||
if !leftover.is_empty() && *index == 0 {
|
||||
delta.insert("content".into(), serde_json::json!(leftover));
|
||||
}
|
||||
if !emitted_tool_calls {
|
||||
let calls = crate::toolcalls::markup_to_tool_calls(filter.captured(), 0);
|
||||
if !calls.is_empty() {
|
||||
emitted_tool_calls = true;
|
||||
delta.insert("tool_calls".into(), serde_json::Value::Array(calls));
|
||||
}
|
||||
}
|
||||
}
|
||||
let choices = vec![serde_json::json!({
|
||||
"index": 0,
|
||||
"delta": serde_json::Value::Object(delta),
|
||||
"finish_reason": if emitted_tool_calls { "tool_calls" } else { "stop" },
|
||||
})];
|
||||
let extra: Vec<(&str, serde_json::Value)> = Vec::new();
|
||||
yield Ok(chunk!(choices, extra));
|
||||
}
|
||||
yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn content_parts_and_null_are_accepted() {
|
||||
let body = serde_json::json!({
|
||||
"model": "cortiq-auto",
|
||||
"messages": [
|
||||
{"role": "user", "content": [{"type": "text", "text": "hello "}, {"type": "image_url", "image_url": {"url": "x"}}, {"type": "text", "text": "world"}]},
|
||||
{"role": "assistant", "content": null},
|
||||
{"role": "tool", "content": "42", "tool_call_id": "call_1"}
|
||||
],
|
||||
"tools": null,
|
||||
"reasoning_effort": "high",
|
||||
"chat_id": "owui-123"
|
||||
});
|
||||
let req: OpenAiChatRequest = serde_json::from_value(body).expect("must parse");
|
||||
assert_eq!(req.messages[0].content, "hello world");
|
||||
assert_eq!(req.messages[1].content, "");
|
||||
assert_eq!(req.messages[2].tool_call_id.as_deref(), Some("call_1"));
|
||||
assert!(req.tools.unwrap_or_default().is_empty());
|
||||
assert_eq!(
|
||||
req.reasoning_effort.as_ref().and_then(Effort::to_budget),
|
||||
Some(8192)
|
||||
);
|
||||
assert!(!sanitize_passthrough(req.rest).contains_key("chat_id"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn finish_reasons_map_to_the_openai_set() {
|
||||
assert_eq!(normalize_finish_reason("max_tokens"), "length");
|
||||
assert_eq!(normalize_finish_reason("eos"), "stop");
|
||||
assert_eq!(normalize_finish_reason("tool_use"), "tool_calls");
|
||||
}
|
||||
|
||||
/// Collect a normalised stream into one string for assertions.
|
||||
async fn drain(chunks: Vec<&'static str>) -> String {
|
||||
use futures::StreamExt;
|
||||
let upstream = futures::stream::iter(
|
||||
chunks
|
||||
.into_iter()
|
||||
.map(|c| Ok(bytes::Bytes::from(c)))
|
||||
.collect::<Vec<_>>(),
|
||||
);
|
||||
normalize_sse(Box::pin(upstream), "cortiq-auto".into())
|
||||
.map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.await
|
||||
.join("")
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn tool_markup_is_not_duplicated_into_content() {
|
||||
// The reported shape: the upstream streams the call as prose AND then
|
||||
// sends it properly structured.
|
||||
let out = drain(vec![
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Sure. <tool_call>\"}}]}\n\n",
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"{\\\"name\\\":\\\"get_weather\\\",\\\"arguments\\\":{\\\"city\\\":\\\"Paris\\\"}}\"}}]}\n\n",
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"</tool_call>\"}}]}\n\n",
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"}}]}}]}\n\n",
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
|
||||
"data: [DONE]\n\n",
|
||||
])
|
||||
.await;
|
||||
assert!(
|
||||
!out.contains("tool_call>"),
|
||||
"markup leaked into content: {out}"
|
||||
);
|
||||
assert!(
|
||||
out.contains("\"id\":\"call_1\""),
|
||||
"structured call missing: {out}"
|
||||
);
|
||||
// exactly one structured call, and the stop reason agrees with it
|
||||
assert_eq!(out.matches("\"tool_calls\":[").count(), 1);
|
||||
assert!(out.contains("\"finish_reason\":\"tool_calls\""));
|
||||
assert!(out.trim_end().ends_with("data: [DONE]"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn unparsed_markup_is_promoted_to_a_real_tool_call() {
|
||||
// An upstream that never parses the markup: the call must still be
|
||||
// executable by the client rather than arriving as prose.
|
||||
let out = drain(vec![
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"<tool_call>{\\\"name\\\":\\\"get_weather\\\",\\\"arguments\\\":{\\\"city\\\":\\\"Paris\\\"}}</tool_call>\"}}]}\n\n",
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
|
||||
"data: [DONE]\n\n",
|
||||
])
|
||||
.await;
|
||||
assert!(!out.contains("tool_call>"), "markup leaked: {out}");
|
||||
assert!(
|
||||
out.contains("\"name\":\"get_weather\""),
|
||||
"call not promoted: {out}"
|
||||
);
|
||||
assert!(
|
||||
out.contains("\"finish_reason\":\"tool_calls\""),
|
||||
"wrong stop reason: {out}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn ordinary_text_is_untouched_by_the_tool_filter() {
|
||||
let out = drain(vec![
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a < b and 2<3\"}}]}\n\n",
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
|
||||
"data: [DONE]\n\n",
|
||||
])
|
||||
.await;
|
||||
assert!(out.contains("a < b and 2<3"), "text mangled: {out}");
|
||||
assert!(out.contains("\"finish_reason\":\"stop\""));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_is_closed_even_when_the_upstream_stops_early() {
|
||||
use futures::StreamExt;
|
||||
let upstream = futures::stream::iter(vec![Ok(bytes::Bytes::from(
|
||||
"data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"}}]}\n\n",
|
||||
))]);
|
||||
let out: Vec<String> = normalize_sse(Box::pin(upstream), "cortiq-auto".into())
|
||||
.map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
|
||||
.collect()
|
||||
.await;
|
||||
let joined = out.join("");
|
||||
assert!(joined.contains("\"role\":\"assistant\""));
|
||||
assert!(joined.contains("\"model\":\"cortiq-auto\""));
|
||||
assert!(joined.contains("\"finish_reason\":\"stop\""));
|
||||
assert!(joined.trim_end().ends_with("data: [DONE]"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
|
||||
use crate::protocols::openai_chat::cortiq_headers;
|
||||
use crate::protocols::openai_chat::{cortiq_headers, normalize_finish_reason, sse_headers};
|
||||
use crate::state::SharedState;
|
||||
use axum::response::{IntoResponse, Response};
|
||||
use axum::{extract::State, routing::post, Json, Router};
|
||||
@@ -56,6 +56,7 @@ fn prompt_to_string(v: &serde_json::Value) -> String {
|
||||
|
||||
async fn handler(
|
||||
State(state): State<SharedState>,
|
||||
acct: Option<axum::Extension<super::AccountTag>>,
|
||||
Json(req): Json<CompletionsRequest>,
|
||||
) -> Result<Response> {
|
||||
if !state.live().cfg.protocols.openai_completions {
|
||||
@@ -70,37 +71,37 @@ async fn handler(
|
||||
));
|
||||
}
|
||||
|
||||
// Echoed back as-is: clients match the reply against the id they asked for.
|
||||
let requested_model = req.model.clone();
|
||||
let canonical = ChatRequest {
|
||||
routing: parse_routing(&req.model),
|
||||
messages: vec![Message {
|
||||
role: "user".into(),
|
||||
content: prompt,
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
}],
|
||||
tools: vec![],
|
||||
params: GenParams {
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
stream: req.stream,
|
||||
meta: RequestMeta {
|
||||
protocol: "openai_completions".into(),
|
||||
account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
|
||||
if req.stream {
|
||||
let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
|
||||
let mut headers = cortiq_headers(&info);
|
||||
headers.insert(
|
||||
axum::http::header::CONTENT_TYPE,
|
||||
axum::http::HeaderValue::from_static("text/event-stream"),
|
||||
);
|
||||
return Ok((
|
||||
headers,
|
||||
sse_headers(&info),
|
||||
axum::body::Body::from_stream(chat_to_completions_sse(stream)),
|
||||
)
|
||||
.into_response());
|
||||
@@ -112,11 +113,12 @@ async fn handler(
|
||||
.first()
|
||||
.map(|c| c.message.content.clone())
|
||||
.unwrap_or_default();
|
||||
let finish = resp
|
||||
.choices
|
||||
.first()
|
||||
.map(|c| c.finish_reason.clone())
|
||||
.unwrap_or_else(|| "stop".into());
|
||||
let finish = normalize_finish_reason(
|
||||
resp.choices
|
||||
.first()
|
||||
.map(|c| c.finish_reason.as_str())
|
||||
.unwrap_or("stop"),
|
||||
);
|
||||
let headers = cortiq_headers(&resp.cortiq);
|
||||
let body = serde_json::json!({
|
||||
"id": resp.id,
|
||||
@@ -125,7 +127,7 @@ async fn handler(
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs(),
|
||||
"model": resp.model_used,
|
||||
"model": requested_model,
|
||||
"choices": [{ "text": answer, "index": 0, "logprobs": serde_json::Value::Null, "finish_reason": finish }],
|
||||
"usage": {
|
||||
"prompt_tokens": resp.usage.prompt_tokens,
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
//! Inbound OpenAI Models adapter: `GET /v1/models`.
|
||||
//! Lists the configured pool (plus the virtual `cortiq-auto`) for client discovery.
|
||||
//! Lists everything a client can address: the virtual `cortiq-auto`, the
|
||||
//! configured `[[models]]` pool, and the managed local `[[cmf.servers]]`.
|
||||
//!
|
||||
//! The managed CMF servers belong here because they are routable model ids like
|
||||
//! any other — omitting them left OpenAI clients (Open WebUI among them) unable
|
||||
//! to pin a local model even though the gateway would happily serve it.
|
||||
|
||||
use crate::error::{GatewayError, Result};
|
||||
use crate::state::SharedState;
|
||||
@@ -9,6 +14,13 @@ pub fn routes() -> Router<SharedState> {
|
||||
Router::new().route("/v1/models", get(handler))
|
||||
}
|
||||
|
||||
fn now_secs() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse> {
|
||||
let live = state.live();
|
||||
if !live.cfg.protocols.openai_models {
|
||||
@@ -16,17 +28,51 @@ async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse>
|
||||
"openai_models protocol is disabled".into(),
|
||||
));
|
||||
}
|
||||
let created = now_secs();
|
||||
let mut data = vec![serde_json::json!({
|
||||
"id": "cortiq-auto",
|
||||
"object": "model",
|
||||
"created": created,
|
||||
"owned_by": "cortiq",
|
||||
})];
|
||||
let mut seen: std::collections::HashSet<String> =
|
||||
std::collections::HashSet::from(["cortiq-auto".to_string()]);
|
||||
|
||||
for m in &live.cfg.models {
|
||||
if !seen.insert(m.id.clone()) {
|
||||
continue;
|
||||
}
|
||||
data.push(serde_json::json!({
|
||||
"id": m.id,
|
||||
"object": "model",
|
||||
"created": created,
|
||||
"owned_by": m.provider,
|
||||
}));
|
||||
}
|
||||
|
||||
// Managed local CMF servers. Health is reported alongside (a non-standard
|
||||
// field clients ignore) rather than used as a filter: a large `.cmf` takes
|
||||
// minutes to load, and a model that vanishes from the list while it warms up
|
||||
// is worse than one marked "not ready yet".
|
||||
let cmf = state.cmf.status();
|
||||
for s in live.cfg.cmf.effective_servers() {
|
||||
if !seen.insert(s.id.clone()) {
|
||||
continue; // a static [[models]] entry with the same id wins
|
||||
}
|
||||
let status = cmf.servers.iter().find(|st| st.id == s.id);
|
||||
data.push(serde_json::json!({
|
||||
"id": s.id,
|
||||
"object": "model",
|
||||
"created": created,
|
||||
"owned_by": "cortiq-cmf",
|
||||
"cortiq": {
|
||||
"kind": "cmf",
|
||||
"local": true,
|
||||
"running": status.map(|st| st.running).unwrap_or(false),
|
||||
"healthy": status.map(|st| st.healthy).unwrap_or(false),
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
Ok(Json(serde_json::json!({ "object": "list", "data": data })))
|
||||
}
|
||||
|
||||
@@ -32,7 +32,13 @@ impl AnthropicProvider {
|
||||
caps: Caps::from_list(&cfg.caps),
|
||||
price_in: cfg.price_in,
|
||||
price_out: cfg.price_out,
|
||||
http: reqwest::Client::new(),
|
||||
// connect timeout: a dead host fails over in seconds instead of
|
||||
// hanging the request; NO overall timeout — that would cut long
|
||||
// generations and streams (OpenRouter can be slow legitimately)
|
||||
http: reqwest::Client::builder()
|
||||
.connect_timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.unwrap_or_else(|_| reqwest::Client::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -140,6 +146,7 @@ impl Provider for AnthropicProvider {
|
||||
role: "assistant".into(),
|
||||
content,
|
||||
tool_calls: vec![],
|
||||
..Default::default()
|
||||
},
|
||||
finish_reason: v["stop_reason"].as_str().unwrap_or("stop").to_string(),
|
||||
}],
|
||||
@@ -157,6 +164,7 @@ impl Provider for AnthropicProvider {
|
||||
router_request_id: None,
|
||||
cost_usd: 0.0,
|
||||
failover: false,
|
||||
tools_dropped: false,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
+35
-4
@@ -28,7 +28,13 @@ impl OpenAiProvider {
|
||||
caps: Caps::from_list(&cfg.caps),
|
||||
price_in: cfg.price_in,
|
||||
price_out: cfg.price_out,
|
||||
http: reqwest::Client::new(),
|
||||
// connect timeout: a dead host fails over in seconds instead of
|
||||
// hanging the request; NO overall timeout — that would cut long
|
||||
// generations and streams (OpenRouter can be slow legitimately)
|
||||
http: reqwest::Client::builder()
|
||||
.connect_timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.unwrap_or_else(|_| reqwest::Client::new()),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -42,9 +48,18 @@ impl OpenAiProvider {
|
||||
if let Some(t) = req.params.temperature {
|
||||
body["temperature"] = t.into();
|
||||
}
|
||||
if let Some(p) = req.params.top_p {
|
||||
body["top_p"] = p.into();
|
||||
}
|
||||
if let Some(m) = req.params.max_tokens {
|
||||
body["max_tokens"] = m.into();
|
||||
}
|
||||
if let Some(tb) = req.params.think_budget {
|
||||
body["think_budget"] = tb.into();
|
||||
body["reasoning_effort"] = tb.into();
|
||||
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
|
||||
body["enable_thinking"] = (tb > 0).into();
|
||||
}
|
||||
if !req.tools.is_empty() {
|
||||
body["tools"] = serde_json::Value::Array(req.tools.clone());
|
||||
}
|
||||
@@ -107,21 +122,36 @@ impl Provider for OpenAiProvider {
|
||||
for (i, c) in choices_arr.iter().enumerate() {
|
||||
let msg_val = &c["message"];
|
||||
let role = msg_val["role"].as_str().unwrap_or("assistant").to_string();
|
||||
let content = msg_val["content"].as_str().unwrap_or("").to_string();
|
||||
let raw_content = msg_val["content"].as_str().unwrap_or("");
|
||||
// Qwen-style templates put the call in the text as `<tool_call>…`.
|
||||
// Strip it: the structured array is the only form a client may act
|
||||
// on, and prose that merely looks like a call is worse than none.
|
||||
let (content, markup) = crate::toolcalls::split_tool_markup(raw_content);
|
||||
|
||||
let tool_calls = msg_val["tool_calls"]
|
||||
let mut tool_calls = msg_val["tool_calls"]
|
||||
.as_array()
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
if tool_calls.is_empty() {
|
||||
// Upstream never parsed the markup — promote it, so the call is
|
||||
// executable instead of decorative.
|
||||
tool_calls = crate::toolcalls::markup_to_tool_calls(&markup, 0);
|
||||
}
|
||||
|
||||
let tool_calls_present = !tool_calls.is_empty();
|
||||
choices.push(crate::model::Choice {
|
||||
index: c["index"].as_u64().unwrap_or(i as u64) as u32,
|
||||
message: crate::model::Message {
|
||||
role,
|
||||
content,
|
||||
tool_calls,
|
||||
..Default::default()
|
||||
},
|
||||
finish_reason: if tool_calls_present {
|
||||
"tool_calls".to_string()
|
||||
} else {
|
||||
c["finish_reason"].as_str().unwrap_or("stop").to_string()
|
||||
},
|
||||
finish_reason: c["finish_reason"].as_str().unwrap_or("stop").to_string(),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -151,6 +181,7 @@ impl Provider for OpenAiProvider {
|
||||
router_request_id: None,
|
||||
cost_usd: 0.0,
|
||||
failover: false,
|
||||
tools_dropped: false,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
+8
-1
@@ -48,7 +48,14 @@ impl Registry {
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
caps: s.caps.clone(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
think_budget: s.think_budget,
|
||||
system_prompt: s.system_prompt.clone(),
|
||||
o1: s.o1.clone(),
|
||||
skip_mtp: s.skip_mtp,
|
||||
};
|
||||
by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
|
||||
}
|
||||
|
||||
@@ -71,6 +71,23 @@ impl RouterClient {
|
||||
|
||||
/// Classify text. Returns `Ok(None)` on network error/timeout —
|
||||
/// the caller treats this as "router unavailable" and uses the default tier.
|
||||
/// `GET /v1/taxonomies` — the account's taxonomies with their label sets.
|
||||
/// `Ok(None)` when the router is disabled or has no key configured.
|
||||
pub async fn taxonomies(&self) -> anyhow::Result<Option<serde_json::Value>> {
|
||||
if self.url.is_empty() {
|
||||
return Ok(None);
|
||||
}
|
||||
let mut req = self.http.get(format!("{}/v1/taxonomies", self.url));
|
||||
if let Some(k) = &self.api_key {
|
||||
req = req.bearer_auth(k);
|
||||
}
|
||||
let resp = req.send().await?;
|
||||
if !resp.status().is_success() {
|
||||
anyhow::bail!("router /v1/taxonomies: HTTP {}", resp.status());
|
||||
}
|
||||
Ok(Some(resp.json().await?))
|
||||
}
|
||||
|
||||
pub async fn route(&self, text: &str, profile: &str) -> anyhow::Result<Option<RouteDecision>> {
|
||||
let mut body = serde_json::json!({
|
||||
"input": { "text": text },
|
||||
|
||||
+212
-17
@@ -3,38 +3,117 @@
|
||||
|
||||
use crate::config::{Config, TierTargets};
|
||||
|
||||
fn to_list(t: &TierTargets) -> Vec<String> {
|
||||
t.models()
|
||||
}
|
||||
|
||||
/// A task rule, normalized: per-tier model lists ("which model serves this
|
||||
/// task at each complexity") plus any-tier models that lead regardless.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct TaskRule {
|
||||
pub any: Vec<String>,
|
||||
pub per: std::collections::HashMap<String, Vec<String>>,
|
||||
}
|
||||
|
||||
pub struct RoutingTable {
|
||||
tiers: std::collections::HashMap<String, Vec<String>>,
|
||||
/// Per-task rules — a matching rule's models lead the candidate chain.
|
||||
tasks: std::collections::HashMap<String, TaskRule>,
|
||||
default: String,
|
||||
}
|
||||
|
||||
impl RoutingTable {
|
||||
pub fn from_config(cfg: &Config) -> Self {
|
||||
let tiers = cfg
|
||||
.routing
|
||||
.tiers
|
||||
.iter()
|
||||
.map(|(tier, targets)| {
|
||||
let list = match targets {
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
(tier.clone(), list)
|
||||
})
|
||||
.collect();
|
||||
Self {
|
||||
tiers,
|
||||
tiers: cfg
|
||||
.routing
|
||||
.tiers
|
||||
.iter()
|
||||
.map(|(tier, targets)| (tier.clone(), to_list(targets)))
|
||||
.collect(),
|
||||
tasks: cfg
|
||||
.routing
|
||||
.tasks
|
||||
.iter()
|
||||
.map(|(task, targets)| {
|
||||
let (any, per) = targets.task_rule();
|
||||
(task.clone(), TaskRule { any, per })
|
||||
})
|
||||
.filter(|(_, r)| !r.any.is_empty() || r.per.values().any(|v| !v.is_empty()))
|
||||
.collect(),
|
||||
default: cfg.routing.default.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Candidate chain for `(task, tier)` and whether a task rule shaped it.
|
||||
///
|
||||
/// Order of preference:
|
||||
/// 1. the task's models FOR THIS TIER (the user said exactly which model
|
||||
/// serves this task at this complexity);
|
||||
/// 2. at tier=high — the global high chain (escalation: any-tier task
|
||||
/// models must not undercut it), then the task's any-tier models;
|
||||
/// otherwise the task's any-tier models, then the global tier chain;
|
||||
/// 3. `default` as the last resort. Deduplicated, empties dropped.
|
||||
pub fn candidates_for(&self, task: &str, tier: &str) -> (Vec<String>, bool) {
|
||||
fn push(out: &mut Vec<String>, id: &str) {
|
||||
if !id.is_empty() && !out.iter().any(|x| x == id) {
|
||||
out.push(id.to_string());
|
||||
}
|
||||
}
|
||||
let mut out: Vec<String> = Vec::new();
|
||||
let rule = self.tasks.get(task);
|
||||
let tier_chain = self
|
||||
.tiers
|
||||
.get(tier)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| vec![self.default.clone()]);
|
||||
if let Some(r) = rule {
|
||||
if let Some(list) = r.per.get(tier) {
|
||||
for id in list {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
}
|
||||
let escalate = tier == "high" && self.tiers.contains_key("high");
|
||||
if escalate {
|
||||
// high verdicts escalate over any-tier task pins
|
||||
for id in &tier_chain {
|
||||
push(&mut out, id);
|
||||
}
|
||||
if let Some(r) = rule {
|
||||
for id in &r.any {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if let Some(r) = rule {
|
||||
for id in &r.any {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
for id in &tier_chain {
|
||||
push(&mut out, id);
|
||||
}
|
||||
}
|
||||
let d = self.default.clone();
|
||||
push(&mut out, &d);
|
||||
// "task-rule" must mean "the rule chose the head of this chain" — a
|
||||
// rule that exists but lost to the tier chain (math has a high pick,
|
||||
// the request came in low) must not claim the credit.
|
||||
let matched = match (rule, out.first()) {
|
||||
(Some(r), Some(head)) => {
|
||||
r.per.get(tier).map(|v| v.contains(head)).unwrap_or(false)
|
||||
|| r.any.first() == Some(head)
|
||||
}
|
||||
_ => false,
|
||||
};
|
||||
(out, matched)
|
||||
}
|
||||
|
||||
/// Candidates for the given complexity tier, in preference order.
|
||||
/// If the tier is not defined, the sole candidate is `default`.
|
||||
pub fn candidates(&self, tier: &str) -> Vec<String> {
|
||||
self.tiers
|
||||
.get(tier)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| vec![self.default.clone()])
|
||||
self.candidates_for("", tier).0
|
||||
}
|
||||
|
||||
/// Default model (used when the router is unavailable).
|
||||
@@ -46,3 +125,119 @@ impl RoutingTable {
|
||||
// TODO(v0.2): cost_aware mode and circuit breaker — pick the cheapest available
|
||||
// model no lower than min_class[tier], respecting max_cost_usd_per_request,
|
||||
// and skip models whose circuit breaker is open.
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn table(tasks: &[(&str, &[&str])], tiers: &[(&str, &[&str])], default: &str) -> RoutingTable {
|
||||
RoutingTable {
|
||||
tiers: tiers
|
||||
.iter()
|
||||
.map(|(k, v)| (k.to_string(), v.iter().map(|s| s.to_string()).collect()))
|
||||
.collect(),
|
||||
tasks: tasks
|
||||
.iter()
|
||||
.map(|(k, v)| {
|
||||
(
|
||||
k.to_string(),
|
||||
TaskRule {
|
||||
any: v.iter().map(|s| s.to_string()).collect(),
|
||||
per: Default::default(),
|
||||
},
|
||||
)
|
||||
})
|
||||
.collect(),
|
||||
default: default.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn with_task_tier_models(
|
||||
mut t: RoutingTable,
|
||||
task: &str,
|
||||
tier: &str,
|
||||
models: &[&str],
|
||||
) -> RoutingTable {
|
||||
t.tasks.entry(task.to_string()).or_default().per.insert(
|
||||
tier.to_string(),
|
||||
models.iter().map(|s| s.to_string()).collect(),
|
||||
);
|
||||
t
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn task_rule_leads_tier_chain_follows() {
|
||||
let t = table(
|
||||
&[("translation", &["local"])],
|
||||
&[("low", &["cheap", "local"]), ("high", &["big"])],
|
||||
"cheap",
|
||||
);
|
||||
// dedup: rule model already in the tier chain appears once, first
|
||||
let (c, matched) = t.candidates_for("translation", "low");
|
||||
assert!(matched);
|
||||
assert_eq!(c, vec!["local", "cheap"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn high_tier_escalates_over_task_rule() {
|
||||
let t = table(
|
||||
&[("code", &["cheap-coder"])],
|
||||
&[("low", &["cheap"]), ("high", &["big"])],
|
||||
"cheap",
|
||||
);
|
||||
// medium (no chain defined → default): task rule still leads
|
||||
let (c, m) = t.candidates_for("code", "medium");
|
||||
assert!(m);
|
||||
assert_eq!(c[0], "cheap-coder");
|
||||
// high: tier chain leads, task rule follows as fallback
|
||||
let (c, m) = t.candidates_for("code", "high");
|
||||
// escalation put the global high chain first → the rule did not pick
|
||||
// the head, so no task-rule credit
|
||||
assert!(!m);
|
||||
// default rides along as the very last fallback
|
||||
assert_eq!(c, vec!["big", "cheap-coder", "cheap"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn per_tier_models_lead_their_tier() {
|
||||
let base = table(
|
||||
&[("code", &["generic-coder"])],
|
||||
&[("low", &["cheap"]), ("high", &["big"])],
|
||||
"cheap",
|
||||
);
|
||||
let t = with_task_tier_models(base, "code", "high", &["best-coder"]);
|
||||
// at high: the task's HIGH pick leads, escalation chain next,
|
||||
// any-tier task models after it, default last
|
||||
let (c, m) = t.candidates_for("code", "high");
|
||||
assert!(m);
|
||||
assert_eq!(c, vec!["best-coder", "big", "generic-coder", "cheap"]);
|
||||
// at low: any-tier task models lead as before (and the badge is
|
||||
// honest — the head came from the rule)
|
||||
let (c, m) = t.candidates_for("code", "low");
|
||||
assert!(m);
|
||||
assert_eq!(c, vec!["generic-coder", "cheap"]);
|
||||
// a tier with an explicit pick but no any-models
|
||||
let t2 = with_task_tier_models(
|
||||
table(&[], &[("low", &["cheap"]), ("high", &["big"])], "cheap"),
|
||||
"legal",
|
||||
"high",
|
||||
&["strong"],
|
||||
);
|
||||
let (c, m) = t2.candidates_for("legal", "high");
|
||||
assert!(m);
|
||||
assert_eq!(c, vec!["strong", "big", "cheap"]);
|
||||
// the same rule at LOW did not shape the head → no task-rule credit
|
||||
let (c, m) = t2.candidates_for("legal", "low");
|
||||
assert!(!m);
|
||||
assert_eq!(c, vec!["cheap"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_rule_falls_back_to_tier_then_default() {
|
||||
let t = table(&[], &[("low", &["a"])], "d");
|
||||
assert_eq!(t.candidates_for("qa", "low").0, vec!["a", "d"]);
|
||||
let (c, matched) = t.candidates_for("qa", "unknown-tier");
|
||||
assert!(!matched);
|
||||
assert_eq!(c, vec!["d"]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,14 +66,63 @@ impl Live {
|
||||
}
|
||||
}
|
||||
|
||||
/// Fixed-window per-key rate limiter (minute buckets). Small and honest:
|
||||
/// at minute boundaries a burst can briefly see up to 2× the limit, which is
|
||||
/// fine for office-grade fairness and costs one HashMap lookup per request.
|
||||
#[derive(Default)]
|
||||
pub struct RateWindows {
|
||||
inner: std::sync::Mutex<std::collections::HashMap<String, (u64, u32)>>,
|
||||
}
|
||||
|
||||
impl RateWindows {
|
||||
pub fn allow(&self, key: &str, per_min: u32) -> bool {
|
||||
let minute = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
/ 60;
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
let e = g.entry(key.to_string()).or_insert((minute, 0));
|
||||
if e.0 != minute {
|
||||
*e = (minute, 0);
|
||||
}
|
||||
if e.1 >= per_min {
|
||||
return false;
|
||||
}
|
||||
e.1 += 1;
|
||||
true
|
||||
}
|
||||
/// Seconds until the current minute window rolls over.
|
||||
pub fn retry_after_secs(&self) -> u64 {
|
||||
60 - (std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
% 60)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct AppState {
|
||||
pub live: ArcSwap<Live>,
|
||||
pub stats: Arc<Stats>,
|
||||
pub promotion: Arc<crate::promotion::Promotion>,
|
||||
pub imports: Arc<crate::import::JobStore>,
|
||||
/// Image/video generation jobs (cortiq imagine / animate).
|
||||
pub media: Arc<crate::media::MediaStore>,
|
||||
/// One-click local model benchmarks (cortiq bench --json).
|
||||
pub bench: Arc<crate::bench::BenchStore>,
|
||||
/// Host metrics sampler (CPU / RAM / disk) for the dashboard.
|
||||
pub sysmon: Arc<crate::sysmon::SysMon>,
|
||||
/// Newest cortiq-gateway version on crates.io, when newer than ours.
|
||||
pub gateway_update: Arc<std::sync::Mutex<Option<String>>>,
|
||||
/// Admin console reachable from other machines without a token — the
|
||||
/// footgun banner flag (set at startup, shown by the SPA).
|
||||
pub admin_open_public: std::sync::atomic::AtomicBool,
|
||||
/// Managed local CMF model server (install/update/spawn lifecycle + status).
|
||||
pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
|
||||
pub cache: Arc<SemanticCache>,
|
||||
/// Per-API-key fixed-window rate limiter.
|
||||
pub rate: RateWindows,
|
||||
pub secrets: SecretStore,
|
||||
pub config_path: String,
|
||||
pub pipeline: crate::pipeline::Pipeline,
|
||||
@@ -113,6 +162,30 @@ impl SharedState {
|
||||
let stats = Stats::new(&cfg.stats);
|
||||
let promotion = crate::promotion::Promotion::new(promotion_cfg(&cfg.shadow));
|
||||
let imports = crate::import::JobStore::new();
|
||||
let media = crate::media::MediaStore::new();
|
||||
let bench = crate::bench::BenchStore::new();
|
||||
// jobs survive restarts: files under <data>/, reconciled on load
|
||||
let data = crate::config::data_dir();
|
||||
let _ = std::fs::create_dir_all(&data);
|
||||
imports.attach_persistence(data.join("jobs-import.json"));
|
||||
media.attach_persistence(data.join("jobs-media.json"));
|
||||
let sysmon = crate::sysmon::spawn(cfg.cmf.models_dir.clone());
|
||||
let gateway_update: Arc<std::sync::Mutex<Option<String>>> =
|
||||
Arc::new(std::sync::Mutex::new(None));
|
||||
{
|
||||
// check crates.io for a newer gateway once at start and twice a day
|
||||
let slot = gateway_update.clone();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
if let Some(latest) = crate::cmf_runtime::latest_gateway_version().await {
|
||||
let newer =
|
||||
crate::cmf_runtime::version_lt(env!("CARGO_PKG_VERSION"), &latest);
|
||||
*slot.lock().unwrap() = newer.then_some(latest);
|
||||
}
|
||||
tokio::time::sleep(std::time::Duration::from_secs(12 * 3600)).await;
|
||||
}
|
||||
});
|
||||
}
|
||||
let cmf = crate::cmf_runtime::CmfRuntime::new();
|
||||
let cache = SemanticCache::new(&cfg.cache);
|
||||
let router_status = RouterLastStatus::default();
|
||||
@@ -122,8 +195,14 @@ impl SharedState {
|
||||
stats,
|
||||
promotion,
|
||||
imports,
|
||||
media,
|
||||
bench,
|
||||
sysmon,
|
||||
gateway_update,
|
||||
admin_open_public: std::sync::atomic::AtomicBool::new(false),
|
||||
cmf,
|
||||
cache,
|
||||
rate: RateWindows::default(),
|
||||
secrets,
|
||||
config_path,
|
||||
pipeline: crate::pipeline::Pipeline::new(),
|
||||
@@ -145,6 +224,14 @@ impl AppState {
|
||||
let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
|
||||
new_cfg.save(&self.config_path)?;
|
||||
self.live.store(Arc::new(live));
|
||||
|
||||
let cmf_rt = self.cmf.clone();
|
||||
let cmf_cfg = new_cfg.cmf.clone();
|
||||
tokio::spawn(async move {
|
||||
cmf_rt.stop().await;
|
||||
crate::cmf_runtime::manage(cmf_rt, cmf_cfg).await;
|
||||
});
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
+350
-42
@@ -55,7 +55,7 @@ pub struct RequestRecord {
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize)]
|
||||
#[derive(Clone, Debug, Default, Serialize, serde::Deserialize)]
|
||||
pub struct Totals {
|
||||
pub requests: u64,
|
||||
pub ok: u64,
|
||||
@@ -83,9 +83,23 @@ impl Totals {
|
||||
self.cost_usd += r.cost_usd;
|
||||
self.latency_ms_sum += r.latency_ms;
|
||||
}
|
||||
/// Accumulate another Totals into this one (for windowed aggregation).
|
||||
fn merge(&mut self, o: &Totals) {
|
||||
self.requests += o.requests;
|
||||
self.ok += o.ok;
|
||||
self.errors += o.errors;
|
||||
self.failovers += o.failovers;
|
||||
self.prompt_tokens += o.prompt_tokens;
|
||||
self.completion_tokens += o.completion_tokens;
|
||||
self.cost_usd += o.cost_usd;
|
||||
self.latency_ms_sum += o.latency_ms_sum;
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-minute bucket for time-series charts.
|
||||
/// Per-minute bucket. Beyond the chart fields it carries the per-group breakdown
|
||||
/// and failovers so the admin snapshot can be computed for ANY time window by
|
||||
/// summing the in-window buckets (the internal fields are not serialized into
|
||||
/// the `series` payload).
|
||||
#[derive(Clone, Debug, Default, Serialize)]
|
||||
pub struct Bucket {
|
||||
pub minute: u64, // unix seconds, rounded down to the minute
|
||||
@@ -95,17 +109,41 @@ pub struct Bucket {
|
||||
pub completion_tokens: u64,
|
||||
pub cost_usd: f64,
|
||||
pub latency_ms_sum: u64,
|
||||
#[serde(skip)]
|
||||
pub failovers: u64,
|
||||
#[serde(skip)]
|
||||
pub by_model: HashMap<String, Totals>,
|
||||
#[serde(skip)]
|
||||
pub by_account: HashMap<String, Totals>,
|
||||
#[serde(skip)]
|
||||
pub by_tier: HashMap<String, Totals>,
|
||||
#[serde(skip)]
|
||||
pub by_label: HashMap<String, Totals>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct Inner {
|
||||
/// All-time totals + per-model (used by the Prometheus counters, which must
|
||||
/// be monotonic — the windowed admin snapshot is derived from the buckets).
|
||||
total: Totals,
|
||||
by_model: HashMap<String, Totals>,
|
||||
by_account: HashMap<String, Totals>,
|
||||
by_tier: HashMap<String, Totals>,
|
||||
by_label: HashMap<String, Totals>,
|
||||
/// Per-minute buckets — the single source for the windowed admin snapshot.
|
||||
buckets: VecDeque<Bucket>,
|
||||
recent: VecDeque<RequestRecord>,
|
||||
/// Per-account totals over EVERY record still in the JSONL file (no
|
||||
/// retention gate) — per-key accounting for the keys page.
|
||||
by_account_alltime: HashMap<String, Totals>,
|
||||
/// Per-account totals folded out of the file by compaction. alltime(key)
|
||||
/// = folded + by_account_alltime, with no double counting: a record is
|
||||
/// either in the file or folded, never both.
|
||||
folded: HashMap<String, Totals>,
|
||||
/// Month-to-date spend per account ("YYYY-MM", usd) — budgets check this.
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
/// Records with ts ≤ this are already reflected in the persisted
|
||||
/// month_spend (watermark against double counting on replay).
|
||||
month_upto_ts: u64,
|
||||
/// Appends since the last compaction size check.
|
||||
writes_since_check: u32,
|
||||
}
|
||||
|
||||
pub struct Stats {
|
||||
@@ -113,6 +151,7 @@ pub struct Stats {
|
||||
file: Option<String>,
|
||||
ring_size: usize,
|
||||
retention_secs: u64,
|
||||
max_file_bytes: u64,
|
||||
inner: Mutex<Inner>,
|
||||
}
|
||||
|
||||
@@ -128,9 +167,12 @@ impl Stats {
|
||||
file,
|
||||
ring_size: cfg.ring_size.max(1),
|
||||
retention_secs: parse_duration_secs(&cfg.retention).unwrap_or(7 * 86_400),
|
||||
max_file_bytes: cfg.max_file_mb.saturating_mul(1024 * 1024),
|
||||
inner: Mutex::new(Inner::default()),
|
||||
};
|
||||
s.load_folded();
|
||||
s.replay();
|
||||
s.maybe_compact(true);
|
||||
std::sync::Arc::new(s)
|
||||
}
|
||||
|
||||
@@ -151,6 +193,12 @@ impl Stats {
|
||||
// push everything to the "recent" ring (it self-limits by size),
|
||||
// but apply to aggregates/series only entries within the window.
|
||||
Self::push_recent(&mut inner, &rec, self.ring_size);
|
||||
inner
|
||||
.by_account_alltime
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
if rec.ts >= cutoff {
|
||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||
}
|
||||
@@ -158,6 +206,33 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
|
||||
fn apply_month(inner: &mut Inner, rec: &RequestRecord) {
|
||||
if rec.ts <= inner.month_upto_ts || rec.cost_usd <= 0.0 {
|
||||
return;
|
||||
}
|
||||
let m = month_of(rec.ts);
|
||||
let e = inner
|
||||
.month_spend
|
||||
.entry(rec.account.clone())
|
||||
.or_insert((m.clone(), 0.0));
|
||||
if e.0 != m {
|
||||
*e = (m, 0.0);
|
||||
}
|
||||
e.1 += rec.cost_usd;
|
||||
}
|
||||
|
||||
/// Month-to-date spend for an account (budget checks).
|
||||
pub fn month_spend(&self, account: &str) -> f64 {
|
||||
let inner = self.inner.lock().unwrap();
|
||||
let m = month_of(now_secs());
|
||||
inner
|
||||
.month_spend
|
||||
.get(account)
|
||||
.filter(|(mm, _)| *mm == m)
|
||||
.map(|(_, v)| *v)
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
|
||||
fn push_recent(inner: &mut Inner, rec: &RequestRecord, ring_size: usize) {
|
||||
inner.recent.push_back(rec.clone());
|
||||
while inner.recent.len() > ring_size {
|
||||
@@ -166,31 +241,14 @@ impl Stats {
|
||||
}
|
||||
|
||||
fn apply_aggregates(inner: &mut Inner, rec: &RequestRecord, retention_secs: u64) {
|
||||
// all-time counters for Prometheus; the windowed snapshot is derived
|
||||
// from the buckets below.
|
||||
inner.total.apply(rec);
|
||||
inner
|
||||
.by_model
|
||||
.entry(rec.model_id.clone())
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
inner
|
||||
.by_account
|
||||
.entry(if rec.account.is_empty() {
|
||||
"anonymous".to_string()
|
||||
} else {
|
||||
rec.account.clone()
|
||||
})
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
inner
|
||||
.by_tier
|
||||
.entry(rec.tier.clone())
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
inner
|
||||
.by_label
|
||||
.entry(rec.task_label.clone())
|
||||
.or_default()
|
||||
.apply(rec);
|
||||
|
||||
let minute = rec.ts - (rec.ts % 60);
|
||||
match inner.buckets.back_mut() {
|
||||
@@ -226,6 +284,13 @@ impl Stats {
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
Self::apply_aggregates(&mut inner, &rec, self.retention_secs);
|
||||
Self::push_recent(&mut inner, &rec, self.ring_size);
|
||||
inner
|
||||
.by_account_alltime
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
Self::apply_month(&mut inner, &rec);
|
||||
inner.writes_since_check += 1;
|
||||
}
|
||||
if let Some(path) = &self.file {
|
||||
if let Ok(line) = serde_json::to_string(&rec) {
|
||||
@@ -238,6 +303,141 @@ impl Stats {
|
||||
}
|
||||
}
|
||||
}
|
||||
self.maybe_compact(false);
|
||||
}
|
||||
|
||||
fn totals_path(&self) -> Option<String> {
|
||||
self.file.as_ref().map(|f| format!("{f}.totals.json"))
|
||||
}
|
||||
|
||||
fn load_folded(&self) {
|
||||
let Some(path) = self.totals_path() else {
|
||||
return;
|
||||
};
|
||||
let Ok(content) = std::fs::read_to_string(&path) else {
|
||||
return;
|
||||
};
|
||||
#[derive(serde::Deserialize)]
|
||||
struct Sidecar {
|
||||
#[serde(default)]
|
||||
by_account: HashMap<String, Totals>,
|
||||
#[serde(default)]
|
||||
month_spend: HashMap<String, (String, f64)>,
|
||||
#[serde(default)]
|
||||
month_upto_ts: u64,
|
||||
}
|
||||
// new format {by_account, month_spend, month_upto_ts}; the original
|
||||
// file held the bare by_account map — accept both
|
||||
if let Ok(sc) = serde_json::from_str::<Sidecar>(&content) {
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
inner.folded = sc.by_account;
|
||||
inner.month_spend = sc.month_spend;
|
||||
inner.month_upto_ts = sc.month_upto_ts;
|
||||
} else if let Ok(m) = serde_json::from_str::<HashMap<String, Totals>>(&content) {
|
||||
self.inner.lock().unwrap().folded = m;
|
||||
}
|
||||
}
|
||||
|
||||
fn write_sidecar(&self, inner: &Inner) {
|
||||
let Some(tpath) = self.totals_path() else {
|
||||
return;
|
||||
};
|
||||
let payload = serde_json::json!({
|
||||
"by_account": inner.folded,
|
||||
"month_spend": inner.month_spend,
|
||||
"month_upto_ts": now_secs(),
|
||||
});
|
||||
let tmp = format!("{tpath}.tmp");
|
||||
if std::fs::write(&tmp, payload.to_string()).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, &tpath);
|
||||
}
|
||||
}
|
||||
|
||||
/// Keep the JSONL bounded: once it outgrows `max_file_bytes`, records
|
||||
/// older than the retention window fold into the cumulative per-account
|
||||
/// sidecar and are dropped from the file (tmp + rename on both files, so
|
||||
/// a crash mid-compaction loses nothing). `force` checks size regardless
|
||||
/// of the write counter (startup).
|
||||
fn maybe_compact(&self, force: bool) {
|
||||
if self.max_file_bytes == 0 {
|
||||
return;
|
||||
}
|
||||
let Some(path) = &self.file else { return };
|
||||
{
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
if !force {
|
||||
if inner.writes_since_check < 256 {
|
||||
return;
|
||||
}
|
||||
inner.writes_since_check = 0;
|
||||
// piggyback: persist month-to-date spend so budgets survive
|
||||
// restarts even between compactions
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
}
|
||||
}
|
||||
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
|
||||
if size <= self.max_file_bytes {
|
||||
return;
|
||||
}
|
||||
let Ok(content) = std::fs::read_to_string(path) else {
|
||||
return;
|
||||
};
|
||||
let cutoff = now_secs().saturating_sub(self.retention_secs);
|
||||
let mut kept = String::with_capacity(content.len() / 2);
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
for line in content.lines() {
|
||||
let line = line.trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
match serde_json::from_str::<RequestRecord>(line) {
|
||||
Ok(rec) if rec.ts < cutoff => {
|
||||
inner
|
||||
.folded
|
||||
.entry(rec.account.clone())
|
||||
.or_default()
|
||||
.apply(&rec);
|
||||
// fold = leave by_account_alltime (record leaves the file,
|
||||
// its share moves from "alltime" into "folded")
|
||||
if let Some(t) = inner.by_account_alltime.get_mut(&rec.account) {
|
||||
// subtract by rebuilding is costly; instead track the
|
||||
// invariant additively: remove the record's share
|
||||
t.requests = t.requests.saturating_sub(1);
|
||||
if rec.outcome == "ok" {
|
||||
t.ok = t.ok.saturating_sub(1);
|
||||
} else {
|
||||
t.errors = t.errors.saturating_sub(1);
|
||||
}
|
||||
if rec.failover {
|
||||
t.failovers = t.failovers.saturating_sub(1);
|
||||
}
|
||||
t.prompt_tokens = t.prompt_tokens.saturating_sub(rec.prompt_tokens as u64);
|
||||
t.completion_tokens = t
|
||||
.completion_tokens
|
||||
.saturating_sub(rec.completion_tokens as u64);
|
||||
t.cost_usd = (t.cost_usd - rec.cost_usd).max(0.0);
|
||||
t.latency_ms_sum = t.latency_ms_sum.saturating_sub(rec.latency_ms);
|
||||
}
|
||||
}
|
||||
Ok(_) => {
|
||||
kept.push_str(line);
|
||||
kept.push('\n');
|
||||
}
|
||||
Err(_) => { /* unparseable line — drop */ }
|
||||
}
|
||||
}
|
||||
// persist the folded totals FIRST, then swap the trimmed file in
|
||||
inner.month_upto_ts = now_secs();
|
||||
self.write_sidecar(&inner);
|
||||
let tmp = format!("{path}.tmp");
|
||||
if std::fs::write(&tmp, kept).is_ok() {
|
||||
let _ = std::fs::rename(&tmp, path);
|
||||
}
|
||||
tracing::info!(
|
||||
new_size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0),
|
||||
"stats log compacted (older records folded into cumulative totals)"
|
||||
);
|
||||
}
|
||||
|
||||
/// Snapshot for the admin API: aggregates + time series for `range_secs` + breakdown by `groupby`.
|
||||
@@ -250,13 +450,31 @@ impl Stats {
|
||||
.filter(|b| b.minute >= cutoff)
|
||||
.collect();
|
||||
|
||||
let breakdown_src = match groupby {
|
||||
"account" => &inner.by_account,
|
||||
"tier" => &inner.by_tier,
|
||||
"label" => &inner.by_label,
|
||||
_ => &inner.by_model,
|
||||
};
|
||||
let mut breakdown: Vec<serde_json::Value> = breakdown_src
|
||||
// Windowed totals + breakdown = sum of the in-window buckets, so the KPIs
|
||||
// and the breakdown recompute whenever the range changes.
|
||||
let mut win = Totals::default();
|
||||
let mut group: HashMap<String, Totals> = HashMap::new();
|
||||
for b in &series {
|
||||
win.requests += b.requests;
|
||||
win.errors += b.errors;
|
||||
win.ok += b.requests.saturating_sub(b.errors);
|
||||
win.failovers += b.failovers;
|
||||
win.prompt_tokens += b.prompt_tokens;
|
||||
win.completion_tokens += b.completion_tokens;
|
||||
win.cost_usd += b.cost_usd;
|
||||
win.latency_ms_sum += b.latency_ms_sum;
|
||||
let src = match groupby {
|
||||
"account" => &b.by_account,
|
||||
"tier" => &b.by_tier,
|
||||
"label" => &b.by_label,
|
||||
_ => &b.by_model,
|
||||
};
|
||||
for (k, t) in src {
|
||||
group.entry(k.clone()).or_default().merge(t);
|
||||
}
|
||||
}
|
||||
|
||||
let mut breakdown: Vec<serde_json::Value> = group
|
||||
.iter()
|
||||
.map(|(k, t)| {
|
||||
serde_json::json!({
|
||||
@@ -278,19 +496,39 @@ impl Stats {
|
||||
.cmp(&a["requests"].as_u64().unwrap_or(0))
|
||||
});
|
||||
|
||||
let mut alltime: HashMap<String, Totals> = inner.folded.clone();
|
||||
for (k, t) in &inner.by_account_alltime {
|
||||
alltime.entry(k.clone()).or_default().merge(t);
|
||||
}
|
||||
let alltime_by_account: serde_json::Map<String, serde_json::Value> = alltime
|
||||
.iter()
|
||||
.map(|(k, t)| {
|
||||
(
|
||||
k.clone(),
|
||||
serde_json::json!({
|
||||
"requests": t.requests,
|
||||
"prompt_tokens": t.prompt_tokens,
|
||||
"completion_tokens": t.completion_tokens,
|
||||
"cost_usd": t.cost_usd,
|
||||
}),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
|
||||
serde_json::json!({
|
||||
"alltime_by_account": alltime_by_account,
|
||||
"totals": {
|
||||
"requests": inner.total.requests,
|
||||
"ok": inner.total.ok,
|
||||
"errors": inner.total.errors,
|
||||
"failovers": inner.total.failovers,
|
||||
"prompt_tokens": inner.total.prompt_tokens,
|
||||
"completion_tokens": inner.total.completion_tokens,
|
||||
"total_tokens": inner.total.prompt_tokens + inner.total.completion_tokens,
|
||||
"cost_usd": inner.total.cost_usd,
|
||||
"avg_latency_ms": avg(inner.total.latency_ms_sum, inner.total.requests),
|
||||
"success_rate": if inner.total.requests > 0 {
|
||||
inner.total.ok as f64 / inner.total.requests as f64
|
||||
"requests": win.requests,
|
||||
"ok": win.ok,
|
||||
"errors": win.errors,
|
||||
"failovers": win.failovers,
|
||||
"prompt_tokens": win.prompt_tokens,
|
||||
"completion_tokens": win.completion_tokens,
|
||||
"total_tokens": win.prompt_tokens + win.completion_tokens,
|
||||
"cost_usd": win.cost_usd,
|
||||
"avg_latency_ms": avg(win.latency_ms_sum, win.requests),
|
||||
"success_rate": if win.requests > 0 {
|
||||
win.ok as f64 / win.requests as f64
|
||||
} else { 0.0 },
|
||||
},
|
||||
"groupby": groupby,
|
||||
@@ -299,6 +537,21 @@ impl Stats {
|
||||
})
|
||||
}
|
||||
|
||||
/// Reset all in-memory stats and truncate the JSONL log — the "clear logs"
|
||||
/// admin action.
|
||||
pub fn clear(&self) {
|
||||
if let Some(tpath) = self.totals_path() {
|
||||
let _ = std::fs::remove_file(tpath);
|
||||
}
|
||||
{
|
||||
let mut inner = self.inner.lock().unwrap();
|
||||
*inner = Inner::default();
|
||||
}
|
||||
if let Some(path) = &self.file {
|
||||
let _ = std::fs::write(path, "");
|
||||
}
|
||||
}
|
||||
|
||||
/// Recent requests (newest first), with pagination.
|
||||
pub fn recent(&self, limit: usize, offset: usize) -> Vec<RequestRecord> {
|
||||
let inner = self.inner.lock().unwrap();
|
||||
@@ -358,10 +611,65 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
|
||||
if r.outcome != "ok" {
|
||||
b.errors += 1;
|
||||
}
|
||||
if r.failover {
|
||||
b.failovers += 1;
|
||||
}
|
||||
b.prompt_tokens += r.prompt_tokens as u64;
|
||||
b.completion_tokens += r.completion_tokens as u64;
|
||||
b.cost_usd += r.cost_usd;
|
||||
b.latency_ms_sum += r.latency_ms;
|
||||
// per-group breakdown, so the snapshot can be windowed by summing buckets
|
||||
b.by_model.entry(r.model_id.clone()).or_default().apply(r);
|
||||
b.by_account
|
||||
.entry(if r.account.is_empty() {
|
||||
"anonymous".to_string()
|
||||
} else {
|
||||
r.account.clone()
|
||||
})
|
||||
.or_default()
|
||||
.apply(r);
|
||||
b.by_tier.entry(r.tier.clone()).or_default().apply(r);
|
||||
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
|
||||
}
|
||||
|
||||
fn month_of(ts: u64) -> String {
|
||||
// календарный месяц UTC без внешних зависимостей: дни с эпохи → (год, месяц)
|
||||
let days = ts / 86_400;
|
||||
let mut year = 1970u64;
|
||||
let mut d = days;
|
||||
loop {
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let len = if leap { 366 } else { 365 };
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
year += 1;
|
||||
}
|
||||
let leap = (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
|
||||
let ml = [
|
||||
31,
|
||||
if leap { 29 } else { 28 },
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
30,
|
||||
31,
|
||||
];
|
||||
let mut month = 1;
|
||||
for len in ml {
|
||||
if d < len {
|
||||
break;
|
||||
}
|
||||
d -= len;
|
||||
month += 1;
|
||||
}
|
||||
format!("{year:04}-{month:02}")
|
||||
}
|
||||
|
||||
fn avg(sum: u64, n: u64) -> u64 {
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
//! Host metrics sampler for the admin dashboard: CPU load, RAM, free disk at
|
||||
//! the models dir, and the gateway's own RSS. One background task refreshes a
|
||||
//! snapshot every 2 s; the API endpoint only reads it.
|
||||
|
||||
use serde::Serialize;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[derive(Clone, Default, Serialize)]
|
||||
pub struct Snapshot {
|
||||
pub ts: u64,
|
||||
/// Whole-machine CPU usage 0..100 (average over all cores).
|
||||
pub cpu_percent: f32,
|
||||
pub cores: usize,
|
||||
pub load1: f64,
|
||||
pub ram_total: u64,
|
||||
pub ram_used: u64,
|
||||
/// Disk that holds models_dir.
|
||||
pub disk_total: u64,
|
||||
pub disk_free: u64,
|
||||
/// Resident memory of the gateway process itself.
|
||||
pub process_rss: u64,
|
||||
}
|
||||
|
||||
pub struct SysMon {
|
||||
snap: Mutex<Snapshot>,
|
||||
}
|
||||
|
||||
impl SysMon {
|
||||
pub fn snapshot(&self) -> Snapshot {
|
||||
self.snap.lock().unwrap().clone()
|
||||
}
|
||||
}
|
||||
|
||||
fn now() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Start the sampler. `models_dir` picks which disk's free space is reported.
|
||||
pub fn spawn(models_dir: String) -> Arc<SysMon> {
|
||||
let mon = Arc::new(SysMon {
|
||||
snap: Mutex::new(Snapshot::default()),
|
||||
});
|
||||
let ret = mon.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
use sysinfo::{Disks, ProcessesToUpdate, System};
|
||||
let mut sys = System::new();
|
||||
let pid = sysinfo::Pid::from_u32(std::process::id());
|
||||
let models_abs = std::fs::canonicalize(&models_dir)
|
||||
.unwrap_or_else(|_| std::path::PathBuf::from(&models_dir));
|
||||
loop {
|
||||
sys.refresh_cpu_usage();
|
||||
sys.refresh_memory();
|
||||
let _ = sys.refresh_processes(ProcessesToUpdate::Some(&[pid]), true);
|
||||
let disks = Disks::new_with_refreshed_list();
|
||||
// the disk whose mount point is the longest prefix of models_dir
|
||||
let mut best: Option<(usize, u64, u64)> = None;
|
||||
for d in disks.list() {
|
||||
let mp = d.mount_point();
|
||||
if models_abs.starts_with(mp) {
|
||||
let len = mp.as_os_str().len();
|
||||
if best.map(|(l, _, _)| len > l).unwrap_or(true) {
|
||||
best = Some((len, d.total_space(), d.available_space()));
|
||||
}
|
||||
}
|
||||
}
|
||||
let (disk_total, disk_free) = best.map(|(_, t, f)| (t, f)).unwrap_or((0, 0));
|
||||
let snap = Snapshot {
|
||||
ts: now(),
|
||||
cpu_percent: sys.global_cpu_usage(),
|
||||
cores: sys.cpus().len(),
|
||||
load1: System::load_average().one,
|
||||
ram_total: sys.total_memory(),
|
||||
ram_used: sys.used_memory(),
|
||||
disk_total,
|
||||
disk_free,
|
||||
process_rss: sys.process(pid).map(|p| p.memory()).unwrap_or(0),
|
||||
};
|
||||
*mon.snap.lock().unwrap() = snap;
|
||||
std::thread::sleep(std::time::Duration::from_secs(2));
|
||||
}
|
||||
});
|
||||
ret
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
//! Tool-call hygiene for OpenAI-compatible responses.
|
||||
//!
|
||||
//! Qwen-family chat templates express a function call as literal markup in the
|
||||
//! generated text:
|
||||
//!
|
||||
//! ```text
|
||||
//! <tool_call>
|
||||
//! {"name": "get_weather", "arguments": {"city": "Paris"}}
|
||||
//! </tool_call>
|
||||
//! ```
|
||||
//!
|
||||
//! Runtimes handle that inconsistently. Some parse it into structured
|
||||
//! `tool_calls` **and** still stream the raw markup as `content` — the client
|
||||
//! then sees the call twice, once as prose it must not show and once as the real
|
||||
//! thing, and tool-loop implementations (Open WebUI's among them) get confused.
|
||||
//! Others never parse it at all, so a perfectly good call reaches the client as
|
||||
//! text no tool loop will ever execute.
|
||||
//!
|
||||
//! This module makes the gateway the place where that is settled: markup is
|
||||
//! stripped from the text, and when nothing structured came with it the markup
|
||||
//! is promoted into a real `tool_calls` array.
|
||||
|
||||
const OPEN: &str = "<tool_call>";
|
||||
const CLOSE: &str = "</tool_call>";
|
||||
|
||||
/// Longest suffix of `s` that is a proper prefix of `pat`. Used to hold back the
|
||||
/// tail of a streamed chunk that might be the beginning of a marker split across
|
||||
/// chunk boundaries (`"<tool"` + `"_call>"`).
|
||||
fn partial_marker_len(s: &str, pat: &str) -> usize {
|
||||
let max = pat.len().saturating_sub(1).min(s.len());
|
||||
(1..=max)
|
||||
.rev()
|
||||
.find(|&n| {
|
||||
s.is_char_boundary(s.len() - n)
|
||||
&& pat.as_bytes().starts_with(&s.as_bytes()[s.len() - n..])
|
||||
})
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Incremental `<tool_call>` filter for SSE deltas.
|
||||
///
|
||||
/// Feed it content fragments in order; it returns the text that is safe to
|
||||
/// forward and keeps the markup payloads for [`Self::captured`].
|
||||
#[derive(Default)]
|
||||
pub struct ToolMarkupFilter {
|
||||
/// Text held back: either an unterminated marker prefix or the inside of an
|
||||
/// open `<tool_call>` block.
|
||||
pending: String,
|
||||
inside: bool,
|
||||
captured: Vec<String>,
|
||||
}
|
||||
|
||||
impl ToolMarkupFilter {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Consume one content fragment; returns the part that may be forwarded now.
|
||||
pub fn push(&mut self, chunk: &str) -> String {
|
||||
self.pending.push_str(chunk);
|
||||
let mut out = String::new();
|
||||
loop {
|
||||
if self.inside {
|
||||
let Some(end) = self.pending.find(CLOSE) else {
|
||||
// still inside the block — nothing to forward
|
||||
return out;
|
||||
};
|
||||
let payload: String = self.pending[..end].to_string();
|
||||
self.captured.push(payload);
|
||||
self.pending = self.pending[end + CLOSE.len()..].to_string();
|
||||
self.inside = false;
|
||||
continue;
|
||||
}
|
||||
match self.pending.find(OPEN) {
|
||||
Some(start) => {
|
||||
out.push_str(&self.pending[..start]);
|
||||
self.pending = self.pending[start + OPEN.len()..].to_string();
|
||||
self.inside = true;
|
||||
}
|
||||
None => {
|
||||
// Hold back a trailing partial marker so a split like
|
||||
// "…<tool" / "_call>…" is not forwarded as visible text.
|
||||
let hold = partial_marker_len(&self.pending, OPEN);
|
||||
let cut = self.pending.len() - hold;
|
||||
out.push_str(&self.pending[..cut]);
|
||||
self.pending = self.pending[cut..].to_string();
|
||||
return out;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// End of stream: an unterminated block was never a tool call after all, so
|
||||
/// give the text back rather than swallowing output.
|
||||
pub fn finish(&mut self) -> String {
|
||||
let mut out = String::new();
|
||||
if self.inside {
|
||||
out.push_str(OPEN);
|
||||
self.inside = false;
|
||||
}
|
||||
out.push_str(&std::mem::take(&mut self.pending));
|
||||
out
|
||||
}
|
||||
|
||||
/// Payloads found between `<tool_call>` and `</tool_call>`.
|
||||
pub fn captured(&self) -> &[String] {
|
||||
&self.captured
|
||||
}
|
||||
}
|
||||
|
||||
/// Strip every complete `<tool_call>` block from a finished string.
|
||||
/// Returns the cleaned text and the payloads that were removed.
|
||||
pub fn split_tool_markup(text: &str) -> (String, Vec<String>) {
|
||||
let mut f = ToolMarkupFilter::new();
|
||||
let mut out = f.push(text);
|
||||
out.push_str(&f.finish());
|
||||
(out, f.captured)
|
||||
}
|
||||
|
||||
/// Turn `<tool_call>` payloads into OpenAI `tool_calls` entries, numbered from
|
||||
/// `start_index`. Payloads that are not a JSON object with a `name` are skipped —
|
||||
/// a stray marker must not become a bogus call.
|
||||
pub fn markup_to_tool_calls(payloads: &[String], start_index: usize) -> Vec<serde_json::Value> {
|
||||
let mut out = Vec::new();
|
||||
for payload in payloads {
|
||||
let Ok(v) = serde_json::from_str::<serde_json::Value>(payload.trim()) else {
|
||||
continue;
|
||||
};
|
||||
let Some(name) = v.get("name").and_then(|n| n.as_str()) else {
|
||||
continue;
|
||||
};
|
||||
// `arguments` is a JSON string on the wire; templates emit an object.
|
||||
let arguments = match v.get("arguments").or_else(|| v.get("parameters")) {
|
||||
Some(serde_json::Value::String(s)) => s.clone(),
|
||||
Some(other) => other.to_string(),
|
||||
None => "{}".to_string(),
|
||||
};
|
||||
out.push(serde_json::json!({
|
||||
"index": start_index + out.len(),
|
||||
"id": new_call_id(),
|
||||
"type": "function",
|
||||
"function": { "name": name, "arguments": arguments },
|
||||
}));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
pub fn new_call_id() -> String {
|
||||
format!("call_{}", crate::admin::random_token(12))
|
||||
}
|
||||
|
||||
/// Bring one streamed `tool_calls` array up to the OpenAI shape.
|
||||
///
|
||||
/// `seen` tracks which choice-local indices already had their opening delta, so
|
||||
/// the identity fields (`id`, `type`) are added once and continuation deltas —
|
||||
/// which legitimately carry nothing but an argument fragment — are left alone.
|
||||
pub fn normalize_delta_tool_calls(
|
||||
raw: &[serde_json::Value],
|
||||
seen: &mut std::collections::HashSet<u64>,
|
||||
) -> Vec<serde_json::Value> {
|
||||
let mut out = Vec::new();
|
||||
for (i, tc) in raw.iter().enumerate() {
|
||||
let index = tc.get("index").and_then(|v| v.as_u64()).unwrap_or(i as u64);
|
||||
let mut entry = serde_json::Map::new();
|
||||
entry.insert("index".into(), serde_json::json!(index));
|
||||
|
||||
let first = seen.insert(index);
|
||||
if let Some(id) = tc
|
||||
.get("id")
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
{
|
||||
entry.insert("id".into(), serde_json::json!(id));
|
||||
} else if first {
|
||||
entry.insert("id".into(), serde_json::json!(new_call_id()));
|
||||
}
|
||||
if first {
|
||||
let kind = tc
|
||||
.get("type")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("function");
|
||||
entry.insert("type".into(), serde_json::json!(kind));
|
||||
}
|
||||
|
||||
if let Some(f) = tc.get("function") {
|
||||
let mut func = serde_json::Map::new();
|
||||
if let Some(name) = f.get("name").and_then(|v| v.as_str()) {
|
||||
func.insert("name".into(), serde_json::json!(name));
|
||||
} else if first {
|
||||
func.insert("name".into(), serde_json::json!(""));
|
||||
}
|
||||
match f.get("arguments") {
|
||||
// Arguments are a string on the wire; some runtimes send the object.
|
||||
Some(serde_json::Value::String(s)) => {
|
||||
func.insert("arguments".into(), serde_json::json!(s));
|
||||
}
|
||||
Some(serde_json::Value::Null) | None => {
|
||||
if first {
|
||||
func.insert("arguments".into(), serde_json::json!(""));
|
||||
}
|
||||
}
|
||||
Some(other) => {
|
||||
func.insert("arguments".into(), serde_json::json!(other.to_string()));
|
||||
}
|
||||
}
|
||||
entry.insert("function".into(), serde_json::Value::Object(func));
|
||||
}
|
||||
out.push(serde_json::Value::Object(entry));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const CALL: &str = r#"<tool_call>
|
||||
{"name": "get_weather", "arguments": {"city": "Paris"}}
|
||||
</tool_call>"#;
|
||||
|
||||
#[test]
|
||||
fn markup_is_stripped_from_finished_text() {
|
||||
let (clean, payloads) = split_tool_markup(&format!("Sure!{CALL}"));
|
||||
assert_eq!(clean, "Sure!");
|
||||
assert_eq!(payloads.len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn markup_split_across_stream_chunks_never_leaks() {
|
||||
let mut f = ToolMarkupFilter::new();
|
||||
let mut seen = String::new();
|
||||
// the marker itself is broken across three fragments
|
||||
for chunk in [
|
||||
"Let me check. <tool",
|
||||
"_call>",
|
||||
"{\"name\":\"get_weather\",",
|
||||
"\"arguments\":{\"city\":\"Paris\"}}",
|
||||
"</tool",
|
||||
"_call>",
|
||||
" done",
|
||||
] {
|
||||
seen.push_str(&f.push(chunk));
|
||||
}
|
||||
seen.push_str(&f.finish());
|
||||
assert_eq!(seen, "Let me check. done");
|
||||
assert!(!seen.contains("tool_call"));
|
||||
assert_eq!(f.captured().len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn captured_markup_becomes_a_real_tool_call() {
|
||||
let (_, payloads) = split_tool_markup(CALL);
|
||||
let calls = markup_to_tool_calls(&payloads, 0);
|
||||
assert_eq!(calls.len(), 1);
|
||||
assert_eq!(calls[0]["type"], "function");
|
||||
assert_eq!(calls[0]["index"], 0);
|
||||
assert_eq!(calls[0]["function"]["name"], "get_weather");
|
||||
// arguments must be a STRING, not an object
|
||||
assert_eq!(
|
||||
calls[0]["function"]["arguments"].as_str().unwrap(),
|
||||
r#"{"city":"Paris"}"#
|
||||
);
|
||||
assert!(calls[0]["id"].as_str().unwrap().starts_with("call_"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_stray_marker_does_not_become_a_call() {
|
||||
let calls = markup_to_tool_calls(&["not json at all".into(), "{\"no_name\":1}".into()], 0);
|
||||
assert!(calls.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unterminated_block_is_given_back_as_text() {
|
||||
let mut f = ToolMarkupFilter::new();
|
||||
let mut out = f.push("thinking <tool_call>{\"name\":\"x\"");
|
||||
out.push_str(&f.finish());
|
||||
assert_eq!(out, "thinking <tool_call>{\"name\":\"x\"");
|
||||
assert!(f.captured().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_tool_calls_get_identity_once_and_object_arguments_become_strings() {
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
let opening = normalize_delta_tool_calls(
|
||||
&[serde_json::json!({"index": 0, "function": {"name": "f", "arguments": {"a": 1}}})],
|
||||
&mut seen,
|
||||
);
|
||||
assert_eq!(opening[0]["type"], "function");
|
||||
assert!(opening[0]["id"].as_str().unwrap().starts_with("call_"));
|
||||
assert_eq!(opening[0]["function"]["arguments"], r#"{"a":1}"#);
|
||||
|
||||
// a continuation delta must not be handed a second id
|
||||
let cont = normalize_delta_tool_calls(
|
||||
&[serde_json::json!({"index": 0, "function": {"arguments": "\"}"}})],
|
||||
&mut seen,
|
||||
);
|
||||
assert!(cont[0].get("id").is_none());
|
||||
assert!(cont[0].get("type").is_none());
|
||||
}
|
||||
}
|
||||
Vendored
+314
@@ -397,3 +397,317 @@ tbody tr:hover td { background: var(--hover); }
|
||||
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
|
||||
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
|
||||
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
|
||||
|
||||
/* ---- responsive shell + touch layout ----------------------------------- */
|
||||
.menu-toggle, .nav-close, .mobile-nav-backdrop { display: none; }
|
||||
.hamburger { width: 18px; display: flex; flex-direction: column; gap: 4px; }
|
||||
.hamburger i { display: block; height: 2px; border-radius: 2px; background: currentColor; }
|
||||
body.modal-open { overflow: hidden; }
|
||||
|
||||
@media (max-width: 980px) {
|
||||
.app { display: block; min-height: 100dvh; }
|
||||
.main { min-height: 100dvh; }
|
||||
.menu-toggle { display: inline-flex; flex: 0 0 auto; }
|
||||
.nav-close { display: inline-flex; margin-left: auto; flex: 0 0 auto; font-size: 24px; line-height: 1; }
|
||||
.brand-copy { min-width: 0; }
|
||||
|
||||
.sidebar {
|
||||
display: flex;
|
||||
position: fixed;
|
||||
inset: 0 auto 0 0;
|
||||
width: min(84vw, 310px);
|
||||
height: 100dvh;
|
||||
padding: max(18px, env(safe-area-inset-top)) 14px max(14px, env(safe-area-inset-bottom));
|
||||
z-index: 70;
|
||||
box-shadow: 18px 0 45px rgba(0, 0, 0, .42);
|
||||
transform: translateX(-104%);
|
||||
visibility: hidden;
|
||||
transition: transform .22s ease, visibility 0s linear .22s;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.app.nav-open .sidebar {
|
||||
transform: translateX(0);
|
||||
visibility: visible;
|
||||
transition: transform .22s ease;
|
||||
}
|
||||
.mobile-nav-backdrop {
|
||||
display: block;
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 65;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border: 0;
|
||||
background: rgba(3, 7, 18, .62);
|
||||
opacity: 0;
|
||||
visibility: hidden;
|
||||
pointer-events: none;
|
||||
transition: opacity .22s ease, visibility 0s linear .22s;
|
||||
}
|
||||
.app.nav-open .mobile-nav-backdrop {
|
||||
opacity: 1;
|
||||
visibility: visible;
|
||||
pointer-events: auto;
|
||||
transition: opacity .22s ease;
|
||||
}
|
||||
body.mobile-nav-open { overflow: hidden; }
|
||||
|
||||
.topbar { padding: 10px 18px; min-height: 58px; gap: 10px; }
|
||||
.topbar .title { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.topbar .right { gap: 8px; min-width: 0; }
|
||||
.content { max-width: none; padding: 22px 20px 52px; }
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
body { font-size: 14px; }
|
||||
.status-pills { display: none; }
|
||||
.topbar { padding: 9px 12px; }
|
||||
.content { padding: 18px 14px 42px; }
|
||||
|
||||
.page-head { margin-bottom: 16px; }
|
||||
.page-head > .flex { align-items: stretch; flex-direction: column; }
|
||||
.page-head > .flex > .btn { width: 100%; }
|
||||
.page-head p { line-height: 1.45; }
|
||||
|
||||
.card { padding: 15px; margin-bottom: 14px; border-radius: 12px; }
|
||||
.card .card-head { align-items: flex-start; flex-wrap: wrap; }
|
||||
.card .card-head .right { width: 100%; margin-left: 0; flex-wrap: wrap; }
|
||||
.grid { gap: 14px; }
|
||||
.kpis { gap: 10px; margin-bottom: 14px; }
|
||||
.kpi { padding: 12px; border-radius: 11px; }
|
||||
.kpi .big { font-size: 21px; }
|
||||
|
||||
.row { display: grid; grid-template-columns: minmax(0, 1fr); gap: 0; }
|
||||
.row > * { min-width: 0; width: 100%; }
|
||||
input, select, textarea { min-height: 44px; font-size: 16px; }
|
||||
input[type="checkbox"], input[type="radio"] { min-height: 0; }
|
||||
.check { min-height: 36px; margin-bottom: 8px; }
|
||||
.btn, .icon-btn { min-height: 44px; }
|
||||
.btn { padding: 10px 14px; }
|
||||
.btn.sm, .btn.small { min-height: 38px; padding: 8px 11px; }
|
||||
|
||||
.table-wrap {
|
||||
width: 100%;
|
||||
overflow-x: auto;
|
||||
overscroll-behavior-x: contain;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: thin;
|
||||
}
|
||||
.table-wrap table { min-width: max-content; }
|
||||
th, td { padding: 10px; }
|
||||
|
||||
.tierrow { align-items: flex-start; flex-direction: column; gap: 8px; }
|
||||
.tierrow .tname { width: auto; }
|
||||
.chips { width: 100%; }
|
||||
.togglerow { align-items: flex-start; }
|
||||
|
||||
.pg { gap: 14px; }
|
||||
.routecard .kv { flex-wrap: wrap; }
|
||||
.routecard .kv .v { max-width: 100%; overflow-wrap: anywhere; }
|
||||
.answer { padding: 12px; }
|
||||
.seg { max-width: 100%; overflow-x: auto; }
|
||||
.seg button { min-height: 40px; white-space: nowrap; }
|
||||
|
||||
.hf-grid { grid-template-columns: 1fr; max-height: 56dvh; }
|
||||
.job-head, .actions { align-items: stretch; flex-wrap: wrap; }
|
||||
.actions .btn { flex: 1 1 150px; }
|
||||
.key-row { flex-direction: column; }
|
||||
.key-row .btn { width: 100%; }
|
||||
.cmf-server-row { align-items: stretch !important; padding: 10px; border: 1px solid var(--border-soft); border-radius: 10px; }
|
||||
.cmf-server-row > input, .cmf-server-row > select { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
|
||||
.cmf-server-row > .btn { flex: 1 1 calc(50% - 6px); }
|
||||
.cmf-server-row > .small { flex: 1 1 100%; }
|
||||
|
||||
.modal-backdrop { align-items: flex-end; padding: 8px 0 0; }
|
||||
.modal {
|
||||
max-width: none;
|
||||
max-height: calc(100dvh - 8px);
|
||||
overflow-y: auto;
|
||||
padding: 18px 15px max(14px, env(safe-area-inset-bottom));
|
||||
border-radius: 18px 18px 0 0;
|
||||
}
|
||||
.modal .modal-foot {
|
||||
position: sticky;
|
||||
bottom: calc(-1 * max(14px, env(safe-area-inset-bottom)));
|
||||
margin: 12px -15px 0;
|
||||
padding: 12px 15px max(14px, env(safe-area-inset-bottom));
|
||||
background: color-mix(in srgb, var(--card) 94%, transparent);
|
||||
backdrop-filter: blur(10px);
|
||||
border-top: 1px solid var(--border-soft);
|
||||
}
|
||||
.modal .modal-foot .btn { flex: 1; }
|
||||
|
||||
.toasts { left: 12px; right: 12px; bottom: max(12px, env(safe-area-inset-bottom)); }
|
||||
.toast { min-width: 0; width: 100%; }
|
||||
.login-wrap { align-items: flex-start; padding: max(20px, env(safe-area-inset-top)) 14px; }
|
||||
.login-card { padding: 22px 18px; margin-top: 8dvh; }
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.topbar .title { font-size: 14px; }
|
||||
.topbar .right { gap: 6px; }
|
||||
.lang-select { width: 54px; min-height: 44px; padding: 6px; }
|
||||
.icon-btn { width: 42px; height: 42px; }
|
||||
.kpis { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.kpi .lbl { line-height: 1.25; }
|
||||
.barlist .b { grid-template-columns: minmax(0, 1fr) auto; gap: 6px 10px; }
|
||||
.barlist .b .track { grid-column: 1 / -1; grid-row: 2; }
|
||||
.barlist .b .val { min-width: 0; }
|
||||
.empty { padding: 28px 12px; }
|
||||
}
|
||||
|
||||
@media (max-width: 360px) {
|
||||
.topbar .title { display: none; }
|
||||
.kpis { grid-template-columns: 1fr; }
|
||||
}
|
||||
|
||||
/* ---- onboarding ------------------------------------------------------------ */
|
||||
.onb-shell { min-height: 100vh; display: flex; align-items: flex-start; justify-content: center; padding: 32px 20px; background: var(--bg-grad); }
|
||||
.onb-content { width: 100%; max-width: 860px; }
|
||||
.onb-wrap { max-width: 860px; margin: 0 auto; }
|
||||
.onb-header { display: flex; align-items: center; justify-content: space-between; margin-bottom: 18px; }
|
||||
.onb-dots { display: flex; gap: 8px; }
|
||||
.onb-dots .dot { width: 28px; height: 6px; border-radius: 999px; background: var(--border); }
|
||||
.onb-dots .dot.on { background: var(--accent); }
|
||||
.onb-dots .dot.done { background: color-mix(in srgb, var(--accent) 55%, transparent); }
|
||||
.onb-title { font-size: 12px; font-weight: 700; color: var(--muted); letter-spacing: 0.04em; text-transform: uppercase; }
|
||||
.onb-step h2 { font-size: 22px; margin-bottom: 6px; }
|
||||
.onb-langs { display: grid; grid-template-columns: repeat(auto-fill, minmax(132px, 1fr)); gap: 12px; margin: 18px 0; }
|
||||
.onb-lang {
|
||||
display: flex; flex-direction: column; align-items: center; gap: 10px;
|
||||
padding: 18px 10px; border-radius: 14px; border: 1.5px solid var(--border);
|
||||
background: var(--card); cursor: pointer; transition: .15s; font-family: inherit;
|
||||
}
|
||||
.onb-lang:hover { border-color: color-mix(in srgb, var(--accent) 45%, transparent); transform: translateY(-1px); }
|
||||
.onb-lang.sel { border-color: var(--accent); background: color-mix(in srgb, var(--accent) 10%, var(--card)); box-shadow: 0 6px 18px rgba(99,102,241,.18); }
|
||||
.onb-lang .flag { font-size: 32px; line-height: 1; }
|
||||
.onb-lang b { font-size: 14px; font-weight: 700; color: var(--text); }
|
||||
/* Featured gallery: the list grows as new models are published, so give it more
|
||||
room than the search grid before it starts scrolling. */
|
||||
.featured-section .hf-grid { max-height: min(62vh, 760px); }
|
||||
/* Money: 4 decimals, last two de-emphasized */
|
||||
.money-tail { font-size: .72em; opacity: .75; margin-left: 1px; }
|
||||
|
||||
.seg { flex-wrap: wrap; }
|
||||
|
||||
.ok { color: var(--good); }
|
||||
|
||||
.pairs { display: flex; flex-direction: column; gap: 6px; align-items: flex-start; }
|
||||
.pair-row { display: flex; gap: 6px; align-items: center; }
|
||||
.pair-model { min-width: 220px; max-width: 340px; padding: 4px 8px; font-size: 12.5px; }
|
||||
.pair-tier { max-width: 150px; padding: 4px 8px; font-size: 12.5px; }
|
||||
|
||||
/* Complexity bands slider */
|
||||
.bands-wrap { margin: 4px 0 16px; }
|
||||
/* one-line command chip (install commands etc.) — theme-aware */
|
||||
.cmd-chip { background: var(--panel); border: 1px solid var(--border); color: var(--text); padding: 2px 8px; border-radius: 6px; }
|
||||
|
||||
.bands-track { position: relative; height: 14px; border-radius: 8px; background: var(--border); margin: 26px 8px 10px; }
|
||||
.bands-track.disabled { opacity: .45; pointer-events: none; }
|
||||
.bands-zone { position: absolute; top: 0; bottom: 0; }
|
||||
.bands-zone.z-low { left: 0; background: color-mix(in srgb, var(--good) 45%, transparent); border-radius: 8px 0 0 8px; }
|
||||
.bands-zone.z-med { background: color-mix(in srgb, var(--warn, #f59e0b) 55%, transparent); }
|
||||
.bands-zone.z-high { background: color-mix(in srgb, #dc2626 65%, transparent); border-radius: 0 8px 8px 0; }
|
||||
.bands-handle { position: absolute; top: -5px; width: 24px; height: 24px; margin-left: -12px; border-radius: 50%;
|
||||
background: #fff; border: 3px solid var(--accent); cursor: grab; touch-action: none; z-index: 2; }
|
||||
.bands-handle:active { cursor: grabbing; }
|
||||
.bands-val { position: absolute; top: -24px; left: 50%; transform: translateX(-50%); font-size: 11.5px;
|
||||
font-family: var(--mono, monospace); color: var(--text); background: var(--panel); padding: 1px 6px;
|
||||
border-radius: 6px; border: 1px solid var(--border, #333); white-space: nowrap; }
|
||||
.bands-val.raised { top: -46px; }
|
||||
.bands-legend { display: flex; gap: 8px; margin: 6px 8px 0; flex-wrap: wrap; }
|
||||
|
||||
/* Benchmark view */
|
||||
@keyframes bm-in { from { opacity: 0; transform: translateY(-3px); } to { opacity: 1; transform: none; } }
|
||||
tbody tr:last-child { animation: bm-in .25s ease; }
|
||||
.bm-sum { margin-bottom: 10px; }
|
||||
.bm-cell { cursor: pointer; white-space: nowrap; }
|
||||
.bm-cell:hover { background: var(--hover); }
|
||||
.badge.ok { background: rgba(52,211,153,.15); color: var(--good); }
|
||||
|
||||
/* Routing: per-task rule rows */
|
||||
.taskrow { align-items: flex-start; }
|
||||
.taskname { min-width: 190px; line-height: 1.35; }
|
||||
.taskname .mono { font-size: 10.5px; opacity: .75; }
|
||||
.chips-empty { padding: 4px 0; font-style: italic; }
|
||||
|
||||
/* Shell notices (open admin warning, update available) */
|
||||
.notice {
|
||||
display: flex; align-items: center; gap: 8px;
|
||||
padding: 8px 18px; font-size: 13px;
|
||||
border-bottom: 1px solid var(--border-soft, #232833);
|
||||
}
|
||||
.notice.warn { background: color-mix(in srgb, #e5a54b 12%, var(--bg)); }
|
||||
.notice.info { background: color-mix(in srgb, var(--accent) 10%, var(--bg)); }
|
||||
.notice .icon-btn { margin-left: auto; }
|
||||
|
||||
/* Dashboard host metrics */
|
||||
.sysgrid { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 16px; }
|
||||
.sysitem .k { color: var(--muted); font-size: 12px; }
|
||||
.sysitem .v { font-weight: 600; }
|
||||
.sysbar {
|
||||
height: 8px; border-radius: 6px; overflow: hidden; margin: 6px 0;
|
||||
background: var(--bg-2, #12151c); border: 1px solid var(--border, #2a2f3a);
|
||||
}
|
||||
.sysbar i { display: block; height: 100%; background: var(--accent, #5b8def); }
|
||||
.sysbar i.bad { background: #e5534b; }
|
||||
|
||||
/* Log viewer */
|
||||
.logs-view { max-height: 68vh; }
|
||||
|
||||
/* Chat dialogue bubbles */
|
||||
.chat-log { max-height: 56vh; overflow-y: auto; display: flex; flex-direction: column; gap: 8px; }
|
||||
.chat-msg { display: flex; }
|
||||
.chat-msg.user { justify-content: flex-end; }
|
||||
.chat-bubble {
|
||||
max-width: 85%; padding: 8px 12px; border-radius: 12px;
|
||||
white-space: pre-wrap; word-break: break-word; font-size: 13.5px;
|
||||
background: var(--bg-2, #12151c); border: 1px solid var(--border-soft, #232833);
|
||||
}
|
||||
.chat-msg.user .chat-bubble {
|
||||
background: color-mix(in srgb, var(--accent) 16%, var(--bg-2));
|
||||
border-color: color-mix(in srgb, var(--accent) 35%, transparent);
|
||||
}
|
||||
|
||||
/* Generation gallery */
|
||||
.gal-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(110px, 1fr)); gap: 8px; }
|
||||
.gal-item {
|
||||
position: relative; cursor: pointer; border: 1px solid var(--border, #2a2f3a);
|
||||
border-radius: 8px; overflow: hidden; background: var(--bg-2, #12151c);
|
||||
}
|
||||
.gal-item:hover { border-color: var(--accent, #5b8def); }
|
||||
.gal-thumb { width: 100%; display: block; min-height: 64px; }
|
||||
.gal-video { display: flex; align-items: center; justify-content: center; font-size: 28px; height: 72px; }
|
||||
.gal-cap { font-size: 10px; color: var(--muted); padding: 3px 6px; }
|
||||
.gal-del {
|
||||
position: absolute; top: 2px; right: 2px; z-index: 2;
|
||||
border: 0; border-radius: 50%; width: 18px; height: 18px; line-height: 16px;
|
||||
background: rgba(0,0,0,.55); color: #fff; cursor: pointer; font-size: 12px;
|
||||
}
|
||||
|
||||
/* Bench + badges */
|
||||
.bench-row { gap: 10px; padding: 6px 0; border-top: 1px solid var(--border-soft, #232833); }
|
||||
.bench-row:first-of-type { border-top: 0; }
|
||||
.badge.warn { background: rgba(229,165,75,.15); color: #e5a54b; }
|
||||
.snippet { max-height: 260px; }
|
||||
|
||||
/* Playground media tabs: generated image / video frame */
|
||||
.media-out { display: flex; flex-direction: column; gap: 8px; }
|
||||
.media-canvas {
|
||||
max-width: 100%; height: auto; border-radius: 8px;
|
||||
border: 1px solid var(--border, #2a2f3a); background: #000;
|
||||
}
|
||||
/* Live download progress inside a featured card (empty until a job runs) */
|
||||
.card-prog:empty { display: none; }
|
||||
.card-prog { margin-top: 6px; }
|
||||
.card-prog .pbar { margin: 4px 0; }
|
||||
.card-prog .btn.sm { padding: 0 6px; line-height: 18px; }
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sidebar, .mobile-nav-backdrop { transition: none; }
|
||||
}
|
||||
|
||||
/* full-screen farewell after "stop everything & power off" */
|
||||
.shutdown-screen { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100vh; gap: 12px; text-align: center; padding: 24px; }
|
||||
.shutdown-screen .shutdown-icon { font-size: 64px; color: var(--bad); }
|
||||
.shutdown-screen .hint { max-width: 480px; }
|
||||
|
||||
Vendored
+1
-1
@@ -19,6 +19,6 @@
|
||||
<noscript>This console requires JavaScript.</noscript>
|
||||
<div id="app"><div class="boot">Loading…</div></div>
|
||||
<div id="toasts" class="toasts"></div>
|
||||
<script type="module" src="/admin/js/app.js"></script>
|
||||
<script type="module" src="/admin/js/app.js?v=3"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Vendored
+101
-5
@@ -19,7 +19,10 @@ export function setToken(tok) {
|
||||
}
|
||||
|
||||
async function j(method, path, body) {
|
||||
const opt = { method, headers: { Authorization: "Bearer " + getToken() } };
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const opt = { method, headers };
|
||||
if (body !== undefined) {
|
||||
opt.headers["content-type"] = "application/json";
|
||||
opt.body = JSON.stringify(body);
|
||||
@@ -65,10 +68,13 @@ export const api = {
|
||||
|
||||
// routing
|
||||
getRouting: () => j("GET", "/routing"),
|
||||
routingLabels: () => j("GET", "/routing/labels"),
|
||||
putRouting: (b) => j("PUT", "/routing", b),
|
||||
|
||||
// protocols
|
||||
getProtocols: () => j("GET", "/protocols"),
|
||||
harnesses: () => j("GET", "/harnesses"),
|
||||
balance: () => j("GET", "/balance"),
|
||||
putProtocols: (b) => j("PUT", "/protocols", b),
|
||||
|
||||
// settings
|
||||
@@ -87,6 +93,7 @@ export const api = {
|
||||
|
||||
// stats
|
||||
stats: (range, groupby) => j("GET", `/stats?range=${range}&groupby=${groupby}`),
|
||||
clearStats: () => j("DELETE", "/stats"),
|
||||
requests: (limit = 50, offset = 0) => j("GET", `/requests?limit=${limit}&offset=${offset}`),
|
||||
|
||||
// playground
|
||||
@@ -113,14 +120,64 @@ export const api = {
|
||||
deleteImport: (job) => j("DELETE", "/import/" + encodeURIComponent(job)),
|
||||
registerImport: (job) =>
|
||||
j("POST", "/import/" + encodeURIComponent(job) + "/register"),
|
||||
|
||||
// media generation (cortiq imagine / animate over local .cmf)
|
||||
mediaModels: () => j("GET", "/media/models"),
|
||||
mediaGenerate: (b) => j("POST", "/media", b),
|
||||
mediaJobs: () => j("GET", "/media"),
|
||||
mediaJob: (id) => j("GET", "/media/" + encodeURIComponent(id)),
|
||||
mediaCancel: (id) => j("POST", "/media/" + encodeURIComponent(id) + "/cancel"),
|
||||
mediaDelete: (id) => j("DELETE", "/media/" + encodeURIComponent(id)),
|
||||
|
||||
// system / logs / lifecycle
|
||||
system: () => j("GET", "/system"),
|
||||
logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
|
||||
clearLogs: () => j("DELETE", "/logs"),
|
||||
restart: () => j("POST", "/restart"),
|
||||
shutdown: () => j("POST", "/shutdown"),
|
||||
benchStart: (model) => j("POST", "/bench", { model }),
|
||||
benchList: () => j("GET", "/bench"),
|
||||
|
||||
// first-run onboarding
|
||||
setupStatus: () => j("GET", "/setup/status"),
|
||||
// Connects every finished download as a local model unless register === false.
|
||||
setupComplete: (lang, register) => j("POST", "/setup/complete", { lang, register }),
|
||||
setupFeatured: () => j("GET", "/setup/featured"),
|
||||
// `name` is the exact output stem from setup/featured — it wins over the
|
||||
// repo/variant guess so the delete always hits the file the download wrote.
|
||||
deleteSetupFile: (repo, variant, name) =>
|
||||
j(
|
||||
"DELETE",
|
||||
`/setup/file?repo=${encodeURIComponent(repo)}` +
|
||||
(variant ? `&variant=${encodeURIComponent(variant)}` : "") +
|
||||
(name ? `&name=${encodeURIComponent(name)}` : "")
|
||||
),
|
||||
};
|
||||
|
||||
// Raw-body POST (no JSON encoding) — the config import takes TOML text as-is.
|
||||
export async function postRaw(path, text) {
|
||||
const tok = getToken();
|
||||
const headers = { "content-type": "application/toml" };
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(BASE + path, { method: "POST", headers, body: text });
|
||||
const body = await r.text();
|
||||
if (!r.ok) {
|
||||
let msg = body;
|
||||
try { msg = JSON.parse(body).error?.message || body; } catch { /* raw */ }
|
||||
throw new Error(msg || r.statusText);
|
||||
}
|
||||
return body ? JSON.parse(body) : null;
|
||||
}
|
||||
|
||||
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
|
||||
// returns the routing decision read from X-Cortiq-* response headers.
|
||||
export async function testStream(body, onDelta) {
|
||||
const tok = getToken();
|
||||
const headers = { "content-type": "application/json" };
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(BASE + "/test/stream", {
|
||||
method: "POST",
|
||||
headers: { Authorization: "Bearer " + getToken(), "content-type": "application/json" },
|
||||
headers,
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (r.status === 401) {
|
||||
@@ -133,6 +190,11 @@ export async function testStream(body, onDelta) {
|
||||
const reader = r.body.getReader();
|
||||
const dec = new TextDecoder();
|
||||
let buf = "";
|
||||
const t0 = performance.now();
|
||||
let ttftMs = null; // time to first content token
|
||||
let usage = null; // OpenAI-style final-chunk usage, when the provider sends it
|
||||
let usageEstimated = false; // gateway-injected estimate rather than tokenizer truth
|
||||
let costUsd = null; // final cost from the gateway's closing event (headers carry 0)
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
@@ -145,8 +207,17 @@ export async function testStream(body, onDelta) {
|
||||
const data = line.slice(5).trim();
|
||||
if (!data || data === "[DONE]") continue;
|
||||
try {
|
||||
const d = JSON.parse(data).choices?.[0]?.delta?.content;
|
||||
if (d) onDelta(d);
|
||||
const chunk = JSON.parse(data);
|
||||
if (chunk.usage) {
|
||||
usage = chunk.usage;
|
||||
usageEstimated = !!chunk.cortiq_estimated;
|
||||
}
|
||||
if (chunk.cortiq_cost_usd != null) costUsd = chunk.cortiq_cost_usd;
|
||||
const d = chunk.choices?.[0]?.delta?.content;
|
||||
if (d) {
|
||||
if (ttftMs == null) ttftMs = Math.round(performance.now() - t0);
|
||||
onDelta(d);
|
||||
}
|
||||
} catch {
|
||||
/* ignore malformed chunk */
|
||||
}
|
||||
@@ -159,6 +230,31 @@ export async function testStream(body, onDelta) {
|
||||
score: parseFloat(h.get("x-cortiq-complexity-score") || "0"),
|
||||
selected_model: h.get("x-cortiq-selected-model") || "",
|
||||
route_source: h.get("x-cortiq-route-source") || "",
|
||||
cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"),
|
||||
cost_usd: costUsd != null ? costUsd : parseFloat(h.get("x-cortiq-cost-usd") || "0"),
|
||||
usage,
|
||||
usage_estimated: usageEstimated,
|
||||
ttft_ms: ttftMs,
|
||||
};
|
||||
}
|
||||
|
||||
// Fetch a media job's output with the admin token and hand back an object URL
|
||||
// (plain <img src>/<video src> cannot carry the Authorization header).
|
||||
export async function mediaFileUrl(job, kind) {
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
|
||||
if (!r.ok) throw new Error((await r.text()) || r.statusText);
|
||||
return { url: URL.createObjectURL(await r.blob()), size: +(r.headers.get("content-length") || 0) };
|
||||
}
|
||||
|
||||
// Same fetch, raw bytes — for outputs the browser cannot display natively
|
||||
// (P6 PPM, MJPEG AVI) that we decode in JS.
|
||||
export async function mediaFileBytes(job, kind) {
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(`${BASE}/media/${encodeURIComponent(job)}/file/${encodeURIComponent(kind)}`, { headers });
|
||||
if (!r.ok) throw new Error((await r.text()) || r.statusText);
|
||||
return r.arrayBuffer();
|
||||
}
|
||||
|
||||
Vendored
+115
-12
@@ -4,13 +4,16 @@ import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
|
||||
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
|
||||
import { renderLogin } from "./views/login.js";
|
||||
import { renderDashboard } from "./views/dashboard.js";
|
||||
import { renderModels } from "./views/models.js";
|
||||
import { renderModels } from "./views/models.js?v=3";
|
||||
import { renderImport } from "./views/import.js";
|
||||
import { renderRouting } from "./views/routing.js";
|
||||
import { renderProtocols } from "./views/protocols.js";
|
||||
import { renderKeys } from "./views/keys.js";
|
||||
import { renderPlayground } from "./views/playground.js";
|
||||
import { renderSettings } from "./views/settings.js";
|
||||
import { renderLogs } from "./views/logs.js";
|
||||
import { renderBenchmark } from "./views/benchmark.js";
|
||||
import { renderOnboarding } from "./views/onboarding.js";
|
||||
|
||||
const ROUTES = [
|
||||
{ id: "dashboard", icon: "▤", view: renderDashboard },
|
||||
@@ -20,7 +23,10 @@ const ROUTES = [
|
||||
{ id: "protocols", icon: "⇆", view: renderProtocols },
|
||||
{ id: "keys", icon: "🔑", view: renderKeys },
|
||||
{ id: "playground", icon: "✦", view: renderPlayground },
|
||||
{ id: "benchmark", icon: "⏱", view: renderBenchmark },
|
||||
{ id: "logs", icon: "📜", view: renderLogs },
|
||||
{ id: "settings", icon: "⚙", view: renderSettings },
|
||||
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
|
||||
];
|
||||
|
||||
export const appState = { meta: null, health: null, version: "" };
|
||||
@@ -28,6 +34,17 @@ export const appState = { meta: null, health: null, version: "" };
|
||||
// allaigate site — where router keys are issued and topped up
|
||||
export const SITE_URL = "https://api.allaigate.com/";
|
||||
|
||||
let mobileNavOpen = false;
|
||||
|
||||
function setMobileNav(open) {
|
||||
mobileNavOpen = !!open;
|
||||
const app = document.querySelector(".app");
|
||||
if (app) app.classList.toggle("nav-open", mobileNavOpen);
|
||||
document.body.classList.toggle("mobile-nav-open", mobileNavOpen);
|
||||
const toggle = document.querySelector(".menu-toggle");
|
||||
if (toggle) toggle.setAttribute("aria-expanded", String(mobileNavOpen));
|
||||
}
|
||||
|
||||
function currentRoute() {
|
||||
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
|
||||
return ROUTES.find((r) => r.id === id) || ROUTES[0];
|
||||
@@ -39,20 +56,43 @@ const FAVICON =
|
||||
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="%233b82f6"/><stop offset=".5" stop-color="%238b5cf6"/><stop offset="1" stop-color="%2306b6d4"/></linearGradient></defs><rect width="32" height="32" rx="8" fill="url(%23g)"/><path d="M9 16h14M17 10l6 6-6 6" stroke="white" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>'
|
||||
);
|
||||
|
||||
// One-line banners under the topbar: open-admin warning, update available.
|
||||
function shellNotices() {
|
||||
const host = h("div", {});
|
||||
const m = appState.meta || {};
|
||||
if (m.admin_open_public && !sessionStorage.getItem("hide_open_admin")) {
|
||||
const close = h("button", { class: "icon-btn", title: "×", onClick: (e) => { sessionStorage.setItem("hide_open_admin", "1"); e.target.closest(".notice").remove(); } }, "×");
|
||||
host.appendChild(
|
||||
h("div", { class: "notice warn" }, "⚠ " + t("notice.openAdmin"), " ",
|
||||
h("a", { href: "#/settings" }, t("nav.settings") + " →"), close)
|
||||
);
|
||||
}
|
||||
if (m.update_available) {
|
||||
host.appendChild(
|
||||
h("div", { class: "notice info" },
|
||||
"⬆ " + t("notice.update", { v: m.update_available }),
|
||||
" ", h("code", { class: "mono" }, "cargo install cortiq-gateway"))
|
||||
);
|
||||
}
|
||||
return host;
|
||||
}
|
||||
|
||||
function sidebar(activeId) {
|
||||
const visibleRoutes = ROUTES.filter((r) => !r.hidden);
|
||||
return h(
|
||||
"aside",
|
||||
{ class: "sidebar", id: "sidebar" },
|
||||
{ class: "sidebar", id: "sidebar", "aria-label": "Main navigation" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "brand" },
|
||||
h("img", { src: FAVICON, alt: "" }),
|
||||
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
|
||||
h("div", { class: "brand-copy" }, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub"))),
|
||||
h("button", { class: "icon-btn nav-close", type: "button", title: "Close menu", "aria-label": "Close menu", onClick: () => setMobileNav(false) }, "×")
|
||||
),
|
||||
...ROUTES.map((r) =>
|
||||
...visibleRoutes.map((r) =>
|
||||
h(
|
||||
"a",
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id, onClick: () => setMobileNav(false) },
|
||||
h("span", { class: "ico" }, r.icon),
|
||||
t("nav." + r.id)
|
||||
)
|
||||
@@ -85,7 +125,7 @@ function statusPills() {
|
||||
: t("status.checking");
|
||||
return h(
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
{ class: "flex status-pills" },
|
||||
h(
|
||||
"span",
|
||||
{ class: "pill" },
|
||||
@@ -109,7 +149,7 @@ function langSelect() {
|
||||
title: "Language",
|
||||
onChange: (e) => setLang(e.target.value),
|
||||
},
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, l.label))
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, `${l.flag} ${l.label}`))
|
||||
);
|
||||
return sel;
|
||||
}
|
||||
@@ -136,6 +176,19 @@ function topbar(activeId) {
|
||||
return h(
|
||||
"header",
|
||||
{ class: "topbar" },
|
||||
h(
|
||||
"button",
|
||||
{
|
||||
class: "icon-btn menu-toggle",
|
||||
type: "button",
|
||||
title: "Menu",
|
||||
"aria-label": "Open menu",
|
||||
"aria-controls": "sidebar",
|
||||
"aria-expanded": String(mobileNavOpen),
|
||||
onClick: () => setMobileNav(!mobileNavOpen),
|
||||
},
|
||||
h("span", { class: "hamburger", "aria-hidden": "true" }, h("i"), h("i"), h("i"))
|
||||
),
|
||||
h("div", { class: "title" }, t("nav." + activeId)),
|
||||
h(
|
||||
"div",
|
||||
@@ -175,13 +228,25 @@ async function renderView() {
|
||||
|
||||
function renderShell() {
|
||||
const route = currentRoute();
|
||||
mobileNavOpen = false;
|
||||
document.body.classList.remove("mobile-nav-open");
|
||||
// onboarding — full screen without sidebar/topbar
|
||||
if (route.id === "onboarding") {
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h("div", { class: "onb-shell" }, h("div", { class: "onb-content", id: "view" }))
|
||||
);
|
||||
renderView();
|
||||
return;
|
||||
}
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "app" },
|
||||
sidebar(route.id),
|
||||
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
|
||||
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
|
||||
h("main", { class: "main" }, topbar(route.id), shellNotices(), h("div", { class: "content", id: "view" }))
|
||||
)
|
||||
);
|
||||
renderView();
|
||||
@@ -191,12 +256,30 @@ function showLogin() {
|
||||
mount(document.getElementById("app"), renderLogin(boot));
|
||||
}
|
||||
|
||||
let staleNotified = false;
|
||||
async function checkUiFresh() {
|
||||
// the SPA lives in the tab until reload; if the gateway restarted (dev
|
||||
// rebuild, update, config restart) the embedded UI may have changed —
|
||||
// tell the user instead of letting them stare at stale views
|
||||
try {
|
||||
const m = await api.meta();
|
||||
if (appState.meta?.started && m.started && m.started !== appState.meta.started && !staleNotified) {
|
||||
staleNotified = true;
|
||||
const bar = h("div", { class: "callout warn", style: "position:fixed; left:50%; transform:translateX(-50%); bottom:18px; z-index:99; box-shadow:0 8px 24px rgba(0,0,0,.35)" },
|
||||
h("span", {}, "🔄 " + t("shell.uiStale")), " ",
|
||||
h("button", { class: "btn sm primary", onClick: () => location.reload() }, t("shell.reload")));
|
||||
document.body.append(bar);
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
|
||||
async function refreshHealth() {
|
||||
try {
|
||||
appState.health = await api.health();
|
||||
} catch {
|
||||
appState.health = null;
|
||||
}
|
||||
checkUiFresh();
|
||||
// re-render topbar only if shell present
|
||||
const route = currentRoute();
|
||||
const main = document.querySelector(".main");
|
||||
@@ -213,20 +296,34 @@ export async function boot() {
|
||||
setToken(urlTok);
|
||||
history.replaceState({}, "", location.pathname + location.hash);
|
||||
}
|
||||
if (!getToken()) {
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// try without token first — open mode (no CORTIQ_ADMIN_TOKEN) succeeds,
|
||||
// protected mode returns 401 and we show login
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch (e) {
|
||||
if (String(e.message) === "unauthorized") {
|
||||
// if we have no token stored, show login; with a stale token also show login
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// gateway reachable but error — still show shell
|
||||
// if meta failed for other reason but token was present, fall through
|
||||
if (!getToken()) {
|
||||
// open mode but gateway unreachable — still try shell; health will show
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch {}
|
||||
}
|
||||
}
|
||||
// first-run check — redirect to onboarding if needed (unless already there)
|
||||
try {
|
||||
const st = await api.setupStatus();
|
||||
if (st.needs_setup && location.hash !== "#/onboarding") {
|
||||
location.hash = "#/onboarding";
|
||||
}
|
||||
} catch {}
|
||||
if (!location.hash) location.hash = "#/dashboard";
|
||||
renderShell();
|
||||
refreshHealth();
|
||||
@@ -243,6 +340,12 @@ onLangChange(() => {
|
||||
window.addEventListener("hashchange", () => {
|
||||
if (document.querySelector(".app")) renderShell();
|
||||
});
|
||||
window.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Escape" && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
window.addEventListener("resize", () => {
|
||||
if (window.innerWidth > 980 && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
setInterval(() => {
|
||||
if (document.querySelector(".app")) refreshHealth();
|
||||
}, 20000);
|
||||
|
||||
Vendored
+12
-7
@@ -10,13 +10,13 @@ import tr from "./i18n/tr.js";
|
||||
|
||||
const DICTS = { en, ru, de, fr, es, zh, tr };
|
||||
export const LANGS = [
|
||||
{ code: "en", label: "English" },
|
||||
{ code: "ru", label: "Русский" },
|
||||
{ code: "de", label: "Deutsch" },
|
||||
{ code: "fr", label: "Français" },
|
||||
{ code: "es", label: "Español" },
|
||||
{ code: "zh", label: "中文" },
|
||||
{ code: "tr", label: "Türkçe" },
|
||||
{ code: "en", label: "English", flag: "🇬🇧" },
|
||||
{ code: "ru", label: "Русский", flag: "🇷🇺" },
|
||||
{ code: "de", label: "Deutsch", flag: "🇩🇪" },
|
||||
{ code: "fr", label: "Français", flag: "🇫🇷" },
|
||||
{ code: "es", label: "Español", flag: "🇪🇸" },
|
||||
{ code: "zh", label: "中文", flag: "🇨🇳" },
|
||||
{ code: "tr", label: "Türkçe", flag: "🇹🇷" },
|
||||
];
|
||||
|
||||
const listeners = new Set();
|
||||
@@ -48,6 +48,11 @@ export function onLangChange(fn) {
|
||||
return () => listeners.delete(fn);
|
||||
}
|
||||
|
||||
// Human name for a complexity tier; custom tier names pass through as-is.
|
||||
export function trTier(tier) {
|
||||
return ["low", "medium", "high"].includes(tier) ? t("tier." + tier) : tier;
|
||||
}
|
||||
|
||||
export function t(key, vars) {
|
||||
const d = DICTS[lang] || en;
|
||||
let s = d[key] ?? en[key] ?? key;
|
||||
|
||||
Vendored
+373
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Routing",
|
||||
"nav.protocols": "Protokolle",
|
||||
"nav.keys": "API-Schlüssel",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Testbereich",
|
||||
"nav.settings": "Einstellungen",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Router-Key & Abrechnung",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "nach Aufgabe",
|
||||
"dash.groupby.account": "nach Account",
|
||||
"dash.recent.title": "Letzte Anfragen",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Playground ausprobieren.",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Testbereich ausprobieren.",
|
||||
"dash.col.time": "Zeit",
|
||||
"dash.col.model": "Modell",
|
||||
"dash.col.tier": "Tier",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "alle",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Testbereich",
|
||||
"pg.subtitle": "Prompt durch die Live-Pipeline senden und Routing-Entscheidung prüfen.",
|
||||
"pg.system": "System-Prompt",
|
||||
"pg.systemPlaceholder": "optionale Systemnachricht",
|
||||
@@ -280,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "Eintrags-TTL",
|
||||
"settings.cache.embed": "Embedding-Modell",
|
||||
"settings.cache.note": "Cache-Einstellungen werden nach einem Neustart wirksam.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Modell importieren",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Logs leeren",
|
||||
"dash.clearConfirm": "Alle erfassten Statistiken und Logs löschen? Das lässt sich nicht rückgängig machen.",
|
||||
"dash.cleared": "Logs geleert",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Gib eine ID oder einen Modellnamen ein",
|
||||
"models.form.apiKey": "API-Schlüssel",
|
||||
"models.form.apiKeyPlaceholder": "API-Schlüssel einfügen",
|
||||
"models.form.apiKeyNotNeeded": "nicht erforderlich (lokaler Server)",
|
||||
"models.form.apiKeyHint": "Wird sicher im Gateway gespeichert und nicht erneut angezeigt.",
|
||||
"models.form.keyStored": "Ein Schlüssel ist gespeichert — leer lassen, um ihn zu behalten, oder einen neuen einfügen.",
|
||||
"models.form.keyFromEnv": "Es wird ein Schlüssel aus einer Umgebungsvariable verwendet.",
|
||||
"models.form.paramsTitle": "Generierungs- und Denkparameter (O(1), MTP, Temperatur)",
|
||||
"models.form.temperature": "Temperatur",
|
||||
"models.form.temperatureHint": "Generierungstemperatur (0.0 = deterministisch, 1.0+ = kreativ).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling-Schwelle (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max. Tokens",
|
||||
"models.form.maxTokensHint": "Maximale Anzahl generierter Tokens.",
|
||||
"models.form.thinkBudget": "Denkmodus / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Token-Budget fürs Nachdenken (0 = aus, N > 0 = max. Reasoning-Tokens).",
|
||||
"models.form.systemPrompt": "Standard-System-Prompt",
|
||||
"models.form.systemPromptHint": "Wird automatisch vorangestellt, wenn die Anfrage keinen System-Prompt enthält.",
|
||||
"models.form.systemPromptPlaceholder": "Du bist ein hilfreicher Assistent…",
|
||||
"models.form.testKey": "Testen & Modelle laden",
|
||||
"models.form.testing": "wird geprüft…",
|
||||
"models.form.foundModels": "{n} Modelle geladen",
|
||||
"models.form.modelPlaceholder": "tippen zum Suchen…",
|
||||
"models.form.modelHint": "Klicke „Testen & Modelle laden“, um die Liste mit deinem Schlüssel abzurufen, und wähle dann eines aus.",
|
||||
"models.managed": "verwaltet",
|
||||
"models.managedHint": "Wird vom verwalteten CMF-Server bereitgestellt (Einstellungen → Lokale Modelle). Dort konfigurieren.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Router verwenden (aus = nur lokale Modelle)",
|
||||
"settings.cmf": "Lokale Modelle (CMF)",
|
||||
"settings.cmf.localOnly": "Nur lokale Modelle verwenden",
|
||||
"settings.cmf.manage": "Lokalen Modellserver starten",
|
||||
"settings.cmf.autoInstall": "cortiq automatisch installieren (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq automatisch aktualisieren",
|
||||
"settings.cmf.model": "Pfad zum lokalen .cmf-Modell",
|
||||
"settings.cmf.port": "Port des lokalen Servers",
|
||||
"settings.cmf.modelId": "Modell-ID im Pool",
|
||||
"settings.cmf.note": "Stellt ein lokales .cmf über cortiq bereit (github.com/infosave2007/cmf). Funktioniert auch, wenn der Router nicht erreichbar ist.",
|
||||
"settings.cmf.threads": "Worker-Threads",
|
||||
"settings.cmf.gpu": "GPU verwenden (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Threads parallelisieren die Decode-Matvecs (~3× schneller; 0 = Standard der Runtime). GPU-Offload hilft vor allem beim Prefill langer Prompts — das Decoding ist speicherbandbreitenbegrenzt und daher auf Apple Silicon ungefähr neutral.",
|
||||
"settings.cmf.pick": "Modell auswählen",
|
||||
"settings.cmf.models": "Lokale Modelle",
|
||||
"settings.cmf.modelsHelp": "Jede Zeile ist ein Modell, das das Gateway ausführt (eigenes cortiq serve auf eigenem Port) und im Pool registriert. Füge so viele hinzu, wie du .cmf-Dateien und freie Ports hast.",
|
||||
"settings.cmf.rowHint": "ID · .cmf-Datei · Port · Threads · GPU. Änderungen greifen nach einem Neustart; das Löschen einer Zeile stoppt diesen Server.",
|
||||
"settings.cmf.addServer": "+ Lokales Modell hinzufügen",
|
||||
"settings.cmf.checkPort": "Port prüfen",
|
||||
"settings.cmf.portFree": "Port ist frei",
|
||||
"settings.cmf.portBusy": "Port ist belegt",
|
||||
"settings.cmf.useSuggested": "{port} verwenden",
|
||||
"settings.cmf.serving": "{model} läuft auf :{port}",
|
||||
"settings.cmf.intro": "Modelle lokal mit der cortiq-Engine ausführen — funktioniert auch offline oder wenn der Router ausfällt.",
|
||||
"settings.cmf.runtime": "cortiq-Runtime",
|
||||
"settings.cmf.runtimeHelp": "Das Kommandozeilenwerkzeug `cortiq`, das .cmf-Modelle lädt und ausführt. Einmal installieren, bevor du einen lokalen Server startest.",
|
||||
"settings.cmf.install": "cortiq installieren",
|
||||
"settings.cmf.reinstall": "Neu installieren / aktualisieren",
|
||||
"settings.cmf.installing": "wird installiert… (kompiliert, einige Minuten)",
|
||||
"settings.cmf.installedOk": "cortiq installiert",
|
||||
"settings.cmf.notInstalled": "nicht installiert",
|
||||
"settings.cmf.addModel": "Modell hinzufügen",
|
||||
"settings.cmf.addModelHelp": "Modelle werden im Tab „Modell importieren“ heruntergeladen und konvertiert (HuggingFace → .cmf). Richte den lokalen Server danach unten auf die .cmf-Datei aus.",
|
||||
"settings.cmf.importNote": "Noch kein Modell? Importiere eines:",
|
||||
"settings.cmf.importLink": "Import öffnen",
|
||||
"settings.cmf.modelHelp": "Pfad zur .cmf-Datei, die bereitgestellt werden soll (z. B. models/qwen.cmf), erzeugt im Import-Tab. Mehrere Modelle für verschiedene Aufgaben? Importiere sie und wechsle diesen Pfad.",
|
||||
"settings.cmf.portHelp": "TCP-Port, auf dem der lokale Modellserver lauscht (Standard 8081). Nur ändern, wenn er mit einem anderen Dienst kollidiert.",
|
||||
"settings.cmf.threadsHelp": "Wie viele CPU-Kerne für die Generierung genutzt werden. Mehr = schneller (~3× gegenüber einem Kern); 0 überlässt die Wahl der Runtime.",
|
||||
"settings.cmf.gpuHelp": "Matrixberechnungen auf die GPU auslagern (Metal/wgpu). Auf Apple Silicon beschleunigt das vor allem lange Prompts — das Decoding ist speicherbandbreitenbegrenzt.",
|
||||
"settings.cmf.localOnlyHelp": "Jede Anfrage an das lokale Modell leiten und dabei Router sowie Cloud-Anbieter ignorieren.",
|
||||
"settings.cmf.manageHelp": "Das Gateway startet beim Hochfahren `cortiq serve` für die untenstehende .cmf-Datei und routet dorthin. Erfordert ein installiertes cortiq (Schritt 1). Greift nach einem Neustart.",
|
||||
"settings.cmf.autoInstallHelp": "Wenn cortiq beim Start fehlt, automatisch von crates.io installieren.",
|
||||
"settings.cmf.autoUpdateHelp": "Beim Start cortiq auf die neueste Version aktualisieren, falls eine neuere existiert.",
|
||||
"settings.router.enabledHelp": "Den cortiq-Router jede Anfrage klassifizieren lassen (Aufgabentyp + Komplexität) und ein Modell wählen. Aus = nur die lokale Routing-Tabelle verwenden.",
|
||||
"settings.router.verifyTlsHelp": "Das TLS-Zertifikat des Routers prüfen. Nur bei einem selbstsignierten oder IP-gebundenen Zertifikat abschalten.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus-Metriken unter GET /metrics bereitstellen.",
|
||||
"settings.cortiq.echoHelp": "Ein Feld `cortiq` (Aufgabe, Modell, Kosten) zu jedem Antwort-Body hinzufügen. Response-Header werden immer gesendet.",
|
||||
"settings.stats.enabledHelp": "Aktuelle Anfragen aufzeichnen, um Dashboard und Diagramme zu speisen.",
|
||||
"settings.cache.enabledHelp": "Semantischer Cache: eine gespeicherte Antwort zurückgeben, wenn ein neuer Prompt einem früheren sehr ähnlich ist, und den Modellaufruf überspringen. Erfordert ein Embedding-Modell im Pool.",
|
||||
"settings.log.bodiesHelp": "Vollständige Nachrichteninhalte von Anfrage und Antwort loggen. Aus Datenschutzgründen standardmäßig aus.",
|
||||
"settings.router.saveKey": "Schlüssel speichern",
|
||||
"settings.router.keySaved": "Router-Schlüssel gespeichert",
|
||||
"settings.router.keyEmpty": "Füge zuerst den Schlüssel ein",
|
||||
"settings.router.secretHelp": "Füge den Schlüssel aus deiner E-Mail von api.allaigate.com ein und klicke auf „Schlüssel speichern“. Das ist KEIN Wert aus „API-Schlüssel“ — die sind für Clients, die dein Gateway aufrufen.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — Schlüssel gespeichert (zum Ersetzen erneut einfügen)",
|
||||
"settings.route.strategyHelp": "Welcher Text zur Klassifizierung an den Router geht: last_user = nur die letzte Nutzernachricht; last_user_plus_system = letzte Nutzernachricht + System-Prompt; concat_all = die gesamte Unterhaltung.",
|
||||
"settings.route.maxCharsHelp": "Maximale Zeichenzahl dieses Textes, die an den Router gesendet wird (längerer Text wird gekürzt). Mehr = genauere Klassifizierung, aber langsamere und teurere Router-Aufrufe.",
|
||||
"settings.route.cacheTtlHelp": "Wie lange eine Routing-Entscheidung für eine identische Anfrage zwischengespeichert wird, bevor der Router erneut gefragt wird (z. B. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Standard-Routing-Profil, wenn eine Anfrage keines angibt: cost-saver (am günstigsten), balanced oder quality-first (bestes Modell). Beeinflusst, welches Modell der Router wählt.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Suche auf HuggingFace, konvertiere in ein lokales .cmf und stelle es bereit.",
|
||||
"import.searchPlaceholder": "HuggingFace-Modelle suchen (z. B. qwen3, llama)…",
|
||||
"import.searching": "Suche läuft…",
|
||||
"import.noResults": "Keine Modelle gefunden.",
|
||||
"import.configure": "Konvertierung einrichten",
|
||||
"import.outName": "Ausgabename",
|
||||
"import.outNameHint": "Name der .cmf-Datei (Buchstaben, Ziffern, - und _).",
|
||||
"import.quant": "Quantisierung",
|
||||
"import.convert": "Konvertieren & laden",
|
||||
"import.converting": "Konvertierung",
|
||||
"import.started": "Konvertierung gestartet",
|
||||
"import.failed": "Konvertierung fehlgeschlagen — siehe Log.",
|
||||
"import.register": "Als lokales Modell registrieren",
|
||||
"import.registered": "Als '{id}' registriert. Starte cortiq-server, um es bereitzustellen.",
|
||||
"import.jobs": "Downloads & Konvertierungen",
|
||||
"import.cancel": "Abbrechen",
|
||||
"import.cancelling": "Wird abgebrochen — unvollständige Ausgabe wird entfernt…",
|
||||
"import.delete": "Löschen",
|
||||
"import.deleteConfirm": "Dieses Modell und seine konvertierte .cmf-Datei von der Festplatte löschen?",
|
||||
"import.deleted": "Mitsamt Dateien gelöscht",
|
||||
"import.state.running": "läuft",
|
||||
"import.state.done": "fertig",
|
||||
"import.state.error": "Fehler",
|
||||
"import.state.cancelled": "abgebrochen",
|
||||
"import.featuredTitle": "Empfohlene fertige .cmf-Modelle",
|
||||
"import.featuredSub": "Vorkonvertierte .cmf-Modelle vom infosave-Konto auf HuggingFace — die Liste wird live gelesen, neue Modelle erscheinen von selbst.",
|
||||
"import.readyCmfBadge": "Fertiges .cmf",
|
||||
"import.directDownload": "Herunterladen",
|
||||
"import.q.q8_2f": "8-Bit Zwei-Feld (𝒲×θ) — empfohlen: bestes Verhältnis von Qualität und Größe.",
|
||||
"import.q.q8_row": "8-Bit pro Zeile — einfach und robust.",
|
||||
"import.q.q4": "4-Bit Block — am kleinsten, geringere Qualität.",
|
||||
"import.q.q4_tiled": "4-Bit gekachelt (Q4_TILED) — 1,66× schneller auf der Metal-GPU.",
|
||||
"import.q.vbit": "Variabel 3–8 Bit — Bit-Budget pro Experte.",
|
||||
"import.q.q1t": "1,58-Bit ternär (Q1T) — BitNet b1.58 mit spärlichen Ausreißern.",
|
||||
"import.q.q1s": "1-Bit spärlich (Q1S) — 1-Bit-Basis mit holografischen Ausreißern.",
|
||||
"import.q.q1p": "1-Bit PTQ (Q1P) — 1 Bit mit Fehlerdiffusion.",
|
||||
"import.q.q1": "1-Bit (Q1) — rein binäres Format.",
|
||||
"import.q.f16": "16-Bit — keine Quantisierung.",
|
||||
"import.q.f32": "32-Bit — volle Präzision (große Datei).",
|
||||
"import.adv.title": "Erweiterte Einstellungen",
|
||||
"import.adv.auto": "automatisch",
|
||||
"import.adv.linearCore": "Linearer Kern",
|
||||
"import.adv.linearHint": "Wie Layer mit linearer Attention übernommen bzw. gefaltet werden.",
|
||||
"import.adv.nphase": "vmf_phase Phasen/Kopf",
|
||||
"import.adv.vbitShape": "VBIT-Budgetform",
|
||||
"import.adv.meanBits": "VBIT mittlere Bits",
|
||||
"import.adv.shard": "Shard max. (GB)",
|
||||
"import.adv.noShard": "kein Sharding",
|
||||
"import.adv.skipMtp": "MTP-Kopf überspringen",
|
||||
"import.adv.o1": "O(1)-Attention",
|
||||
"import.adv.o1Hint": "Streaming-Attention mit konstantem Speicher: der KV-Cache wird nach dem Prefill freigegeben, lange Kontexte lassen den Speicher also nicht wachsen. Die Gewichte bleiben Byte-identisch — gespeichert wird nur ein Hinweis für die Ladezeit; du kannst jederzeit zurückwechseln (mit --o1 off starten). Auf Qwen3-0.6B validiert: alle Layer ≈ ×1,18 Perplexität.",
|
||||
"import.adv.o1Off": "aus — exakte Attention (Standard)",
|
||||
"import.adv.o1All": "alle — jeden Softmax-Layer konvertieren",
|
||||
"import.adv.o1Deep": "deep N — nur die N tiefsten Layer",
|
||||
"import.adv.o1Custom": "benutzerdefiniert — explizite Layer-Liste",
|
||||
"import.adv.o1DeepN": "N tiefe Layer",
|
||||
"import.adv.o1Layers": "Layer (kommagetrennt)",
|
||||
"import.adv.o1mHint": "Landmark-Budget (Standard 32)",
|
||||
"import.adv.o1windowHint": "Breite des exakten Fensters (Standard 128)",
|
||||
"import.adv.o1sinkHint": "permanente exakte Sink-Keys (Standard 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Ersteinrichtung",
|
||||
"onboarding.welcome": "Willkommen bei Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Wähle deine Sprache — du kannst sie jederzeit in der Kopfzeile ändern.",
|
||||
"onboarding.next": "Weiter",
|
||||
"onboarding.back": "Zurück",
|
||||
"onboarding.skip": "Vorerst überspringen",
|
||||
"onboarding.skipSetup": "Überspringen",
|
||||
"onboarding.finish": "Einrichtung abschließen",
|
||||
"onboarding.featuredTitle": "Fertige Modelle (ein Klick)",
|
||||
"onboarding.featuredSub": "Das sind bereits konvertierte .cmf-Modelle von infosave auf HuggingFace — einfach herunterladen, keine Konvertierung nötig. Wähle eines oder mehrere für den Sofortstart.",
|
||||
"onboarding.download": "Herunterladen",
|
||||
"onboarding.hfTitle": "Jedes HuggingFace-Modell — automatisch konvertiert",
|
||||
"onboarding.hfSub": "Suche ein beliebiges Modell, wähle eine Quantisierung, und wir konvertieren es automatisch nach .cmf. Keine manuellen Config-Änderungen.",
|
||||
"onboarding.noJobs": "Noch keine Downloads.",
|
||||
"onboarding.doneTitle": "Alles bereit",
|
||||
"onboarding.doneSub": "Heruntergeladene Modelle werden automatisch registriert. Weitere kannst du unter Import oder Modelle hinzufügen.",
|
||||
"onboarding.doneHint": "„Einrichtung abschließen“ verbindet jeden fertigen Download als lokales Modell, startet dessen .cmf-Server und richtet das Routing darauf aus. Alles lässt sich später unter Modelle ändern.",
|
||||
"onboarding.willRegister": "Werden als lokale Modelle verbunden",
|
||||
"onboarding.nothingToRegister": "Noch keine abgeschlossenen Downloads — nichts zu verbinden. Geh einen Schritt zurück und wähle ein Modell.",
|
||||
"onboarding.registeredN": "{n} Modell(e) verbunden: {ids}",
|
||||
"onboarding.runtimeMissing": "Die cortiq-Runtime ist nicht installiert — installiere sie unter Einstellungen → Lokale Modelle, um diese Modelle bereitzustellen.",
|
||||
"onboarding.downloaded": "heruntergeladen",
|
||||
"onboarding.update": "Aktualisieren",
|
||||
"onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen",
|
||||
"onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.",
|
||||
"onboarding.unregistered": "Ebenfalls abgemeldet: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Bild",
|
||||
"pg.tab.video": "Video",
|
||||
"pg.generate": "Generieren",
|
||||
"pg.generating": "Wird generiert…",
|
||||
"pg.genFailed": "Generierung fehlgeschlagen — siehe Log.",
|
||||
"pg.result": "Ergebnis",
|
||||
"pg.genMetrics": "Generierungsmetriken",
|
||||
"pg.imagePromptPlaceholder": "ein Rotfuchs im Schnee bei Sonnenuntergang, fotorealistisch, detailliertes Fell",
|
||||
"pg.videoPromptPlaceholder": "ein Corgi mit Kochmütze wirft einen Pfannkuchen, Brutzeln und ein fröhliches Bellen",
|
||||
"pg.imageEmpty": "Das generierte Bild erscheint hier.",
|
||||
"pg.videoEmpty": "Der generierte Clip (mit Ton) erscheint hier.",
|
||||
"pg.width": "Breite",
|
||||
"pg.height": "Höhe",
|
||||
"pg.steps": "Schritte",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Zufälliger Seed",
|
||||
"pg.cfgHint": "Guidance-Stärke; ≤ 1 schaltet CFG ab und halbiert die Arbeit.",
|
||||
"pg.frames": "Frames",
|
||||
"pg.framesHint": "Bei 24 fps; auf das Raster des Modells gerundet.",
|
||||
"pg.quality": "JPEG-Qualität",
|
||||
"pg.videoSizeHint": "Vielfache von 32; die trainierte kurze Kante ist 768.",
|
||||
"pg.videoStepsHint": "Die Turbo-LoRA ist auf 4 Schritte trainiert.",
|
||||
"pg.firstFrame": "Erstes Frame (optional)",
|
||||
"pg.lastFrame": "Letztes Frame (optional)",
|
||||
"pg.keyframeHint": "Macht den Lauf zu Keyframe-to-Video.",
|
||||
"pg.resolution": "Auflösung",
|
||||
"pg.genTime": "Generierungszeit",
|
||||
"pg.timePerStep": "Pro Schritt",
|
||||
"pg.timePerFrame": "Pro Frame",
|
||||
"pg.fileSize": "Dateigröße",
|
||||
"pg.duration": "Dauer",
|
||||
"pg.play": "Abspielen",
|
||||
"pg.noMediaModels": "Noch kein lokales {kind}-Modell. Lade eines (z. B. aus der infosave-Sammlung) unter Import herunter:",
|
||||
"pg.tokSpeed": "Geschwindigkeit",
|
||||
"pg.ttft": "Erstes Token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Logs",
|
||||
"logs.subtitle": "Jüngstes Gateway-Log — das, was das Terminal zeigen würde.",
|
||||
"logs.filterPlaceholder": "filtern…",
|
||||
"logs.pause": "Pause",
|
||||
"logs.resume": "Fortsetzen",
|
||||
"logs.empty": "Nichts gefunden.",
|
||||
"notice.openAdmin": "Die Konsole ist ohne Token offen und von anderen Rechnern erreichbar. Setze einen Token, wenn diese Maschine nicht privat ist.",
|
||||
"notice.update": "Update verfügbar: v{v}.",
|
||||
"dash.sys.title": "System",
|
||||
"dash.sys.cores": "Kerne",
|
||||
"dash.sys.disk": "Festplatte (Modelle)",
|
||||
"dash.sys.free": "frei",
|
||||
"dash.sys.gateway": "Gateway",
|
||||
"fit.disk": "zu wenig Speicherplatz",
|
||||
"fit.diskTitle": "Der Download ist größer als der freie Platz auf der Modell-Festplatte.",
|
||||
"fit.ram": "passt evtl. nicht in den RAM",
|
||||
"fit.ramTitle": "Das Modell ist nahe an oder größer als der RAM dieser Maschine — es läuft womöglich nicht.",
|
||||
"pg.eta": "übrig",
|
||||
"pg.min": "Min",
|
||||
"pg.queued": "In der Warteschlange — Position {n}. Startet, sobald die aktuelle Generierung fertig ist.",
|
||||
"pg.gallery": "Letzte Generierungen",
|
||||
"pg.galDeleteConfirm": "Diese Generierung samt Dateien löschen?",
|
||||
"pg.clearChat": "Leeren",
|
||||
"keys.snip.title": "Deine App verbinden",
|
||||
"keys.snip.hint": "Füge eines der Beispiele in dein Projekt ein — nur der Schlüssel-Platzhalter muss ersetzt werden.",
|
||||
"keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
|
||||
"keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
|
||||
"keys.snip.keyPh": "dein API-Schlüssel",
|
||||
"settings.restart": "Alles neu starten",
|
||||
"settings.restartConfirm": "Gateway und alle lokalen Modellserver neu starten? Laufende Anfragen werden abgebrochen.",
|
||||
"settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
|
||||
"settings.shutdown": "Alles stoppen & ausschalten",
|
||||
"settings.shutdownConfirm": "Alle lokalen Modellserver stoppen und das Gateway ausschalten? API und dieses Panel antworten erst wieder, wenn Sie das Gateway neu starten.",
|
||||
"settings.shutdownDone": "Das Gateway ist ausgeschaltet.",
|
||||
"settings.shutdownHint": "Alle lokalen Modellserver sind gestoppt, der Prozess wurde beendet. Starten Sie das Gateway auf dem Server neu, um das Panel zurückzubekommen.",
|
||||
"bench.title": "Geschwindigkeit lokaler Modelle",
|
||||
"bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
|
||||
"bench.run": "Messen",
|
||||
"bench.running": "misst…",
|
||||
"bench.failed": "fehlgeschlagen",
|
||||
"bench.none": "Noch keine lokalen .cmf-Chat-Modelle.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Smalltalk",
|
||||
"task.code": "Code",
|
||||
"task.creative-writing": "Kreatives Schreiben",
|
||||
"task.extraction": "Datenextraktion",
|
||||
"task.math": "Mathematik",
|
||||
"task.qa": "Fragen & Antworten",
|
||||
"task.summarization": "Zusammenfassung",
|
||||
"task.translation": "Übersetzung",
|
||||
"routing.tasks.title": "Aufgabenbasiertes Routing",
|
||||
"routing.tasks.hint": "Die Modelle einer Regel kommen für ihre Aufgabe zuerst; die Komplexitätskette bleibt als Fallback. Leer = nur nach Komplexität.",
|
||||
"routing.tasks.inherit": "nach Komplexität (keine Regel)",
|
||||
"routing.tasks.share": "{p}% der Anfragen (7 T)",
|
||||
"routing.tasks.fromRouter": "Labels live vom Router",
|
||||
"routing.tasks.builtin": "Router nicht erreichbar — eingebauter Labelsatz",
|
||||
"routing.tasks.addCustom": "Aufgabe hinzufügen",
|
||||
"routing.tasks.addPlaceholder": "eigenes Label…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Qualitäts- und Tempo-Proben durch die Live-Pipeline: Antworten per Schlüsselwort geprüft, Schleifen erkannt, tok/s und Zeit bis zum ersten Token je Modell.",
|
||||
"bm.models": "Zu testende Modelle",
|
||||
"bm.quickMode": "Schnelltest",
|
||||
"bm.quickHint": "Die schnelle Teilmenge mit kurzen Antworten — etwa eine Minute pro lokalem Modell.",
|
||||
"bm.suiteInfo": "Eingebaute Suite: {q} schnelle / {f} volle Tests, plus Ihre eigenen.",
|
||||
"bm.run": "Starten",
|
||||
"bm.pickModel": "Wählen Sie mindestens ein Modell.",
|
||||
"bm.testOf": "Test {i} von {n}",
|
||||
"bm.results": "Ergebnisse",
|
||||
"bm.quick": "schnell",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Klick auf eine Zelle zeigt die vollständige Antwort.",
|
||||
"bm.history": "Frühere Läufe (in diesem Browser gespeichert)",
|
||||
"bm.custom": "Eigene Tests",
|
||||
"bm.customHint": "Prompt + kommagetrennte Schlüsselwörter, die die Antwort enthalten muss (| für Synonyme). Im Browser gespeichert.",
|
||||
"bm.promptPh": "Prompt…",
|
||||
"bm.checkPh": "Schlüsselwörter, kommagetrennt",
|
||||
"bm.addTest": "Test hinzufügen",
|
||||
"bm.emptyAnswer": "leere Antwort",
|
||||
"bm.loopDetected": "Schleife erkannt",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Smartes Routing",
|
||||
"bm.mode.models": "Einzelne Modelle",
|
||||
"bm.mode.routingDesc": "Das Kernfeature des Produkts im Test: jede Probe läuft über cortiq-auto — Sie sehen, welche Aufgabe der Router erkannt hat, welches Modell er wählte, ob das den Erwartungen entspricht und was es gekostet hat.",
|
||||
"bm.mode.modelsDesc": "Dieselbe Suite, an gewählte Modelle gepinnt — Qualität, Tempo und Kosten im direkten Vergleich, am Router vorbei.",
|
||||
"bm.answersOk": "Antworten korrekt",
|
||||
"bm.classAcc": "Aufgaben-Klassifikation",
|
||||
"bm.modelDist": "Anfragen gingen an",
|
||||
"bm.expectedLbl": "erwartet:",
|
||||
"bm.answer": "Antwort",
|
||||
"bm.liveRunning": "läuft…",
|
||||
"bm.spent": "ausgegeben",
|
||||
"bm.verdict": "🏁 Fertig: Antworten {p}, ausgegeben {s}",
|
||||
"bm.clearHistory": "Alle löschen",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — ins Terminal einfügen (das Gateway spricht das Anthropic-Protokoll nativ):",
|
||||
"keys.h.codex": "Codex — hängt den Provider an ~/.codex/config.toml an und startet:",
|
||||
"keys.h.opencode": "OpenCode — schreibt ~/.config/opencode/opencode.json (überschreibt!) und startet:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — hängt das Modell an ~/.continue/config.yaml an:",
|
||||
"keys.h.cline1": "Cline (VS Code) — in den Erweiterungseinstellungen wählen:",
|
||||
"keys.h.cline2": "Einstellungen → API Provider → OpenAI Compatible, dann die Werte oben einfügen.",
|
||||
"keys.h.note": "cortiq-auto = smartes Routing: der Router wählt das Modell pro Anfrage. Für ein festes Modell den exakten Namen angeben. Unter Windows die Skripte in Git Bash / WSL ausführen oder dieselben Variablen über die Systemumgebung setzen.",
|
||||
"keys.h.protoOff": "Das Anthropic-Protokoll (/v1/messages) ist deaktiviert — Claude Code kann sich nicht verbinden.",
|
||||
"keys.h.enableProto": "Aktivieren",
|
||||
"keys.h.st.installed": "Auf der Gateway-Maschine installiert",
|
||||
"keys.h.st.configured": "Bereits für dieses Gateway konfiguriert",
|
||||
"keys.h.st.notFound": "Auf der Gateway-Maschine nicht gefunden",
|
||||
"bm.degraded": "Router degradiert",
|
||||
"bm.routerDegraded": "Der Router hat nicht rechtzeitig geantwortet — die Anfrage nutzte die Fallback-Kette; die Klassifikation wird nicht gewertet.",
|
||||
"bm.cutByCap": "Budget ging fürs Denken drauf — keine Antwort mehr",
|
||||
"shell.uiStale": "Das Gateway wurde aktualisiert — diese Seite zeigt eine ältere Oberfläche.",
|
||||
"shell.reload": "Neu laden",
|
||||
"keys.col.requests": "Anfragen",
|
||||
"keys.col.tokens": "Tokens ein/aus",
|
||||
"keys.col.costAll": "$ gesamt",
|
||||
"routing.tasks.tier": "Komplexität",
|
||||
"routing.tasks.tierAuto": "Auto (Router)",
|
||||
"routing.tasks.tierHint": "Welches Modell diese Aufgabe bei welcher Komplexität bedient; \"jede\" führt auf allen Stufen. Reihenfolge = Fallback-Reihenfolge. High eskaliert weiterhin über \"jede\"-Einträge.",
|
||||
"routing.tasks.tierAny": "jede Komplexität",
|
||||
"routing.tasks.addModel": "Modell",
|
||||
"routing.bands.enable": "Eigene Komplexitätsgrenzen",
|
||||
"routing.bands.hint": "Ziehen Sie die Griffe: wo low endet und high beginnt — die Griffe schieben einander, die Zahlen aktualisieren sich live. Der kontinuierliche Router-Score wird mit IHREN Grenzen neu eingeteilt; ohne Häkchen gelten die Grenzen des Routers.",
|
||||
"tier.low": "leicht",
|
||||
"tier.medium": "mittel",
|
||||
"tier.high": "schwer",
|
||||
"bm.scoreHint": "Der kontinuierliche Komplexitätsscore des Routers (0..1) — die Grenzregler auf der Routing-Seite teilen ihn in Stufen.",
|
||||
"settings.cache.noEmbed": "Der Cache ist aktiviert, hat aber KEIN Embedding-Modell — er tut nichts. Fügen Sie dem Pool ein Modell mit kind = \"embedding\" hinzu (z. B. OpenAI text-embedding) oder wählen Sie unten eines.",
|
||||
"settings.cache.entries": "Einträge",
|
||||
"settings.cache.hits": "Treffer",
|
||||
"settings.cache.saved": "gespart",
|
||||
"keys.form.budget": "Monatsbudget, $ (0 = unbegrenzt)",
|
||||
"keys.form.budgetHint": "Überschreiten die Monatsausgaben des Kontos die Grenze, werden Anfragen mit 429 insufficient_quota abgelehnt.",
|
||||
"keys.budgetOf": "Budget {b}/Mon.",
|
||||
"dash.kpi.orBalance": "OpenRouter-Guthaben",
|
||||
"keys.h.install": "Installieren:",
|
||||
"keys.snip.keyInputPh": "API-Schlüssel einfügen",
|
||||
"keys.snip.apply": "Schlüssel einsetzen",
|
||||
};
|
||||
|
||||
Vendored
+252
-1
@@ -59,6 +59,9 @@ export default {
|
||||
// dashboard
|
||||
"dash.title": "Dashboard",
|
||||
"dash.subtitle": "Live traffic, cost and routing analytics for your model pool.",
|
||||
"dash.clear": "Clear logs",
|
||||
"dash.clearConfirm": "Clear all recorded stats and logs? This cannot be undone.",
|
||||
"dash.cleared": "Logs cleared",
|
||||
"dash.range": "Range",
|
||||
"dash.range.1h": "1h",
|
||||
"dash.range.24h": "24h",
|
||||
@@ -129,12 +132,25 @@ export default {
|
||||
"models.form.secretMissing": "not set",
|
||||
"models.form.secretPlaceholder": "set or replace key (write-only)",
|
||||
"models.deleteConfirm": "Delete model \"{id}\"?",
|
||||
"models.form.idOrModel": "Enter an id or a model name",
|
||||
"models.form.apiKey": "API Key",
|
||||
"models.form.apiKeyPlaceholder": "paste your API key",
|
||||
"models.form.apiKeyNotNeeded": "not required (local server)",
|
||||
"models.form.apiKeyHint": "Stored securely on the gateway; it is not shown again.",
|
||||
"models.form.keyStored": "A key is saved — leave blank to keep it, or paste a new one.",
|
||||
"models.form.keyFromEnv": "Using a key from an environment variable.",
|
||||
"models.form.paramsTitle": "Generation & Thinking Parameters (O(1), MTP, Temperature)",
|
||||
"models.form.temperature": "Temperature",
|
||||
"models.form.temperatureHint": "Generation temperature (0.0 = deterministic, 1.0+ = creative).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling threshold (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max Tokens",
|
||||
"models.form.maxTokensHint": "Maximum completion tokens to generate.",
|
||||
"models.form.thinkBudget": "Thinking Mode / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Reasoning token budget (0 = disabled, N > 0 = max reasoning tokens).",
|
||||
"models.form.systemPrompt": "Default System Prompt",
|
||||
"models.form.systemPromptHint": "Prepend automatically if request lacks a system prompt.",
|
||||
"models.form.systemPromptPlaceholder": "You are a helpful assistant...",
|
||||
"models.form.testKey": "Test & load models",
|
||||
"models.form.testing": "checking…",
|
||||
"models.form.foundModels": "{n} models loaded",
|
||||
@@ -315,6 +331,10 @@ export default {
|
||||
"settings.route.maxChars": "Max chars",
|
||||
"settings.route.cacheTtl": "Cache TTL",
|
||||
"settings.route.profile": "Default profile",
|
||||
"settings.route.strategyHelp": "What text is sent to the router for classification: last_user = only the last user message; last_user_plus_system = last user message + system prompt; concat_all = the whole conversation.",
|
||||
"settings.route.maxCharsHelp": "Max characters of that text sent to the router (longer is truncated). More = more accurate classification, but slower/pricier router calls.",
|
||||
"settings.route.cacheTtlHelp": "How long a routing decision is cached for an identical request before the router is asked again (e.g. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Default routing profile when a request doesn't specify one: cost-saver (cheapest), balanced, or quality-first (best model). Biases which model the router picks.",
|
||||
"settings.breaker": "Circuit breaker",
|
||||
"settings.breaker.threshold": "Error threshold",
|
||||
"settings.breaker.cooldown": "Cooldown",
|
||||
@@ -366,7 +386,7 @@ export default {
|
||||
"import.failed": "Conversion failed — see the log.",
|
||||
"import.register": "Register as local model",
|
||||
"import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
|
||||
"import.jobs": "Conversions",
|
||||
"import.jobs": "Downloads & Conversions",
|
||||
"import.cancel": "Cancel",
|
||||
"import.cancelling": "Cancelling — cleaning up partial output…",
|
||||
"import.delete": "Delete",
|
||||
@@ -376,12 +396,22 @@ export default {
|
||||
"import.state.done": "done",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelled",
|
||||
"import.featuredTitle": "Featured Ready-to-Run .cmf Models",
|
||||
"import.featuredSub": "Pre-converted .cmf models from the infosave account on HuggingFace — the list is read live, so newly published models appear here on their own.",
|
||||
"import.readyCmfBadge": "Ready .cmf",
|
||||
"import.directDownload": "Download",
|
||||
"import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
|
||||
"import.q.q8_row": "8-bit per-row — simple, robust.",
|
||||
"import.q.q4": "4-bit block — smallest, lower quality.",
|
||||
"import.q.q4_tiled": "4-bit tiled (Q4_TILED) — 1.66x faster on Metal GPU.",
|
||||
"import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
|
||||
"import.q.q1t": "1.58-bit ternary (Q1T) — BitNet b1.58 with sparse outliers.",
|
||||
"import.q.q1s": "1-bit sparse (Q1S) — 1-bit base with holographic outliers.",
|
||||
"import.q.q1p": "1-bit PTQ (Q1P) — 1-bit with error diffusion.",
|
||||
"import.q.q1": "1-bit (Q1) — pure binary format.",
|
||||
"import.q.f16": "16-bit — no quantization.",
|
||||
"import.q.f32": "32-bit — full precision (large file).",
|
||||
|
||||
"import.adv.title": "Advanced settings",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Linear core",
|
||||
@@ -392,4 +422,225 @@ export default {
|
||||
"import.adv.shard": "Shard max (GB)",
|
||||
"import.adv.noShard": "no sharding",
|
||||
"import.adv.skipMtp": "skip MTP head",
|
||||
"import.adv.o1": "O(1) attention",
|
||||
"import.adv.o1Hint": "Constant-memory streaming attention: the KV cache is released after prefill, so long contexts don't grow memory. Weights stay byte-identical — only a load-time hint is recorded; you can switch back anytime (serve with --o1 off). Validated on Qwen3-0.6B: all layers ≈ ×1.18 perplexity.",
|
||||
"import.adv.o1Off": "off — exact attention (default)",
|
||||
"import.adv.o1All": "all — convert every softmax layer",
|
||||
"import.adv.o1Deep": "deep N — only the N deepest layers",
|
||||
"import.adv.o1Custom": "custom — explicit layer list",
|
||||
"import.adv.o1DeepN": "N deep layers",
|
||||
"import.adv.o1Layers": "Layers (comma-separated)",
|
||||
"import.adv.o1mHint": "landmark budget (default 32)",
|
||||
"import.adv.o1windowHint": "exact-window width (default 128)",
|
||||
"import.adv.o1sinkHint": "permanent exact sink keys (default 4)",
|
||||
|
||||
// onboarding (first run)
|
||||
"onboarding.title": "Welcome setup",
|
||||
"onboarding.welcome": "Welcome to Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choose your language — you can change it anytime in the header.",
|
||||
"onboarding.next": "Next",
|
||||
"onboarding.back": "Back",
|
||||
"onboarding.skip": "Skip for now",
|
||||
"onboarding.skipSetup": "Skip",
|
||||
"onboarding.finish": "Finish setup",
|
||||
"onboarding.featuredTitle": "Ready-made models (one click)",
|
||||
"onboarding.featuredSub": "These are pre-converted .cmf models from infosave on HuggingFace — just Download, no conversion needed. Pick one or more to get started instantly.",
|
||||
"onboarding.download": "Download",
|
||||
"onboarding.hfTitle": "Any HuggingFace model — auto convert",
|
||||
"onboarding.hfSub": "Search any model, pick a quantization, and we convert it to .cmf automatically. No manual config edits.",
|
||||
"onboarding.noJobs": "No downloads yet.",
|
||||
"onboarding.doneTitle": "You're all set",
|
||||
"onboarding.doneSub": "Models you downloaded will be registered automatically. You can add more in Import or Models.",
|
||||
"onboarding.doneHint": "Finish connects every completed download as a local model, starts its .cmf server and points routing at it. You can change all of it later in Models.",
|
||||
"onboarding.willRegister": "Will be connected as local models",
|
||||
"onboarding.nothingToRegister": "No finished downloads yet — nothing to connect. Go back a step to pick a model.",
|
||||
"onboarding.registeredN": "Connected {n} model(s): {ids}",
|
||||
"onboarding.runtimeMissing": "The cortiq runtime isn't installed — install it in Settings → Local models to serve these.",
|
||||
"onboarding.downloaded": "downloaded",
|
||||
"onboarding.update": "Update",
|
||||
"onboarding.deleteFile": "Delete the .cmf file from disk",
|
||||
"onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.",
|
||||
"onboarding.unregistered": "Also unregistered: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Image",
|
||||
"pg.tab.video": "Video",
|
||||
"pg.generate": "Generate",
|
||||
"pg.generating": "Generating…",
|
||||
"pg.genFailed": "Generation failed — see the log.",
|
||||
"pg.result": "Result",
|
||||
"pg.genMetrics": "Generation metrics",
|
||||
"pg.imagePromptPlaceholder": "a red fox sitting in snow at sunset, photorealistic, detailed fur",
|
||||
"pg.videoPromptPlaceholder": "a corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark",
|
||||
"pg.imageEmpty": "The generated image will appear here.",
|
||||
"pg.videoEmpty": "The generated clip (with audio) will appear here.",
|
||||
"pg.width": "Width",
|
||||
"pg.height": "Height",
|
||||
"pg.steps": "Steps",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Random seed",
|
||||
"pg.cfgHint": "Guidance scale; ≤ 1 disables CFG and halves the work.",
|
||||
"pg.frames": "Frames",
|
||||
"pg.framesHint": "At 24 fps; snapped to the model's grid.",
|
||||
"pg.quality": "JPEG quality",
|
||||
"pg.videoSizeHint": "Multiples of 32; the trained short edge is 768.",
|
||||
"pg.videoStepsHint": "The Turbo LoRA is trained for 4 steps.",
|
||||
"pg.firstFrame": "First frame (optional)",
|
||||
"pg.lastFrame": "Last frame (optional)",
|
||||
"pg.keyframeHint": "Turns the run into keyframe-to-video.",
|
||||
"pg.resolution": "Resolution",
|
||||
"pg.genTime": "Generation time",
|
||||
"pg.timePerStep": "Per step",
|
||||
"pg.timePerFrame": "Per frame",
|
||||
"pg.fileSize": "File size",
|
||||
"pg.duration": "Duration",
|
||||
"pg.play": "Play",
|
||||
"pg.noMediaModels": "No local {kind} model yet. Download one (e.g. from the infosave collection) in Import:",
|
||||
"pg.tokSpeed": "Speed",
|
||||
"pg.ttft": "First token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Logs",
|
||||
"logs.subtitle": "Recent gateway log — what the terminal would show.",
|
||||
"logs.filterPlaceholder": "filter…",
|
||||
"logs.pause": "Pause",
|
||||
"logs.resume": "Resume",
|
||||
"logs.empty": "Nothing matches.",
|
||||
"notice.openAdmin": "The admin console is open without a token and reachable from other machines. Set a token if this box is not private.",
|
||||
"notice.update": "Update available: v{v}.",
|
||||
"dash.sys.title": "System",
|
||||
"dash.sys.cores": "cores",
|
||||
"dash.sys.disk": "Disk (models)",
|
||||
"dash.sys.free": "free",
|
||||
"dash.sys.gateway": "gateway",
|
||||
"fit.disk": "not enough disk",
|
||||
"fit.diskTitle": "The download is larger than the free space on the models disk.",
|
||||
"fit.ram": "may not fit in RAM",
|
||||
"fit.ramTitle": "The model is close to or larger than this machine's RAM — it may not run.",
|
||||
"pg.eta": "left",
|
||||
"pg.min": "min",
|
||||
"pg.queued": "In queue — position {n}. It starts as soon as the current generation finishes.",
|
||||
"pg.gallery": "Recent generations",
|
||||
"pg.galDeleteConfirm": "Delete this generation and its files?",
|
||||
"pg.clearChat": "Clear",
|
||||
"keys.snip.title": "Connect your app",
|
||||
"keys.snip.hint": "Paste one of these into your project — only the API key placeholder needs replacing.",
|
||||
"keys.snip.withKey": "Snippets below include the key you just created.",
|
||||
"keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
|
||||
"keys.snip.keyPh": "your API key",
|
||||
"settings.restart": "Restart everything",
|
||||
"settings.restartConfirm": "Restart the gateway and all local model servers? Requests in flight will be dropped.",
|
||||
"settings.restarting": "Restarting — the page reloads when the gateway is back…",
|
||||
"settings.shutdown": "Stop everything & power off",
|
||||
"settings.shutdownConfirm": "Stop all local model servers and power the gateway off? The API and this panel stop answering until you start the gateway again.",
|
||||
"settings.shutdownDone": "The gateway is off.",
|
||||
"settings.shutdownHint": "All local model servers are stopped and the process has exited. Start the gateway again on the server to bring the panel back.",
|
||||
"bench.title": "Local model speed",
|
||||
"bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
|
||||
"bench.run": "Measure",
|
||||
"bench.running": "measuring…",
|
||||
"bench.failed": "failed",
|
||||
"bench.none": "No local .cmf chat models yet.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Small talk",
|
||||
"task.code": "Code",
|
||||
"task.creative-writing": "Creative writing",
|
||||
"task.extraction": "Data extraction",
|
||||
"task.math": "Math",
|
||||
"task.qa": "Q&A",
|
||||
"task.summarization": "Summarization",
|
||||
"task.translation": "Translation",
|
||||
"routing.tasks.title": "Task-based routing",
|
||||
"routing.tasks.hint": "A rule's models go first for that task; the complexity chain below still applies as fallback. Empty = route by complexity only.",
|
||||
"routing.tasks.inherit": "by complexity (no rule)",
|
||||
"routing.tasks.share": "{p}% of requests (7d)",
|
||||
"routing.tasks.fromRouter": "labels live from the router",
|
||||
"routing.tasks.builtin": "router unreachable — built-in label set",
|
||||
"routing.tasks.addCustom": "Add task",
|
||||
"routing.tasks.addPlaceholder": "custom label…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Quality and speed probes through the live pipeline: keyword-checked answers, anti-loop detection, tok/s and time-to-first-token per model.",
|
||||
"bm.models": "Models to test",
|
||||
"bm.quickMode": "Quick test",
|
||||
"bm.quickHint": "The fast subset with short answers — about a minute per local model.",
|
||||
"bm.suiteInfo": "Built-in suite: {q} quick / {f} full tests, plus your custom ones.",
|
||||
"bm.run": "Run",
|
||||
"bm.pickModel": "Pick at least one model.",
|
||||
"bm.testOf": "test {i} of {n}",
|
||||
"bm.results": "Results",
|
||||
"bm.quick": "quick",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Click a cell to see the model's full answer.",
|
||||
"bm.history": "Previous runs (stored in this browser)",
|
||||
"bm.custom": "Custom tests",
|
||||
"bm.customHint": "Prompt + comma-separated keywords the answer must contain (use | for synonyms). Stored in this browser.",
|
||||
"bm.promptPh": "prompt…",
|
||||
"bm.checkPh": "keywords, comma-separated",
|
||||
"bm.addTest": "Add test",
|
||||
"bm.emptyAnswer": "empty answer",
|
||||
"bm.loopDetected": "loop detected",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Smart routing",
|
||||
"bm.mode.models": "Individual models",
|
||||
"bm.mode.routingDesc": "The product's core feature under test: every probe goes through cortiq-auto — you see which task the router detected, which model it picked, whether that matches expectations, and what it cost.",
|
||||
"bm.mode.modelsDesc": "The same suite pinned to selected models — compare their quality, speed and cost head-to-head, bypassing the router.",
|
||||
"bm.answersOk": "answers correct",
|
||||
"bm.classAcc": "task classification",
|
||||
"bm.modelDist": "Requests went to",
|
||||
"bm.expectedLbl": "expected:",
|
||||
"bm.answer": "Answer",
|
||||
"bm.liveRunning": "running…",
|
||||
"bm.spent": "spent",
|
||||
"bm.verdict": "🏁 Done: answers {p}, spent {s}",
|
||||
"bm.clearHistory": "Clear all",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — paste into the terminal (the gateway speaks the Anthropic protocol natively):",
|
||||
"keys.h.codex": "Codex — appends the provider to ~/.codex/config.toml and starts:",
|
||||
"keys.h.opencode": "OpenCode — writes ~/.config/opencode/opencode.json (overwrites!) and starts:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — appends the model to ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline (VS Code) — in extension settings pick:",
|
||||
"keys.h.cline2": "Settings → API Provider → OpenAI Compatible, then paste the values above.",
|
||||
"keys.h.note": "cortiq-auto = smart routing: the router picks the model per request. Use an exact model name to pin one. On Windows run the scripts in Git Bash / WSL, or set the same variables via the system environment settings.",
|
||||
"keys.h.protoOff": "The Anthropic protocol (/v1/messages) is disabled — Claude Code will not connect.",
|
||||
"keys.h.enableProto": "Enable",
|
||||
"keys.h.st.installed": "Installed on the gateway machine",
|
||||
"keys.h.st.configured": "Already configured for this gateway",
|
||||
"keys.h.st.notFound": "Not found on the gateway machine",
|
||||
"bm.degraded": "router degraded",
|
||||
"bm.routerDegraded": "The router did not answer in time — the request used the fallback chain; classification is not judged.",
|
||||
"bm.cutByCap": "budget spent on reasoning — answer never came",
|
||||
"shell.uiStale": "The gateway was updated — this page is running an older interface.",
|
||||
"shell.reload": "Reload",
|
||||
"keys.col.requests": "Requests",
|
||||
"keys.col.tokens": "Tokens in/out",
|
||||
"keys.col.costAll": "$ all-time",
|
||||
"routing.tasks.tier": "Complexity",
|
||||
"routing.tasks.tierAuto": "Auto (router)",
|
||||
"routing.tasks.tierHint": "Which model serves this task at which complexity; \"any\" leads at every tier. Order = fallback order. High verdicts still escalate over any-tier pins.",
|
||||
"routing.tasks.tierAny": "any complexity",
|
||||
"routing.tasks.addModel": "model",
|
||||
"routing.bands.enable": "My complexity boundaries",
|
||||
"routing.bands.hint": "Drag the handles to set where low ends and where high begins — the handles push each other, the numbers update live. The router's continuous score is re-mapped with YOUR boundaries; unchecked = the router's own bands.",
|
||||
"tier.low": "low",
|
||||
"tier.medium": "medium",
|
||||
"tier.high": "high",
|
||||
"bm.scoreHint": "The router's continuous complexity score (0..1) — the boundary sliders on the Routing page cut it into tiers.",
|
||||
"settings.cache.noEmbed": "The cache is enabled but has NO embedding model to work with — it is doing nothing. Add a model with kind = \"embedding\" to the pool (e.g. an OpenAI text-embedding model) or pick one below.",
|
||||
"settings.cache.entries": "entries",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "saved",
|
||||
"keys.form.budget": "Monthly budget, $ (0 = unlimited)",
|
||||
"keys.form.budgetHint": "Requests are rejected with 429 insufficient_quota once the account's month-to-date spend crosses this.",
|
||||
"keys.budgetOf": "budget {b}/mo",
|
||||
"dash.kpi.orBalance": "OpenRouter balance",
|
||||
"keys.h.install": "Install:",
|
||||
"keys.snip.keyInputPh": "paste your API key",
|
||||
"keys.snip.apply": "Insert key",
|
||||
};
|
||||
|
||||
Vendored
+373
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Enrutamiento",
|
||||
"nav.protocols": "Protocolos",
|
||||
"nav.keys": "Claves API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Zona de pruebas",
|
||||
"nav.settings": "Configuración",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clave del router y pagos",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "por tarea",
|
||||
"dash.groupby.account": "por cuenta",
|
||||
"dash.recent.title": "Solicitudes recientes",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba el Playground.",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba la Zona de pruebas.",
|
||||
"dash.col.time": "Hora",
|
||||
"dash.col.model": "Modelo",
|
||||
"dash.col.tier": "Nivel",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "todos",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Zona de pruebas",
|
||||
"pg.subtitle": "Envía un prompt por el pipeline en vivo e inspecciona la decisión de enrutamiento.",
|
||||
"pg.system": "Prompt del sistema",
|
||||
"pg.systemPlaceholder": "mensaje de sistema opcional",
|
||||
@@ -280,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "TTL de entrada",
|
||||
"settings.cache.embed": "Modelo de embeddings",
|
||||
"settings.cache.note": "Los ajustes del caché se aplican tras reiniciar.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importar modelo",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Borrar registros",
|
||||
"dash.clearConfirm": "¿Borrar todas las estadísticas y registros guardados? Esta acción no se puede deshacer.",
|
||||
"dash.cleared": "Registros borrados",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Introduce un id o un nombre de modelo",
|
||||
"models.form.apiKey": "Clave API",
|
||||
"models.form.apiKeyPlaceholder": "pega tu clave API",
|
||||
"models.form.apiKeyNotNeeded": "no necesaria (servidor local)",
|
||||
"models.form.apiKeyHint": "Se guarda de forma segura en la pasarela; no se vuelve a mostrar.",
|
||||
"models.form.keyStored": "Hay una clave guardada — déjalo vacío para conservarla o pega una nueva.",
|
||||
"models.form.keyFromEnv": "Se está usando una clave de una variable de entorno.",
|
||||
"models.form.paramsTitle": "Parámetros de generación y razonamiento (O(1), MTP, temperatura)",
|
||||
"models.form.temperature": "Temperatura",
|
||||
"models.form.temperatureHint": "Temperatura de generación (0.0 = determinista, 1.0+ = creativa).",
|
||||
"models.form.topP": "Top-P (muestreo nucleus)",
|
||||
"models.form.topPHint": "Umbral de muestreo (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens máx.",
|
||||
"models.form.maxTokensHint": "Número máximo de tokens generados.",
|
||||
"models.form.thinkBudget": "Modo razonamiento / presupuesto de pensamiento",
|
||||
"models.form.thinkBudgetHint": "Presupuesto de tokens de razonamiento (0 = desactivado, N > 0 = máximo de tokens de razonamiento).",
|
||||
"models.form.systemPrompt": "Prompt de sistema por defecto",
|
||||
"models.form.systemPromptHint": "Se antepone automáticamente si la petición no trae prompt de sistema.",
|
||||
"models.form.systemPromptPlaceholder": "Eres un asistente útil…",
|
||||
"models.form.testKey": "Probar y cargar modelos",
|
||||
"models.form.testing": "comprobando…",
|
||||
"models.form.foundModels": "{n} modelos cargados",
|
||||
"models.form.modelPlaceholder": "escribe para buscar…",
|
||||
"models.form.modelHint": "Pulsa «Probar y cargar modelos» para traer la lista con tu clave y luego elige uno.",
|
||||
"models.managed": "gestionado",
|
||||
"models.managedHint": "Lo proporciona el servidor CMF gestionado (Ajustes → Modelos locales). Configúralo allí.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Usar el router (desactivado = solo modelos locales)",
|
||||
"settings.cmf": "Modelos locales (CMF)",
|
||||
"settings.cmf.localOnly": "Usar solo modelos locales",
|
||||
"settings.cmf.manage": "Ejecutar un servidor de modelo local",
|
||||
"settings.cmf.autoInstall": "Instalar cortiq automáticamente (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Actualizar cortiq automáticamente",
|
||||
"settings.cmf.model": "Ruta del modelo .cmf local",
|
||||
"settings.cmf.port": "Puerto del servidor local",
|
||||
"settings.cmf.modelId": "Id del modelo en el pool",
|
||||
"settings.cmf.note": "Sirve un .cmf local mediante cortiq (github.com/infosave2007/cmf). Funciona incluso cuando el router no está disponible.",
|
||||
"settings.cmf.threads": "Hilos de trabajo",
|
||||
"settings.cmf.gpu": "Usar GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Los hilos paralelizan los matvec de decodificación (~3× más rápido; 0 = valor por defecto del runtime). Descargar en GPU ayuda sobre todo al prefill de prompts largos — la decodificación está limitada por el ancho de banda de memoria, así que en Apple Silicon es prácticamente neutro.",
|
||||
"settings.cmf.pick": "selecciona un modelo",
|
||||
"settings.cmf.models": "Modelos locales",
|
||||
"settings.cmf.modelsHelp": "Cada fila es un modelo que la pasarela ejecuta (su propio cortiq serve en su propio puerto) y registra en el pool. Añade tantos como archivos .cmf y puertos libres tengas.",
|
||||
"settings.cmf.rowHint": "id · archivo .cmf · puerto · hilos · GPU. Los cambios se aplican al reiniciar; borrar una fila detiene ese servidor.",
|
||||
"settings.cmf.addServer": "+ Añadir modelo local",
|
||||
"settings.cmf.checkPort": "Comprobar puerto",
|
||||
"settings.cmf.portFree": "puerto libre",
|
||||
"settings.cmf.portBusy": "puerto ocupado",
|
||||
"settings.cmf.useSuggested": "usar {port}",
|
||||
"settings.cmf.serving": "sirviendo {model} en :{port}",
|
||||
"settings.cmf.intro": "Ejecuta modelos en local con el motor cortiq — funciona incluso sin conexión o cuando el router está caído.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "La herramienta de línea de comandos `cortiq` que carga y ejecuta modelos .cmf. Instálala una vez antes de arrancar un servidor local.",
|
||||
"settings.cmf.install": "Instalar cortiq",
|
||||
"settings.cmf.reinstall": "Reinstalar / actualizar",
|
||||
"settings.cmf.installing": "instalando… (compila, unos minutos)",
|
||||
"settings.cmf.installedOk": "cortiq instalado",
|
||||
"settings.cmf.notInstalled": "no instalado",
|
||||
"settings.cmf.addModel": "Añadir un modelo",
|
||||
"settings.cmf.addModelHelp": "Los modelos se descargan y convierten en la pestaña Importar (HuggingFace → .cmf). Después apunta el servidor local al archivo .cmf de abajo.",
|
||||
"settings.cmf.importNote": "¿Aún no tienes modelo? Importa uno:",
|
||||
"settings.cmf.importLink": "Abrir Importar",
|
||||
"settings.cmf.modelHelp": "Ruta del archivo .cmf que se va a servir (p. ej. models/qwen.cmf), generado en la pestaña Importar. ¿Varios modelos para distintas tareas? Impórtalos y cambia esta ruta.",
|
||||
"settings.cmf.portHelp": "Puerto TCP en el que escucha el servidor de modelo local (8081 por defecto). Cámbialo solo si choca con otro servicio.",
|
||||
"settings.cmf.threadsHelp": "Cuántos núcleos de CPU usar para la generación. Más = más rápido (~3× frente a un núcleo); 0 deja que decida el runtime.",
|
||||
"settings.cmf.gpuHelp": "Descargar el cálculo matricial en la GPU (Metal/wgpu). En Apple Silicon acelera sobre todo los prompts largos — la decodificación está limitada por el ancho de banda de memoria.",
|
||||
"settings.cmf.localOnlyHelp": "Enviar cada petición al modelo local, ignorando el router y cualquier proveedor en la nube.",
|
||||
"settings.cmf.manageHelp": "La pasarela arranca `cortiq serve` para el .cmf de abajo al iniciarse y enruta hacia él. Requiere cortiq instalado (paso 1). Se aplica al reiniciar.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq falta al arrancar, instalarlo automáticamente desde crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Al arrancar, actualizar cortiq a la versión más reciente si existe una más nueva.",
|
||||
"settings.router.enabledHelp": "Dejar que el router de cortiq clasifique cada petición (tipo de tarea + complejidad) y elija un modelo. Desactivado = usar solo la tabla de enrutado local.",
|
||||
"settings.router.verifyTlsHelp": "Verificar el certificado TLS del router. Desactívalo solo con un certificado autofirmado o ligado a una IP.",
|
||||
"settings.telemetry.metricsHelp": "Exponer métricas Prometheus en GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Añadir un campo `cortiq` (tarea, modelo, coste) al cuerpo de cada respuesta. Las cabeceras de respuesta se envían siempre.",
|
||||
"settings.stats.enabledHelp": "Registrar las peticiones recientes para alimentar el panel y las gráficas.",
|
||||
"settings.cache.enabledHelp": "Caché semántica: devolver una respuesta guardada cuando un prompt nuevo se parece mucho a uno anterior, evitando la llamada al modelo. Requiere un modelo de embeddings en el pool.",
|
||||
"settings.log.bodiesHelp": "Registrar el cuerpo completo de los mensajes de petición y respuesta. Desactivado por defecto por privacidad.",
|
||||
"settings.router.saveKey": "Guardar clave",
|
||||
"settings.router.keySaved": "Clave del router guardada",
|
||||
"settings.router.keyEmpty": "Pega antes la clave",
|
||||
"settings.router.secretHelp": "Pega la clave del correo de api.allaigate.com y pulsa Guardar clave. NO es un valor de «Claves API» — esas son para los clientes que llaman a tu pasarela.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clave guardada (pega otra para reemplazarla)",
|
||||
"settings.route.strategyHelp": "Qué texto se envía al router para clasificar: last_user = solo el último mensaje del usuario; last_user_plus_system = último mensaje del usuario + prompt de sistema; concat_all = toda la conversación.",
|
||||
"settings.route.maxCharsHelp": "Máximo de caracteres de ese texto enviados al router (lo que exceda se recorta). Más = clasificación más precisa, pero llamadas al router más lentas y caras.",
|
||||
"settings.route.cacheTtlHelp": "Cuánto tiempo se cachea una decisión de enrutado para una petición idéntica antes de volver a preguntar al router (p. ej. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Perfil de enrutado por defecto cuando la petición no indica ninguno: cost-saver (el más barato), balanced o quality-first (el mejor modelo). Influye en qué modelo elige el router.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Busca en HuggingFace, convierte a un .cmf local y sírvelo.",
|
||||
"import.searchPlaceholder": "Buscar modelos en HuggingFace (p. ej. qwen3, llama)…",
|
||||
"import.searching": "Buscando…",
|
||||
"import.noResults": "No se encontraron modelos.",
|
||||
"import.configure": "Configurar la conversión",
|
||||
"import.outName": "Nombre de salida",
|
||||
"import.outNameHint": "Nombre del archivo .cmf (letras, dígitos, - y _).",
|
||||
"import.quant": "Cuantización",
|
||||
"import.convert": "Convertir y cargar",
|
||||
"import.converting": "Convirtiendo",
|
||||
"import.started": "Conversión iniciada",
|
||||
"import.failed": "La conversión falló — revisa el registro.",
|
||||
"import.register": "Registrar como modelo local",
|
||||
"import.registered": "Registrado como «{id}». Ejecuta cortiq-server para servirlo.",
|
||||
"import.jobs": "Descargas y conversiones",
|
||||
"import.cancel": "Cancelar",
|
||||
"import.cancelling": "Cancelando — limpiando el archivo incompleto…",
|
||||
"import.delete": "Eliminar",
|
||||
"import.deleteConfirm": "¿Eliminar este modelo y su archivo .cmf convertido del disco?",
|
||||
"import.deleted": "Eliminado junto con sus archivos",
|
||||
"import.state.running": "en curso",
|
||||
"import.state.done": "listo",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelado",
|
||||
"import.featuredTitle": "Modelos .cmf listos para usar",
|
||||
"import.featuredSub": "Modelos .cmf ya convertidos de la cuenta infosave en HuggingFace — la lista se lee en vivo, los modelos nuevos aparecen solos.",
|
||||
"import.readyCmfBadge": ".cmf listo",
|
||||
"import.directDownload": "Descargar",
|
||||
"import.q.q8_2f": "8 bits de dos campos (𝒲×θ) — recomendado: mejor relación calidad/tamaño.",
|
||||
"import.q.q8_row": "8 bits por fila — simple y robusto.",
|
||||
"import.q.q4": "4 bits por bloque — el más pequeño, menor calidad.",
|
||||
"import.q.q4_tiled": "4 bits en mosaico (Q4_TILED) — 1,66× más rápido en GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — presupuesto de bits por experto.",
|
||||
"import.q.q1t": "Ternario de 1,58 bits (Q1T) — BitNet b1.58 con valores atípicos dispersos.",
|
||||
"import.q.q1s": "1 bit disperso (Q1S) — base de 1 bit con valores atípicos holográficos.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit con difusión de error.",
|
||||
"import.q.q1": "1 bit (Q1) — formato puramente binario.",
|
||||
"import.q.f16": "16 bits — sin cuantización.",
|
||||
"import.q.f32": "32 bits — precisión completa (archivo grande).",
|
||||
"import.adv.title": "Ajustes avanzados",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Núcleo lineal",
|
||||
"import.adv.linearHint": "Cómo se trasladan o pliegan las capas de atención lineal.",
|
||||
"import.adv.nphase": "vmf_phase fases/cabeza",
|
||||
"import.adv.vbitShape": "Forma del presupuesto VBIT",
|
||||
"import.adv.meanBits": "Bits medios VBIT",
|
||||
"import.adv.shard": "Tamaño máx. de shard (GB)",
|
||||
"import.adv.noShard": "sin fragmentar",
|
||||
"import.adv.skipMtp": "omitir la cabeza MTP",
|
||||
"import.adv.o1": "Atención O(1)",
|
||||
"import.adv.o1Hint": "Atención en streaming con memoria constante: la caché KV se libera tras el prefill, así que los contextos largos no hacen crecer la memoria. Los pesos quedan idénticos byte a byte — solo se guarda una indicación para el momento de carga; puedes volver atrás cuando quieras (servir con --o1 off). Validado en Qwen3-0.6B: todas las capas ≈ ×1,18 de perplejidad.",
|
||||
"import.adv.o1Off": "desactivada — atención exacta (por defecto)",
|
||||
"import.adv.o1All": "all — convertir todas las capas softmax",
|
||||
"import.adv.o1Deep": "deep N — solo las N capas más profundas",
|
||||
"import.adv.o1Custom": "personalizado — lista explícita de capas",
|
||||
"import.adv.o1DeepN": "N capas profundas",
|
||||
"import.adv.o1Layers": "Capas (separadas por comas)",
|
||||
"import.adv.o1mHint": "presupuesto de landmarks (32 por defecto)",
|
||||
"import.adv.o1windowHint": "ancho de la ventana exacta (128 por defecto)",
|
||||
"import.adv.o1sinkHint": "claves sink exactas permanentes (4 por defecto)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuración inicial",
|
||||
"onboarding.welcome": "Bienvenido a Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Elige tu idioma — puedes cambiarlo cuando quieras en la cabecera.",
|
||||
"onboarding.next": "Siguiente",
|
||||
"onboarding.back": "Atrás",
|
||||
"onboarding.skip": "Omitir por ahora",
|
||||
"onboarding.skipSetup": "Omitir",
|
||||
"onboarding.finish": "Finalizar configuración",
|
||||
"onboarding.featuredTitle": "Modelos listos para usar (un clic)",
|
||||
"onboarding.featuredSub": "Son modelos .cmf ya convertidos de infosave en HuggingFace — basta con descargarlos, sin conversión. Elige uno o varios para empezar al instante.",
|
||||
"onboarding.download": "Descargar",
|
||||
"onboarding.hfTitle": "Cualquier modelo de HuggingFace — conversión automática",
|
||||
"onboarding.hfSub": "Busca cualquier modelo, elige una cuantización y lo convertimos a .cmf automáticamente. Sin editar la configuración a mano.",
|
||||
"onboarding.noJobs": "Todavía no hay descargas.",
|
||||
"onboarding.doneTitle": "Todo listo",
|
||||
"onboarding.doneSub": "Los modelos descargados se registrarán automáticamente. Puedes añadir más en Importar o Modelos.",
|
||||
"onboarding.doneHint": "«Finalizar configuración» conecta cada descarga completada como modelo local, arranca su servidor .cmf y apunta el enrutado hacia él. Todo esto se puede cambiar luego en Modelos.",
|
||||
"onboarding.willRegister": "Se conectarán como modelos locales",
|
||||
"onboarding.nothingToRegister": "Aún no hay descargas terminadas — no hay nada que conectar. Vuelve un paso atrás y elige un modelo.",
|
||||
"onboarding.registeredN": "{n} modelo(s) conectado(s): {ids}",
|
||||
"onboarding.runtimeMissing": "El runtime cortiq no está instalado — instálalo en Ajustes → Modelos locales para poder servirlos.",
|
||||
"onboarding.downloaded": "descargado",
|
||||
"onboarding.update": "Actualizar",
|
||||
"onboarding.deleteFile": "Eliminar el archivo .cmf del disco",
|
||||
"onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.",
|
||||
"onboarding.unregistered": "También se dio de baja: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Imagen",
|
||||
"pg.tab.video": "Vídeo",
|
||||
"pg.generate": "Generar",
|
||||
"pg.generating": "Generando…",
|
||||
"pg.genFailed": "La generación falló — revisa el registro.",
|
||||
"pg.result": "Resultado",
|
||||
"pg.genMetrics": "Métricas de generación",
|
||||
"pg.imagePromptPlaceholder": "un zorro rojo en la nieve al atardecer, fotorrealista, pelaje detallado",
|
||||
"pg.videoPromptPlaceholder": "un corgi con gorro de chef voltea una tortita, chisporroteo y un ladrido alegre",
|
||||
"pg.imageEmpty": "La imagen generada aparecerá aquí.",
|
||||
"pg.videoEmpty": "El clip generado (con audio) aparecerá aquí.",
|
||||
"pg.width": "Ancho",
|
||||
"pg.height": "Alto",
|
||||
"pg.steps": "Pasos",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Seed aleatorio",
|
||||
"pg.cfgHint": "Escala de guidance; ≤ 1 desactiva el CFG y reduce el trabajo a la mitad.",
|
||||
"pg.frames": "Fotogramas",
|
||||
"pg.framesHint": "A 24 fps; se ajusta a la rejilla del modelo.",
|
||||
"pg.quality": "Calidad JPEG",
|
||||
"pg.videoSizeHint": "Múltiplos de 32; el lado corto entrenado es 768.",
|
||||
"pg.videoStepsHint": "La LoRA Turbo está entrenada para 4 pasos.",
|
||||
"pg.firstFrame": "Primer fotograma (opcional)",
|
||||
"pg.lastFrame": "Último fotograma (opcional)",
|
||||
"pg.keyframeHint": "Convierte la ejecución en keyframe-to-video.",
|
||||
"pg.resolution": "Resolución",
|
||||
"pg.genTime": "Tiempo de generación",
|
||||
"pg.timePerStep": "Por paso",
|
||||
"pg.timePerFrame": "Por fotograma",
|
||||
"pg.fileSize": "Tamaño del archivo",
|
||||
"pg.duration": "Duración",
|
||||
"pg.play": "Reproducir",
|
||||
"pg.noMediaModels": "Aún no hay un modelo {kind} local. Descarga uno (p. ej. de la colección infosave) en Importar:",
|
||||
"pg.tokSpeed": "Velocidad",
|
||||
"pg.ttft": "Primer token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Registros",
|
||||
"logs.subtitle": "Registro reciente de la pasarela — lo que mostraría la terminal.",
|
||||
"logs.filterPlaceholder": "filtrar…",
|
||||
"logs.pause": "Pausa",
|
||||
"logs.resume": "Reanudar",
|
||||
"logs.empty": "Nada coincide.",
|
||||
"notice.openAdmin": "La consola está abierta sin token y accesible desde otras máquinas. Configura un token si esta máquina no es privada.",
|
||||
"notice.update": "Actualización disponible: v{v}.",
|
||||
"dash.sys.title": "Sistema",
|
||||
"dash.sys.cores": "núcleos",
|
||||
"dash.sys.disk": "Disco (modelos)",
|
||||
"dash.sys.free": "libre",
|
||||
"dash.sys.gateway": "pasarela",
|
||||
"fit.disk": "no hay espacio suficiente",
|
||||
"fit.diskTitle": "La descarga supera el espacio libre del disco de modelos.",
|
||||
"fit.ram": "puede no caber en RAM",
|
||||
"fit.ramTitle": "El modelo se acerca o supera la RAM de esta máquina — puede que no funcione.",
|
||||
"pg.eta": "restante",
|
||||
"pg.min": "min",
|
||||
"pg.queued": "En cola — posición {n}. Arranca en cuanto termine la generación actual.",
|
||||
"pg.gallery": "Generaciones recientes",
|
||||
"pg.galDeleteConfirm": "¿Eliminar esta generación y sus archivos?",
|
||||
"pg.clearChat": "Limpiar",
|
||||
"keys.snip.title": "Conecta tu aplicación",
|
||||
"keys.snip.hint": "Pega uno de estos ejemplos en tu proyecto — solo hay que sustituir el marcador de la clave.",
|
||||
"keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
|
||||
"keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
|
||||
"keys.snip.keyPh": "tu clave API",
|
||||
"settings.restart": "Reiniciar todo",
|
||||
"settings.restartConfirm": "¿Reiniciar la pasarela y todos los servidores de modelos locales? Las solicitudes en curso se interrumpirán.",
|
||||
"settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
|
||||
"settings.shutdown": "Detener todo y apagar",
|
||||
"settings.shutdownConfirm": "¿Detener todos los servidores de modelos locales y apagar la pasarela? La API y este panel dejarán de responder hasta que vuelva a iniciar la pasarela.",
|
||||
"settings.shutdownDone": "La pasarela está apagada.",
|
||||
"settings.shutdownHint": "Todos los servidores de modelos locales están detenidos y el proceso ha terminado. Inicie de nuevo la pasarela en el servidor para recuperar el panel.",
|
||||
"bench.title": "Velocidad de modelos locales",
|
||||
"bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
|
||||
"bench.run": "Medir",
|
||||
"bench.running": "midiendo…",
|
||||
"bench.failed": "falló",
|
||||
"bench.none": "Aún no hay modelos de chat .cmf locales.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Charla",
|
||||
"task.code": "Código",
|
||||
"task.creative-writing": "Escritura creativa",
|
||||
"task.extraction": "Extracción de datos",
|
||||
"task.math": "Matemáticas",
|
||||
"task.qa": "Preguntas y respuestas",
|
||||
"task.summarization": "Resumen",
|
||||
"task.translation": "Traducción",
|
||||
"routing.tasks.title": "Enrutado por tarea",
|
||||
"routing.tasks.hint": "Los modelos de una regla van primero para su tarea; la cadena por complejidad sigue como respaldo. Vacío = solo por complejidad.",
|
||||
"routing.tasks.inherit": "por complejidad (sin regla)",
|
||||
"routing.tasks.share": "{p}% de las peticiones (7 d)",
|
||||
"routing.tasks.fromRouter": "etiquetas en vivo del router",
|
||||
"routing.tasks.builtin": "router inaccesible — conjunto integrado",
|
||||
"routing.tasks.addCustom": "Añadir tarea",
|
||||
"routing.tasks.addPlaceholder": "etiqueta propia…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Pruebas de calidad y velocidad a través del pipeline en vivo: respuestas verificadas por palabras clave, detección de bucles, tok/s y tiempo del primer token por modelo.",
|
||||
"bm.models": "Modelos a probar",
|
||||
"bm.quickMode": "Prueba rápida",
|
||||
"bm.quickHint": "El subconjunto rápido con respuestas cortas — alrededor de un minuto por modelo local.",
|
||||
"bm.suiteInfo": "Suite integrada: {q} pruebas rápidas / {f} completas, más las tuyas.",
|
||||
"bm.run": "Ejecutar",
|
||||
"bm.pickModel": "Elige al menos un modelo.",
|
||||
"bm.testOf": "prueba {i} de {n}",
|
||||
"bm.results": "Resultados",
|
||||
"bm.quick": "rápida",
|
||||
"bm.test": "Prueba",
|
||||
"bm.cellHint": "Haz clic en una celda para ver la respuesta completa.",
|
||||
"bm.history": "Ejecuciones anteriores (guardadas en este navegador)",
|
||||
"bm.custom": "Pruebas propias",
|
||||
"bm.customHint": "Prompt + palabras clave separadas por comas que la respuesta debe contener (| para sinónimos). Se guardan en este navegador.",
|
||||
"bm.promptPh": "prompt…",
|
||||
"bm.checkPh": "palabras clave, separadas por comas",
|
||||
"bm.addTest": "Añadir prueba",
|
||||
"bm.emptyAnswer": "respuesta vacía",
|
||||
"bm.loopDetected": "bucle detectado",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Enrutado inteligente",
|
||||
"bm.mode.models": "Modelos individuales",
|
||||
"bm.mode.routingDesc": "La función clave del producto a prueba: cada sonda pasa por cortiq-auto — ves qué tarea detectó el router, qué modelo eligió, si coincide con lo esperado y cuánto costó.",
|
||||
"bm.mode.modelsDesc": "La misma suite fijada a los modelos elegidos — compara su calidad, velocidad y coste directamente, sin pasar por el router.",
|
||||
"bm.answersOk": "respuestas correctas",
|
||||
"bm.classAcc": "clasificación de tareas",
|
||||
"bm.modelDist": "Las peticiones fueron a",
|
||||
"bm.expectedLbl": "esperado:",
|
||||
"bm.answer": "Respuesta",
|
||||
"bm.liveRunning": "en curso…",
|
||||
"bm.spent": "gastado",
|
||||
"bm.verdict": "🏁 Listo: respuestas {p}, gastado {s}",
|
||||
"bm.clearHistory": "Borrar todo",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — pega en la terminal (la pasarela habla el protocolo Anthropic de forma nativa):",
|
||||
"keys.h.codex": "Codex — añade el proveedor a ~/.codex/config.toml y arranca:",
|
||||
"keys.h.opencode": "OpenCode — escribe ~/.config/opencode/opencode.json (¡sobrescribe!) y arranca:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — añade el modelo a ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline (VS Code) — en los ajustes de la extensión elige:",
|
||||
"keys.h.cline2": "Ajustes → API Provider → OpenAI Compatible, luego pega los valores de arriba.",
|
||||
"keys.h.note": "cortiq-auto = enrutado inteligente: el router elige el modelo en cada petición. Para fijar uno, indica su nombre exacto. En Windows ejecuta los scripts en Git Bash / WSL o define las mismas variables en el entorno del sistema.",
|
||||
"keys.h.protoOff": "El protocolo Anthropic (/v1/messages) está desactivado — Claude Code no podrá conectarse.",
|
||||
"keys.h.enableProto": "Activar",
|
||||
"keys.h.st.installed": "Instalado en la máquina de la pasarela",
|
||||
"keys.h.st.configured": "Ya configurado para esta pasarela",
|
||||
"keys.h.st.notFound": "No encontrado en la máquina de la pasarela",
|
||||
"bm.degraded": "router degradado",
|
||||
"bm.routerDegraded": "El router no respondió a tiempo — la petición usó la cadena de reserva; la clasificación no se cuenta.",
|
||||
"bm.cutByCap": "presupuesto gastado en razonar — sin respuesta final",
|
||||
"shell.uiStale": "La pasarela se actualizó — esta página usa una interfaz antigua.",
|
||||
"shell.reload": "Recargar",
|
||||
"keys.col.requests": "Peticiones",
|
||||
"keys.col.tokens": "Tokens ent/sal",
|
||||
"keys.col.costAll": "$ total",
|
||||
"routing.tasks.tier": "Complejidad",
|
||||
"routing.tasks.tierAuto": "Auto (router)",
|
||||
"routing.tasks.tierHint": "Qué modelo atiende esta tarea en qué complejidad; «cualquiera» lidera en todos los niveles. El orden = orden de respaldo. Un veredicto high sigue escalando sobre las entradas «cualquiera».",
|
||||
"routing.tasks.tierAny": "cualquier complejidad",
|
||||
"routing.tasks.addModel": "modelo",
|
||||
"routing.bands.enable": "Mis límites de complejidad",
|
||||
"routing.bands.hint": "Arrastra los tiradores: dónde termina low y dónde empieza high — se empujan entre sí y las cifras se actualizan en vivo. La puntuación continua del router se re-mapea con TUS límites; sin marcar = las bandas del propio router.",
|
||||
"tier.low": "fácil",
|
||||
"tier.medium": "medio",
|
||||
"tier.high": "difícil",
|
||||
"bm.scoreHint": "La puntuación continua de complejidad del router (0..1) — los deslizadores de límites de la página Enrutado la dividen en niveles.",
|
||||
"settings.cache.noEmbed": "La caché está activada pero NO tiene modelo de embeddings — no hace nada. Añade al pool un modelo con kind = \"embedding\" (p. ej. OpenAI text-embedding) o elige uno abajo.",
|
||||
"settings.cache.entries": "entradas",
|
||||
"settings.cache.hits": "aciertos",
|
||||
"settings.cache.saved": "ahorrado",
|
||||
"keys.form.budget": "Presupuesto mensual, $ (0 = sin límite)",
|
||||
"keys.form.budgetHint": "Cuando el gasto del mes supera el techo, las peticiones se rechazan con 429 insufficient_quota.",
|
||||
"keys.budgetOf": "presupuesto {b}/mes",
|
||||
"dash.kpi.orBalance": "Saldo OpenRouter",
|
||||
"keys.h.install": "Instalar:",
|
||||
"keys.snip.keyInputPh": "pega tu clave API",
|
||||
"keys.snip.apply": "Insertar clave",
|
||||
};
|
||||
|
||||
Vendored
+373
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Routage",
|
||||
"nav.protocols": "Protocoles",
|
||||
"nav.keys": "Clés API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Bac à sable",
|
||||
"nav.settings": "Paramètres",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clé routeur & facturation",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "par tâche",
|
||||
"dash.groupby.account": "par compte",
|
||||
"dash.recent.title": "Requêtes récentes",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Playground.",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Bac à sable.",
|
||||
"dash.col.time": "Heure",
|
||||
"dash.col.model": "Modèle",
|
||||
"dash.col.tier": "Niveau",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "tous",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Bac à sable",
|
||||
"pg.subtitle": "Envoyez une invite via le pipeline en direct et inspectez la décision de routage.",
|
||||
"pg.system": "Invite système",
|
||||
"pg.systemPlaceholder": "message système optionnel",
|
||||
@@ -280,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "TTL d'entrée",
|
||||
"settings.cache.embed": "Modèle d'embeddings",
|
||||
"settings.cache.note": "Les paramètres du cache prennent effet après un redémarrage.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importer un modèle",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Effacer les journaux",
|
||||
"dash.clearConfirm": "Effacer toutes les statistiques et les journaux enregistrés ? Action irréversible.",
|
||||
"dash.cleared": "Journaux effacés",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Saisissez un identifiant ou un nom de modèle",
|
||||
"models.form.apiKey": "Clé API",
|
||||
"models.form.apiKeyPlaceholder": "collez votre clé API",
|
||||
"models.form.apiKeyNotNeeded": "non requise (serveur local)",
|
||||
"models.form.apiKeyHint": "Stockée de façon sécurisée sur la passerelle ; elle n'est plus affichée ensuite.",
|
||||
"models.form.keyStored": "Une clé est enregistrée — laissez vide pour la conserver, ou collez-en une nouvelle.",
|
||||
"models.form.keyFromEnv": "Une clé issue d'une variable d'environnement est utilisée.",
|
||||
"models.form.paramsTitle": "Paramètres de génération et de raisonnement (O(1), MTP, température)",
|
||||
"models.form.temperature": "Température",
|
||||
"models.form.temperatureHint": "Température de génération (0.0 = déterministe, 1.0+ = créatif).",
|
||||
"models.form.topP": "Top-P (échantillonnage nucleus)",
|
||||
"models.form.topPHint": "Seuil d'échantillonnage (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens max.",
|
||||
"models.form.maxTokensHint": "Nombre maximal de tokens générés.",
|
||||
"models.form.thinkBudget": "Mode raisonnement / budget de réflexion",
|
||||
"models.form.thinkBudgetHint": "Budget de tokens de raisonnement (0 = désactivé, N > 0 = maximum de tokens de raisonnement).",
|
||||
"models.form.systemPrompt": "Prompt système par défaut",
|
||||
"models.form.systemPromptHint": "Ajouté automatiquement si la requête ne contient pas de prompt système.",
|
||||
"models.form.systemPromptPlaceholder": "Vous êtes un assistant utile…",
|
||||
"models.form.testKey": "Tester et charger les modèles",
|
||||
"models.form.testing": "vérification…",
|
||||
"models.form.foundModels": "{n} modèles chargés",
|
||||
"models.form.modelPlaceholder": "tapez pour rechercher…",
|
||||
"models.form.modelHint": "Cliquez sur « Tester et charger les modèles » pour récupérer la liste liée à votre clé, puis choisissez-en un.",
|
||||
"models.managed": "géré",
|
||||
"models.managedHint": "Fourni par le serveur CMF géré (Paramètres → Modèles locaux). Configurez-le là-bas.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Utiliser le routeur (désactivé = modèles locaux uniquement)",
|
||||
"settings.cmf": "Modèles locaux (CMF)",
|
||||
"settings.cmf.localOnly": "Utiliser uniquement les modèles locaux",
|
||||
"settings.cmf.manage": "Lancer un serveur de modèle local",
|
||||
"settings.cmf.autoInstall": "Installer cortiq automatiquement (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Mettre cortiq à jour automatiquement",
|
||||
"settings.cmf.model": "Chemin du modèle .cmf local",
|
||||
"settings.cmf.port": "Port du serveur local",
|
||||
"settings.cmf.modelId": "Identifiant du modèle dans le pool",
|
||||
"settings.cmf.note": "Sert un .cmf local via cortiq (github.com/infosave2007/cmf). Fonctionne même quand le routeur est indisponible.",
|
||||
"settings.cmf.threads": "Threads de travail",
|
||||
"settings.cmf.gpu": "Utiliser le GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Les threads parallélisent les matvecs de décodage (~3× plus rapide ; 0 = valeur par défaut du runtime). Le déport GPU aide surtout au prefill des longs prompts — le décodage est limité par la bande passante mémoire, donc sur Apple Silicon l'effet est à peu près neutre.",
|
||||
"settings.cmf.pick": "choisir un modèle",
|
||||
"settings.cmf.models": "Modèles locaux",
|
||||
"settings.cmf.modelsHelp": "Chaque ligne est un modèle que la passerelle exécute (son propre cortiq serve sur son propre port) et enregistre dans le pool. Ajoutez-en autant que vous avez de fichiers .cmf et de ports libres.",
|
||||
"settings.cmf.rowHint": "id · fichier .cmf · port · threads · GPU. Les changements prennent effet au redémarrage ; supprimer une ligne arrête ce serveur.",
|
||||
"settings.cmf.addServer": "+ Ajouter un modèle local",
|
||||
"settings.cmf.checkPort": "Vérifier le port",
|
||||
"settings.cmf.portFree": "port libre",
|
||||
"settings.cmf.portBusy": "port occupé",
|
||||
"settings.cmf.useSuggested": "utiliser {port}",
|
||||
"settings.cmf.serving": "{model} servi sur :{port}",
|
||||
"settings.cmf.intro": "Exécutez des modèles en local avec le moteur cortiq — fonctionne même hors ligne ou quand le routeur est en panne.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "L'outil en ligne de commande `cortiq` qui charge et exécute les modèles .cmf. Installez-le une fois avant de démarrer un serveur local.",
|
||||
"settings.cmf.install": "Installer cortiq",
|
||||
"settings.cmf.reinstall": "Réinstaller / mettre à jour",
|
||||
"settings.cmf.installing": "installation… (compilation, quelques minutes)",
|
||||
"settings.cmf.installedOk": "cortiq installé",
|
||||
"settings.cmf.notInstalled": "non installé",
|
||||
"settings.cmf.addModel": "Ajouter un modèle",
|
||||
"settings.cmf.addModelHelp": "Les modèles se téléchargent et se convertissent dans l'onglet Importer (HuggingFace → .cmf). Faites ensuite pointer le serveur local vers le fichier .cmf ci-dessous.",
|
||||
"settings.cmf.importNote": "Pas encore de modèle ? Importez-en un :",
|
||||
"settings.cmf.importLink": "Ouvrir Importer",
|
||||
"settings.cmf.modelHelp": "Chemin du fichier .cmf à servir (par ex. models/qwen.cmf), produit par l'onglet Importer. Plusieurs modèles pour des tâches différentes ? Importez-les et changez ce chemin.",
|
||||
"settings.cmf.portHelp": "Port TCP sur lequel écoute le serveur de modèle local (8081 par défaut). À changer seulement en cas de conflit avec un autre service.",
|
||||
"settings.cmf.threadsHelp": "Nombre de cœurs CPU utilisés pour la génération. Plus = plus rapide (~3× par rapport à un seul cœur) ; 0 laisse le runtime décider.",
|
||||
"settings.cmf.gpuHelp": "Déporter les calculs matriciels sur le GPU (Metal/wgpu). Sur Apple Silicon, cela accélère surtout les longs prompts — le décodage est limité par la bande passante mémoire.",
|
||||
"settings.cmf.localOnlyHelp": "Router chaque requête vers le modèle local, en ignorant le routeur et tous les fournisseurs cloud.",
|
||||
"settings.cmf.manageHelp": "Au démarrage, la passerelle lance `cortiq serve` pour le .cmf ci-dessous et y route les requêtes. Nécessite cortiq installé (étape 1). Prend effet au redémarrage.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq est absent au démarrage, l'installer automatiquement depuis crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Au démarrage, mettre cortiq à niveau vers la dernière version si elle est plus récente.",
|
||||
"settings.router.enabledHelp": "Laisser le routeur cortiq classer chaque requête (type de tâche + complexité) et choisir un modèle. Désactivé = utiliser uniquement la table de routage locale.",
|
||||
"settings.router.verifyTlsHelp": "Vérifier le certificat TLS du routeur. À désactiver uniquement pour un certificat auto-signé ou lié à une IP.",
|
||||
"settings.telemetry.metricsHelp": "Exposer les métriques Prometheus sur GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Ajouter un champ `cortiq` (tâche, modèle, coût) au corps de chaque réponse. Les en-têtes de réponse sont toujours envoyés.",
|
||||
"settings.stats.enabledHelp": "Enregistrer les requêtes récentes pour alimenter le tableau de bord et les graphiques.",
|
||||
"settings.cache.enabledHelp": "Cache sémantique : renvoyer une réponse enregistrée quand un nouveau prompt ressemble beaucoup à un précédent, en évitant l'appel au modèle. Nécessite un modèle d'embedding dans le pool.",
|
||||
"settings.log.bodiesHelp": "Journaliser le corps complet des messages de requête et de réponse. Désactivé par défaut pour la confidentialité.",
|
||||
"settings.router.saveKey": "Enregistrer la clé",
|
||||
"settings.router.keySaved": "Clé du routeur enregistrée",
|
||||
"settings.router.keyEmpty": "Collez d'abord la clé",
|
||||
"settings.router.secretHelp": "Collez la clé reçue par e-mail depuis api.allaigate.com, puis cliquez sur Enregistrer la clé. Ce n'est PAS une valeur de « Clés API » — celles-ci servent aux clients qui appellent votre passerelle.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clé enregistrée (recollez-en une pour la remplacer)",
|
||||
"settings.route.strategyHelp": "Texte envoyé au routeur pour la classification : last_user = uniquement le dernier message utilisateur ; last_user_plus_system = dernier message utilisateur + prompt système ; concat_all = toute la conversation.",
|
||||
"settings.route.maxCharsHelp": "Nombre maximal de caractères de ce texte envoyés au routeur (au-delà, il est tronqué). Plus = classification plus précise, mais appels au routeur plus lents et plus coûteux.",
|
||||
"settings.route.cacheTtlHelp": "Durée de mise en cache d'une décision de routage pour une requête identique avant de réinterroger le routeur (par ex. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Profil de routage par défaut quand la requête n'en précise aucun : cost-saver (le moins cher), balanced ou quality-first (meilleur modèle). Influence le modèle choisi par le routeur.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Cherchez sur HuggingFace, convertissez en .cmf local et servez-le.",
|
||||
"import.searchPlaceholder": "Rechercher des modèles HuggingFace (par ex. qwen3, llama)…",
|
||||
"import.searching": "Recherche…",
|
||||
"import.noResults": "Aucun modèle trouvé.",
|
||||
"import.configure": "Configurer la conversion",
|
||||
"import.outName": "Nom de sortie",
|
||||
"import.outNameHint": "Nom du fichier .cmf (lettres, chiffres, - et _).",
|
||||
"import.quant": "Quantification",
|
||||
"import.convert": "Convertir et charger",
|
||||
"import.converting": "Conversion",
|
||||
"import.started": "Conversion lancée",
|
||||
"import.failed": "Échec de la conversion — voir le journal.",
|
||||
"import.register": "Enregistrer comme modèle local",
|
||||
"import.registered": "Enregistré sous « {id} ». Lancez cortiq-server pour le servir.",
|
||||
"import.jobs": "Téléchargements et conversions",
|
||||
"import.cancel": "Annuler",
|
||||
"import.cancelling": "Annulation — nettoyage du fichier incomplet…",
|
||||
"import.delete": "Supprimer",
|
||||
"import.deleteConfirm": "Supprimer ce modèle et son fichier .cmf converti du disque ?",
|
||||
"import.deleted": "Supprimé avec ses fichiers",
|
||||
"import.state.running": "en cours",
|
||||
"import.state.done": "terminé",
|
||||
"import.state.error": "erreur",
|
||||
"import.state.cancelled": "annulé",
|
||||
"import.featuredTitle": "Modèles .cmf prêts à l'emploi",
|
||||
"import.featuredSub": "Modèles .cmf déjà convertis du compte infosave sur HuggingFace — la liste est lue en direct, les nouveaux modèles apparaissent d'eux-mêmes.",
|
||||
"import.readyCmfBadge": ".cmf prêt",
|
||||
"import.directDownload": "Télécharger",
|
||||
"import.q.q8_2f": "8 bits à deux champs (𝒲×θ) — recommandé : meilleur rapport qualité/taille.",
|
||||
"import.q.q8_row": "8 bits par ligne — simple et robuste.",
|
||||
"import.q.q4": "4 bits par bloc — le plus compact, qualité moindre.",
|
||||
"import.q.q4_tiled": "4 bits en tuiles (Q4_TILED) — 1,66× plus rapide sur GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — budget de bits par expert.",
|
||||
"import.q.q1t": "Ternaire 1,58 bit (Q1T) — BitNet b1.58 avec valeurs aberrantes éparses.",
|
||||
"import.q.q1s": "1 bit épars (Q1S) — base 1 bit avec valeurs aberrantes holographiques.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit avec diffusion d'erreur.",
|
||||
"import.q.q1": "1 bit (Q1) — format purement binaire.",
|
||||
"import.q.f16": "16 bits — sans quantification.",
|
||||
"import.q.f32": "32 bits — pleine précision (fichier volumineux).",
|
||||
"import.adv.title": "Paramètres avancés",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Cœur linéaire",
|
||||
"import.adv.linearHint": "Comment les couches d'attention linéaire sont reportées ou fusionnées.",
|
||||
"import.adv.nphase": "vmf_phase phases/tête",
|
||||
"import.adv.vbitShape": "Forme du budget VBIT",
|
||||
"import.adv.meanBits": "Bits moyens VBIT",
|
||||
"import.adv.shard": "Taille max. des shards (Go)",
|
||||
"import.adv.noShard": "pas de découpage",
|
||||
"import.adv.skipMtp": "ignorer la tête MTP",
|
||||
"import.adv.o1": "Attention O(1)",
|
||||
"import.adv.o1Hint": "Attention en flux à mémoire constante : le cache KV est libéré après le prefill, la mémoire ne croît donc pas avec de longs contextes. Les poids restent identiques octet pour octet — seule une indication de chargement est enregistrée ; vous pouvez revenir en arrière à tout moment (servir avec --o1 off). Validé sur Qwen3-0.6B : toutes couches ≈ ×1,18 de perplexité.",
|
||||
"import.adv.o1Off": "désactivé — attention exacte (par défaut)",
|
||||
"import.adv.o1All": "all — convertir chaque couche softmax",
|
||||
"import.adv.o1Deep": "deep N — seulement les N couches les plus profondes",
|
||||
"import.adv.o1Custom": "personnalisé — liste explicite de couches",
|
||||
"import.adv.o1DeepN": "N couches profondes",
|
||||
"import.adv.o1Layers": "Couches (séparées par des virgules)",
|
||||
"import.adv.o1mHint": "budget de landmarks (32 par défaut)",
|
||||
"import.adv.o1windowHint": "largeur de la fenêtre exacte (128 par défaut)",
|
||||
"import.adv.o1sinkHint": "clés sink exactes permanentes (4 par défaut)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuration initiale",
|
||||
"onboarding.welcome": "Bienvenue dans Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choisissez votre langue — vous pourrez la changer à tout moment dans l'en-tête.",
|
||||
"onboarding.next": "Suivant",
|
||||
"onboarding.back": "Retour",
|
||||
"onboarding.skip": "Passer pour l'instant",
|
||||
"onboarding.skipSetup": "Passer",
|
||||
"onboarding.finish": "Terminer la configuration",
|
||||
"onboarding.featuredTitle": "Modèles prêts à l'emploi (en un clic)",
|
||||
"onboarding.featuredSub": "Ce sont des modèles .cmf déjà convertis, publiés par infosave sur HuggingFace — il suffit de les télécharger, aucune conversion nécessaire. Choisissez-en un ou plusieurs pour démarrer immédiatement.",
|
||||
"onboarding.download": "Télécharger",
|
||||
"onboarding.hfTitle": "N'importe quel modèle HuggingFace — conversion automatique",
|
||||
"onboarding.hfSub": "Cherchez un modèle, choisissez une quantification, et nous le convertissons automatiquement en .cmf. Aucune modification manuelle de la configuration.",
|
||||
"onboarding.noJobs": "Aucun téléchargement pour l'instant.",
|
||||
"onboarding.doneTitle": "Tout est prêt",
|
||||
"onboarding.doneSub": "Les modèles téléchargés seront enregistrés automatiquement. Vous pourrez en ajouter d'autres dans Importer ou Modèles.",
|
||||
"onboarding.doneHint": "« Terminer la configuration » connecte chaque téléchargement achevé comme modèle local, démarre son serveur .cmf et y dirige le routage. Tout cela reste modifiable ensuite dans Modèles.",
|
||||
"onboarding.willRegister": "Seront connectés comme modèles locaux",
|
||||
"onboarding.nothingToRegister": "Aucun téléchargement terminé — rien à connecter. Revenez à l'étape précédente pour choisir un modèle.",
|
||||
"onboarding.registeredN": "{n} modèle(s) connecté(s) : {ids}",
|
||||
"onboarding.runtimeMissing": "Le runtime cortiq n'est pas installé — installez-le dans Paramètres → Modèles locaux pour les servir.",
|
||||
"onboarding.downloaded": "téléchargé",
|
||||
"onboarding.update": "Mettre à jour",
|
||||
"onboarding.deleteFile": "Supprimer le fichier .cmf du disque",
|
||||
"onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.",
|
||||
"onboarding.unregistered": "Également désenregistré : {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Chat",
|
||||
"pg.tab.image": "Image",
|
||||
"pg.tab.video": "Vidéo",
|
||||
"pg.generate": "Générer",
|
||||
"pg.generating": "Génération…",
|
||||
"pg.genFailed": "Échec de la génération — voir le journal.",
|
||||
"pg.result": "Résultat",
|
||||
"pg.genMetrics": "Métriques de génération",
|
||||
"pg.imagePromptPlaceholder": "un renard roux dans la neige au coucher du soleil, photoréaliste, fourrure détaillée",
|
||||
"pg.videoPromptPlaceholder": "un corgi en toque de chef fait sauter une crêpe, grésillements et aboiement joyeux",
|
||||
"pg.imageEmpty": "L'image générée apparaîtra ici.",
|
||||
"pg.videoEmpty": "Le clip généré (avec le son) apparaîtra ici.",
|
||||
"pg.width": "Largeur",
|
||||
"pg.height": "Hauteur",
|
||||
"pg.steps": "Étapes",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Seed aléatoire",
|
||||
"pg.cfgHint": "Force du guidance ; ≤ 1 désactive le CFG et divise le travail par deux.",
|
||||
"pg.frames": "Images",
|
||||
"pg.framesHint": "À 24 fps ; arrondi à la grille du modèle.",
|
||||
"pg.quality": "Qualité JPEG",
|
||||
"pg.videoSizeHint": "Multiples de 32 ; le petit côté entraîné est 768.",
|
||||
"pg.videoStepsHint": "La LoRA Turbo est entraînée pour 4 étapes.",
|
||||
"pg.firstFrame": "Première image (optionnel)",
|
||||
"pg.lastFrame": "Dernière image (optionnel)",
|
||||
"pg.keyframeHint": "Transforme la génération en keyframe-to-video.",
|
||||
"pg.resolution": "Résolution",
|
||||
"pg.genTime": "Temps de génération",
|
||||
"pg.timePerStep": "Par étape",
|
||||
"pg.timePerFrame": "Par image",
|
||||
"pg.fileSize": "Taille du fichier",
|
||||
"pg.duration": "Durée",
|
||||
"pg.play": "Lire",
|
||||
"pg.noMediaModels": "Pas encore de modèle {kind} local. Téléchargez-en un (par ex. de la collection infosave) dans Importer :",
|
||||
"pg.tokSpeed": "Vitesse",
|
||||
"pg.ttft": "Premier token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Journaux",
|
||||
"logs.subtitle": "Journal récent de la passerelle — ce que montrerait le terminal.",
|
||||
"logs.filterPlaceholder": "filtrer…",
|
||||
"logs.pause": "Pause",
|
||||
"logs.resume": "Reprendre",
|
||||
"logs.empty": "Rien ne correspond.",
|
||||
"notice.openAdmin": "La console est ouverte sans jeton et accessible depuis d'autres machines. Définissez un jeton si cette machine n'est pas privée.",
|
||||
"notice.update": "Mise à jour disponible : v{v}.",
|
||||
"dash.sys.title": "Système",
|
||||
"dash.sys.cores": "cœurs",
|
||||
"dash.sys.disk": "Disque (modèles)",
|
||||
"dash.sys.free": "libre",
|
||||
"dash.sys.gateway": "passerelle",
|
||||
"fit.disk": "espace disque insuffisant",
|
||||
"fit.diskTitle": "Le téléchargement dépasse l'espace libre du disque des modèles.",
|
||||
"fit.ram": "risque de ne pas tenir en RAM",
|
||||
"fit.ramTitle": "Le modèle approche ou dépasse la RAM de cette machine — il pourrait ne pas tourner.",
|
||||
"pg.eta": "restant",
|
||||
"pg.min": "min",
|
||||
"pg.queued": "En file d'attente — position {n}. Démarre dès la fin de la génération en cours.",
|
||||
"pg.gallery": "Générations récentes",
|
||||
"pg.galDeleteConfirm": "Supprimer cette génération et ses fichiers ?",
|
||||
"pg.clearChat": "Effacer",
|
||||
"keys.snip.title": "Connecter votre application",
|
||||
"keys.snip.hint": "Collez un de ces exemples dans votre projet — seul le espace réservé de la clé est à remplacer.",
|
||||
"keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
|
||||
"keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
|
||||
"keys.snip.keyPh": "votre clé API",
|
||||
"settings.restart": "Tout redémarrer",
|
||||
"settings.restartConfirm": "Redémarrer la passerelle et tous les serveurs de modèles locaux ? Les requêtes en cours seront interrompues.",
|
||||
"settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
|
||||
"settings.shutdown": "Tout arrêter et éteindre",
|
||||
"settings.shutdownConfirm": "Arrêter tous les serveurs de modèles locaux et éteindre la passerelle ? L'API et ce panneau ne répondront plus tant que vous n'aurez pas relancé la passerelle.",
|
||||
"settings.shutdownDone": "La passerelle est éteinte.",
|
||||
"settings.shutdownHint": "Tous les serveurs de modèles locaux sont arrêtés et le processus est terminé. Relancez la passerelle sur le serveur pour retrouver le panneau.",
|
||||
"bench.title": "Vitesse des modèles locaux",
|
||||
"bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
|
||||
"bench.run": "Mesurer",
|
||||
"bench.running": "mesure…",
|
||||
"bench.failed": "échec",
|
||||
"bench.none": "Pas encore de modèles de chat .cmf locaux.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Bavardage",
|
||||
"task.code": "Code",
|
||||
"task.creative-writing": "Écriture créative",
|
||||
"task.extraction": "Extraction de données",
|
||||
"task.math": "Mathématiques",
|
||||
"task.qa": "Questions-réponses",
|
||||
"task.summarization": "Résumé",
|
||||
"task.translation": "Traduction",
|
||||
"routing.tasks.title": "Routage par tâche",
|
||||
"routing.tasks.hint": "Les modèles d'une règle passent en premier pour leur tâche ; la chaîne par complexité reste en secours. Vide = complexité seule.",
|
||||
"routing.tasks.inherit": "par complexité (aucune règle)",
|
||||
"routing.tasks.share": "{p}% des requêtes (7 j)",
|
||||
"routing.tasks.fromRouter": "étiquettes reçues du routeur",
|
||||
"routing.tasks.builtin": "routeur injoignable — jeu d'étiquettes intégré",
|
||||
"routing.tasks.addCustom": "Ajouter une tâche",
|
||||
"routing.tasks.addPlaceholder": "étiquette personnalisée…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Benchmark",
|
||||
"bm.title": "Benchmark",
|
||||
"bm.subtitle": "Sondes de qualité et de vitesse via le pipeline en direct : réponses vérifiées par mots-clés, boucles détectées, tok/s et temps du premier token par modèle.",
|
||||
"bm.models": "Modèles à tester",
|
||||
"bm.quickMode": "Test rapide",
|
||||
"bm.quickHint": "Le sous-ensemble rapide aux réponses courtes — environ une minute par modèle local.",
|
||||
"bm.suiteInfo": "Suite intégrée : {q} tests rapides / {f} complets, plus les vôtres.",
|
||||
"bm.run": "Lancer",
|
||||
"bm.pickModel": "Choisissez au moins un modèle.",
|
||||
"bm.testOf": "test {i} sur {n}",
|
||||
"bm.results": "Résultats",
|
||||
"bm.quick": "rapide",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Cliquez sur une cellule pour voir la réponse complète.",
|
||||
"bm.history": "Exécutions précédentes (stockées dans ce navigateur)",
|
||||
"bm.custom": "Tests personnalisés",
|
||||
"bm.customHint": "Prompt + mots-clés séparés par des virgules que la réponse doit contenir (| pour les synonymes). Stockés dans ce navigateur.",
|
||||
"bm.promptPh": "prompt…",
|
||||
"bm.checkPh": "mots-clés, séparés par des virgules",
|
||||
"bm.addTest": "Ajouter un test",
|
||||
"bm.emptyAnswer": "réponse vide",
|
||||
"bm.loopDetected": "boucle détectée",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Routage intelligent",
|
||||
"bm.mode.models": "Modèles individuels",
|
||||
"bm.mode.routingDesc": "La fonctionnalité clé du produit à l'épreuve : chaque sonde passe par cortiq-auto — vous voyez quelle tâche le routeur a détectée, quel modèle il a choisi, si cela correspond aux attentes et combien cela a coûté.",
|
||||
"bm.mode.modelsDesc": "La même suite épinglée aux modèles choisis — comparez leur qualité, vitesse et coût en direct, sans passer par le routeur.",
|
||||
"bm.answersOk": "réponses correctes",
|
||||
"bm.classAcc": "classification des tâches",
|
||||
"bm.modelDist": "Requêtes envoyées vers",
|
||||
"bm.expectedLbl": "attendu :",
|
||||
"bm.answer": "Réponse",
|
||||
"bm.liveRunning": "en cours…",
|
||||
"bm.spent": "dépensé",
|
||||
"bm.verdict": "🏁 Terminé : réponses {p}, dépensé {s}",
|
||||
"bm.clearHistory": "Tout effacer",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — collez dans le terminal (la passerelle parle nativement le protocole Anthropic) :",
|
||||
"keys.h.codex": "Codex — ajoute le fournisseur à ~/.codex/config.toml et démarre :",
|
||||
"keys.h.opencode": "OpenCode — écrit ~/.config/opencode/opencode.json (écrase !) et démarre :",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — ajoute le modèle à ~/.continue/config.yaml :",
|
||||
"keys.h.cline1": "Cline (VS Code) — dans les réglages de l'extension, choisissez :",
|
||||
"keys.h.cline2": "Réglages → API Provider → OpenAI Compatible, puis collez les valeurs ci-dessus.",
|
||||
"keys.h.note": "cortiq-auto = routage intelligent : le routeur choisit le modèle à chaque requête. Pour en fixer un, indiquez son nom exact. Sous Windows, lancez les scripts dans Git Bash / WSL ou définissez les mêmes variables via l'environnement système.",
|
||||
"keys.h.protoOff": "Le protocole Anthropic (/v1/messages) est désactivé — Claude Code ne pourra pas se connecter.",
|
||||
"keys.h.enableProto": "Activer",
|
||||
"keys.h.st.installed": "Installé sur la machine de la passerelle",
|
||||
"keys.h.st.configured": "Déjà configuré pour cette passerelle",
|
||||
"keys.h.st.notFound": "Introuvable sur la machine de la passerelle",
|
||||
"bm.degraded": "routeur dégradé",
|
||||
"bm.routerDegraded": "Le routeur n'a pas répondu à temps — la requête a suivi la chaîne de secours ; la classification n'est pas comptée.",
|
||||
"bm.cutByCap": "budget épuisé en raisonnement — pas de réponse",
|
||||
"shell.uiStale": "La passerelle a été mise à jour — cette page utilise une ancienne interface.",
|
||||
"shell.reload": "Recharger",
|
||||
"keys.col.requests": "Requêtes",
|
||||
"keys.col.tokens": "Tokens in/out",
|
||||
"keys.col.costAll": "$ au total",
|
||||
"routing.tasks.tier": "Complexité",
|
||||
"routing.tasks.tierAuto": "Auto (routeur)",
|
||||
"routing.tasks.tierHint": "Quel modèle sert cette tâche à quelle complexité ; « toute » mène à tous les niveaux. L'ordre = ordre de repli. Un verdict high escalade toujours au-dessus des entrées « toute ».",
|
||||
"routing.tasks.tierAny": "toute complexité",
|
||||
"routing.tasks.addModel": "modèle",
|
||||
"routing.bands.enable": "Mes limites de complexité",
|
||||
"routing.bands.hint": "Faites glisser les poignées : où finit low et où commence high — elles se poussent l'une l'autre, les chiffres se mettent à jour en direct. Le score continu du routeur est re-découpé avec VOS limites ; décoché = les bandes du routeur.",
|
||||
"tier.low": "facile",
|
||||
"tier.medium": "moyen",
|
||||
"tier.high": "difficile",
|
||||
"bm.scoreHint": "Le score de complexité continu du routeur (0..1) — les curseurs de limites de la page Routage le découpent en niveaux.",
|
||||
"settings.cache.noEmbed": "Le cache est activé mais n'a AUCUN modèle d'embedding — il ne fait rien. Ajoutez au pool un modèle avec kind = \"embedding\" (p. ex. OpenAI text-embedding) ou choisissez-en un ci-dessous.",
|
||||
"settings.cache.entries": "entrées",
|
||||
"settings.cache.hits": "hits",
|
||||
"settings.cache.saved": "économisé",
|
||||
"keys.form.budget": "Budget mensuel, $ (0 = illimité)",
|
||||
"keys.form.budgetHint": "Quand les dépenses du mois dépassent le plafond, les requêtes sont rejetées avec 429 insufficient_quota.",
|
||||
"keys.budgetOf": "budget {b}/mois",
|
||||
"dash.kpi.orBalance": "Solde OpenRouter",
|
||||
"keys.h.install": "Installer :",
|
||||
"keys.snip.keyInputPh": "collez votre clé API",
|
||||
"keys.snip.apply": "Insérer la clé",
|
||||
};
|
||||
|
||||
Vendored
+257
-6
@@ -25,13 +25,13 @@ export default {
|
||||
"common.all": "все",
|
||||
|
||||
// nav
|
||||
"nav.dashboard": "Дашборд",
|
||||
"nav.dashboard": "Рабочий стол",
|
||||
"nav.models": "Модели",
|
||||
"nav.import": "Импорт модели",
|
||||
"nav.routing": "Роутинг",
|
||||
"nav.protocols": "Протоколы",
|
||||
"nav.keys": "API-ключи",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Песочница",
|
||||
"nav.settings": "Настройки",
|
||||
"nav.foot": "Шлюз · v{v}",
|
||||
"nav.billing": "Ключ роутера и оплата",
|
||||
@@ -56,8 +56,11 @@ export default {
|
||||
"login.failed": "Неверный токен администратора.",
|
||||
|
||||
// dashboard
|
||||
"dash.title": "Дашборд",
|
||||
"dash.title": "Рабочий стол",
|
||||
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
|
||||
"dash.clear": "Очистить логи",
|
||||
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
|
||||
"dash.cleared": "Логи очищены",
|
||||
"dash.range": "Период",
|
||||
"dash.range.1h": "1ч",
|
||||
"dash.range.24h": "24ч",
|
||||
@@ -77,7 +80,7 @@ export default {
|
||||
"dash.groupby.label": "по задаче",
|
||||
"dash.groupby.account": "по аккаунту",
|
||||
"dash.recent.title": "Последние запросы",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Playground.",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Песочницу.",
|
||||
"dash.col.time": "Время",
|
||||
"dash.col.model": "Модель",
|
||||
"dash.col.tier": "Уровень",
|
||||
@@ -128,12 +131,25 @@ export default {
|
||||
"models.form.secretMissing": "не задан",
|
||||
"models.form.secretPlaceholder": "задать или заменить ключ (только запись)",
|
||||
"models.deleteConfirm": "Удалить модель \"{id}\"?",
|
||||
"models.form.idOrModel": "Укажите id или имя модели",
|
||||
"models.form.apiKey": "API-ключ",
|
||||
"models.form.apiKeyPlaceholder": "вставьте API-ключ",
|
||||
"models.form.apiKeyNotNeeded": "не нужен (локальный сервер)",
|
||||
"models.form.apiKeyHint": "Хранится безопасно на шлюзе; больше не показывается.",
|
||||
"models.form.keyStored": "Ключ сохранён — оставьте пустым, чтобы не менять, или вставьте новый.",
|
||||
"models.form.keyFromEnv": "Используется ключ из переменной окружения.",
|
||||
"models.form.paramsTitle": "Параметры генерации, O(1) и режим размышления",
|
||||
"models.form.temperature": "Температура",
|
||||
"models.form.temperatureHint": "Температура генерации (0.0 — точные факты, 1.0+ — творчество).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Порог сэмплирования токенов (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Макс. токенов (Max Tokens)",
|
||||
"models.form.maxTokensHint": "Максимальное количество генерируемых токенов.",
|
||||
"models.form.thinkBudget": "Режим размышления / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Бюджет токенов мыслей (0 = выключен, N > 0 = лимит рассуждений).",
|
||||
"models.form.systemPrompt": "Системный промпт по умолчанию",
|
||||
"models.form.systemPromptHint": "Автоматически подставляется, если запрос не содержит системное сообщение.",
|
||||
"models.form.systemPromptPlaceholder": "Вы — полезный ассистент...",
|
||||
"models.form.testKey": "Проверить и загрузить модели",
|
||||
"models.form.testing": "проверка…",
|
||||
"models.form.foundModels": "загружено моделей: {n}",
|
||||
@@ -203,7 +219,7 @@ export default {
|
||||
"keys.allowAll": "все",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Песочница",
|
||||
"pg.subtitle": "Отправьте запрос через активный пайплайн и изучите решение роутера.",
|
||||
"pg.system": "Системный промпт",
|
||||
"pg.systemPlaceholder": "необязательное системное сообщение",
|
||||
@@ -314,6 +330,10 @@ export default {
|
||||
"settings.route.maxChars": "Макс. символов",
|
||||
"settings.route.cacheTtl": "TTL кэша",
|
||||
"settings.route.profile": "Профиль по умолчанию",
|
||||
"settings.route.strategyHelp": "Какой текст отправляется роутеру для классификации: last_user — только последнее сообщение пользователя; last_user_plus_system — последнее сообщение + системный промпт; concat_all — вся переписка.",
|
||||
"settings.route.maxCharsHelp": "Максимум символов этого текста, отправляемого роутеру (длиннее — обрезается). Больше = точнее классификация, но медленнее/дороже вызовы роутера.",
|
||||
"settings.route.cacheTtlHelp": "Сколько кэшируется решение роутинга для одинакового запроса, прежде чем роутер спрашивается снова (напр. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Профиль роутинга по умолчанию, если запрос его не задаёт: cost-saver (дешевле), balanced (баланс) или quality-first (качество). Влияет на выбор модели роутером.",
|
||||
"settings.breaker": "Автоматический выключатель",
|
||||
"settings.breaker.threshold": "Порог ошибок",
|
||||
"settings.breaker.cooldown": "Время восстановления",
|
||||
@@ -365,7 +385,7 @@ export default {
|
||||
"import.failed": "Конвертация не удалась — смотрите лог.",
|
||||
"import.register": "Подключить как локальную модель",
|
||||
"import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
|
||||
"import.jobs": "Конвертации",
|
||||
"import.jobs": "Загрузки / Конвертации",
|
||||
"import.cancel": "Отменить",
|
||||
"import.cancelling": "Отмена — удаляю недописанный файл…",
|
||||
"import.delete": "Удалить",
|
||||
@@ -375,12 +395,22 @@ export default {
|
||||
"import.state.done": "готово",
|
||||
"import.state.error": "ошибка",
|
||||
"import.state.cancelled": "отменено",
|
||||
"import.featuredTitle": "Рекомендуемые готовые .cmf модели",
|
||||
"import.featuredSub": "Готовые .cmf модели из аккаунта infosave на HuggingFace — список читается вживую, новые модели появляются здесь сами.",
|
||||
"import.readyCmfBadge": "Готовая .cmf",
|
||||
"import.directDownload": "Скачать",
|
||||
"import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
|
||||
"import.q.q8_row": "8-бит построчно — просто, надёжно.",
|
||||
"import.q.q4": "4-бит блочно — компактнее, ниже качество.",
|
||||
"import.q.q4_tiled": "4-бит тайловый (Q4_TILED) — х1.66 быстрее на Metal GPU.",
|
||||
"import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
|
||||
"import.q.q1t": "1.58-бит тернарный (Q1T) — BitNet b1.58 с оутлайерами.",
|
||||
"import.q.q1s": "1-бит спарс (Q1S) — 1-бит база с голографическими оутлайерами.",
|
||||
"import.q.q1p": "1-бит PTQ (Q1P) — 1-бит с переносом ошибки (error diffusion).",
|
||||
"import.q.q1": "1-бит (Q1) — чистый бинарный формат.",
|
||||
"import.q.f16": "16-бит — без квантования.",
|
||||
"import.q.f32": "32-бит — полная точность (большой файл).",
|
||||
|
||||
"import.adv.title": "Расширенные настройки",
|
||||
"import.adv.auto": "авто",
|
||||
"import.adv.linearCore": "Линейное ядро",
|
||||
@@ -391,4 +421,225 @@ export default {
|
||||
"import.adv.shard": "Шард макс (ГБ)",
|
||||
"import.adv.noShard": "без шардинга",
|
||||
"import.adv.skipMtp": "пропустить MTP-голову",
|
||||
"import.adv.o1": "O(1)-внимание",
|
||||
"import.adv.o1Hint": "Потоковое внимание с постоянной памятью: KV-кэш освобождается после prefill, длинный контекст не раздувает память. Веса не меняются — записывается лишь подсказка на загрузку; вернуться можно всегда (запуск с --o1 off). Проверено на Qwen3-0.6B: все слои ≈ ×1.18 перплексии.",
|
||||
"import.adv.o1Off": "выкл — точное внимание (по умолчанию)",
|
||||
"import.adv.o1All": "all — конвертировать все softmax-слои",
|
||||
"import.adv.o1Deep": "deep N — только N самых глубоких слоёв",
|
||||
"import.adv.o1Custom": "вручную — явный список слоёв",
|
||||
"import.adv.o1DeepN": "N глубоких слоёв",
|
||||
"import.adv.o1Layers": "Слои (через запятую)",
|
||||
"import.adv.o1mHint": "число опорных точек (по умолчанию 32)",
|
||||
"import.adv.o1windowHint": "ширина точного окна (по умолчанию 128)",
|
||||
"import.adv.o1sinkHint": "постоянные точные sink-ключи (по умолчанию 4)",
|
||||
|
||||
// онбординг (первый запуск)
|
||||
"onboarding.title": "Первоначальная настройка",
|
||||
"onboarding.welcome": "Добро пожаловать в Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Выберите язык — его можно сменить в шапке в любой момент.",
|
||||
"onboarding.next": "Далее",
|
||||
"onboarding.back": "Назад",
|
||||
"onboarding.skip": "Пропустить",
|
||||
"onboarding.skipSetup": "Пропустить",
|
||||
"onboarding.finish": "Завершить",
|
||||
"onboarding.featuredTitle": "Готовые модели в один клик",
|
||||
"onboarding.featuredSub": "Это уже сконвертированные .cmf модели от infosave на HuggingFace — просто Скачать, без конвертации. Выберите одну или несколько для мгновенного старта.",
|
||||
"onboarding.download": "Скачать",
|
||||
"onboarding.hfTitle": "Любая модель с HuggingFace — автоконвертация",
|
||||
"onboarding.hfSub": "Найдите любую модель, выберите квантизацию — мы автоматически сконвертируем её в .cmf. Никаких ручных правок конфига.",
|
||||
"onboarding.noJobs": "Пока нет загрузок.",
|
||||
"onboarding.doneTitle": "Готово",
|
||||
"onboarding.doneSub": "Скачанные модели зарегистрируются автоматически. Добавить ещё можно во вкладке Импорт или Модели.",
|
||||
"onboarding.doneHint": "Кнопка «Завершить» подключит все скачанные модели как локальные, поднимет для них .cmf сервер и настроит роутинг. Всё это потом можно изменить во вкладке Модели.",
|
||||
"onboarding.willRegister": "Будут подключены как локальные модели",
|
||||
"onboarding.nothingToRegister": "Завершённых загрузок пока нет — подключать нечего. Вернитесь на шаг назад и выберите модель.",
|
||||
"onboarding.registeredN": "Подключено моделей: {n} — {ids}",
|
||||
"onboarding.runtimeMissing": "Среда cortiq не установлена — установите её в Настройки → Локальные модели, чтобы запускать эти модели.",
|
||||
"onboarding.downloaded": "скачана",
|
||||
"onboarding.update": "Обновить",
|
||||
"onboarding.deleteFile": "Удалить файл .cmf с диска",
|
||||
"onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.",
|
||||
"onboarding.unregistered": "Также отключено из моделей: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Чат",
|
||||
"pg.tab.image": "Изображение",
|
||||
"pg.tab.video": "Видео",
|
||||
"pg.generate": "Сгенерировать",
|
||||
"pg.generating": "Генерация…",
|
||||
"pg.genFailed": "Генерация не удалась — смотрите лог.",
|
||||
"pg.result": "Результат",
|
||||
"pg.genMetrics": "Метрики генерации",
|
||||
"pg.imagePromptPlaceholder": "рыжая лиса в снегу на закате, фотореалистично, детальный мех",
|
||||
"pg.videoPromptPlaceholder": "корги в поварском колпаке подбрасывает блин, шипение сковороды и весёлый лай",
|
||||
"pg.imageEmpty": "Сгенерированное изображение появится здесь.",
|
||||
"pg.videoEmpty": "Сгенерированный ролик (со звуком) появится здесь.",
|
||||
"pg.width": "Ширина",
|
||||
"pg.height": "Высота",
|
||||
"pg.steps": "Шаги",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Случайный seed",
|
||||
"pg.cfgHint": "Сила guidance; ≤ 1 отключает CFG и вдвое ускоряет генерацию.",
|
||||
"pg.frames": "Кадры",
|
||||
"pg.framesHint": "При 24 fps; округляется до сетки модели.",
|
||||
"pg.quality": "Качество JPEG",
|
||||
"pg.videoSizeHint": "Кратно 32; обученная короткая сторона — 768.",
|
||||
"pg.videoStepsHint": "Turbo LoRA обучена на 4 шага.",
|
||||
"pg.firstFrame": "Первый кадр (необязательно)",
|
||||
"pg.lastFrame": "Последний кадр (необязательно)",
|
||||
"pg.keyframeHint": "Превращает запуск в keyframe-to-video.",
|
||||
"pg.resolution": "Разрешение",
|
||||
"pg.genTime": "Время генерации",
|
||||
"pg.timePerStep": "На шаг",
|
||||
"pg.timePerFrame": "На кадр",
|
||||
"pg.fileSize": "Размер файла",
|
||||
"pg.duration": "Длительность",
|
||||
"pg.play": "Играть",
|
||||
"pg.noMediaModels": "Локальной модели ({kind}) ещё нет. Скачайте её (например, из коллекции infosave) во вкладке Импорт:",
|
||||
"pg.tokSpeed": "Скорость",
|
||||
"pg.ttft": "Первый токен",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Логи",
|
||||
"logs.subtitle": "Последние записи журнала шлюза — то, что показал бы терминал.",
|
||||
"logs.filterPlaceholder": "фильтр…",
|
||||
"logs.pause": "Пауза",
|
||||
"logs.resume": "Продолжить",
|
||||
"logs.empty": "Ничего не найдено.",
|
||||
"notice.openAdmin": "Панель управления открыта без токена и доступна с других машин. Задайте токен, если эта машина не приватная.",
|
||||
"notice.update": "Доступно обновление: v{v}.",
|
||||
"dash.sys.title": "Система",
|
||||
"dash.sys.cores": "ядер",
|
||||
"dash.sys.disk": "Диск (модели)",
|
||||
"dash.sys.free": "свободно",
|
||||
"dash.sys.gateway": "шлюз",
|
||||
"fit.disk": "не хватит места",
|
||||
"fit.diskTitle": "Загрузка больше свободного места на диске с моделями.",
|
||||
"fit.ram": "может не влезть в RAM",
|
||||
"fit.ramTitle": "Модель сравнима с объёмом RAM этой машины или больше — может не запуститься.",
|
||||
"pg.eta": "осталось",
|
||||
"pg.min": "мин",
|
||||
"pg.queued": "В очереди — позиция {n}. Стартует, как только закончится текущая генерация.",
|
||||
"pg.gallery": "Последние генерации",
|
||||
"pg.galDeleteConfirm": "Удалить эту генерацию и её файлы?",
|
||||
"pg.clearChat": "Очистить",
|
||||
"keys.snip.title": "Подключите ваше приложение",
|
||||
"keys.snip.hint": "Вставьте один из примеров в свой проект — заменить нужно только плейсхолдер ключа.",
|
||||
"keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
|
||||
"keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
|
||||
"keys.snip.keyPh": "ваш API-ключ",
|
||||
"settings.restart": "Перезапустить всё",
|
||||
"settings.restartConfirm": "Перезапустить шлюз и все серверы локальных моделей? Текущие запросы оборвутся.",
|
||||
"settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
|
||||
"settings.shutdown": "Остановить всё и выключить",
|
||||
"settings.shutdownConfirm": "Остановить все серверы локальных моделей и выключить шлюз? API и эта панель перестанут отвечать, пока вы не запустите шлюз заново.",
|
||||
"settings.shutdownDone": "Шлюз выключен.",
|
||||
"settings.shutdownHint": "Все серверы локальных моделей остановлены, процесс завершён. Запустите шлюз на сервере заново, чтобы вернуть панель.",
|
||||
"bench.title": "Скорость локальных моделей",
|
||||
"bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
|
||||
"bench.run": "Замерить",
|
||||
"bench.running": "замер…",
|
||||
"bench.failed": "ошибка",
|
||||
"bench.none": "Локальных чат-моделей .cmf пока нет.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Светская беседа",
|
||||
"task.code": "Код",
|
||||
"task.creative-writing": "Творческие тексты",
|
||||
"task.extraction": "Извлечение данных",
|
||||
"task.math": "Математика",
|
||||
"task.qa": "Вопросы и ответы",
|
||||
"task.summarization": "Суммаризация",
|
||||
"task.translation": "Перевод",
|
||||
"routing.tasks.title": "Распределение по задачам",
|
||||
"routing.tasks.hint": "Модели правила идут первыми для своей задачи; цепочка по сложности остаётся резервом. Пусто = только по сложности.",
|
||||
"routing.tasks.inherit": "по сложности (без правила)",
|
||||
"routing.tasks.share": "{p}% запросов (7д)",
|
||||
"routing.tasks.fromRouter": "метки получены с роутера",
|
||||
"routing.tasks.builtin": "роутер недоступен — встроенный набор меток",
|
||||
"routing.tasks.addCustom": "Добавить задачу",
|
||||
"routing.tasks.addPlaceholder": "своя метка…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Бенчмарк",
|
||||
"bm.title": "Бенчмарк",
|
||||
"bm.subtitle": "Проверки качества и скорости через живой пайплайн: ответы сверяются по ключевым словам, ловится зацикливание, замеряются tok/s и время первого токена.",
|
||||
"bm.models": "Какие модели тестировать",
|
||||
"bm.quickMode": "Быстрый тест",
|
||||
"bm.quickHint": "Короткое подмножество с краткими ответами — около минуты на локальную модель.",
|
||||
"bm.suiteInfo": "Встроенный набор: {q} быстрых / {f} полных тестов, плюс ваши собственные.",
|
||||
"bm.run": "Запустить",
|
||||
"bm.pickModel": "Выберите хотя бы одну модель.",
|
||||
"bm.testOf": "тест {i} из {n}",
|
||||
"bm.results": "Результаты",
|
||||
"bm.quick": "быстрый",
|
||||
"bm.test": "Тест",
|
||||
"bm.cellHint": "Клик по ячейке — полный ответ модели.",
|
||||
"bm.history": "Прошлые прогоны (хранятся в этом браузере)",
|
||||
"bm.custom": "Свои тесты",
|
||||
"bm.customHint": "Промпт + ключевые слова через запятую, которые должны быть в ответе (| — синонимы). Хранятся в этом браузере.",
|
||||
"bm.promptPh": "промпт…",
|
||||
"bm.checkPh": "ключевые слова через запятую",
|
||||
"bm.addTest": "Добавить тест",
|
||||
"bm.emptyAnswer": "пустой ответ",
|
||||
"bm.loopDetected": "зацикливание",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Умный роутинг",
|
||||
"bm.mode.models": "Отдельные модели",
|
||||
"bm.mode.routingDesc": "Проверка главной фишки продукта: каждый тест идёт через cortiq-auto — видно, какую задачу распознал роутер, какую модель выбрал, совпало ли это с ожиданием и сколько это стоило.",
|
||||
"bm.mode.modelsDesc": "Тот же набор тестов, но закреплённый за выбранными моделями — сравнение их качества, скорости и стоимости напрямую, мимо роутера.",
|
||||
"bm.answersOk": "ответы верны",
|
||||
"bm.classAcc": "классификация задач",
|
||||
"bm.modelDist": "Запросы ушли в",
|
||||
"bm.expectedLbl": "ожидалось:",
|
||||
"bm.answer": "Ответ",
|
||||
"bm.liveRunning": "идёт…",
|
||||
"bm.spent": "потрачено",
|
||||
"bm.verdict": "🏁 Готово: ответы {p}, потрачено {s}",
|
||||
"bm.clearHistory": "Очистить всё",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — вставьте в терминал (шлюз говорит на протоколе Anthropic нативно):",
|
||||
"keys.h.codex": "Codex — добавит провайдера в ~/.codex/config.toml и запустит:",
|
||||
"keys.h.opencode": "OpenCode — запишет ~/.config/opencode/opencode.json (перезапишет!) и запустит:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — добавит модель в ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline (VS Code) — в настройках расширения укажите:",
|
||||
"keys.h.cline2": "Настройки → API Provider → OpenAI Compatible, затем вставьте значения выше.",
|
||||
"keys.h.note": "cortiq-auto — умный роутинг: роутер сам выбирает модель под каждый запрос. Чтобы закрепить конкретную, укажите её точное имя. На Windows запускайте скрипты в Git Bash / WSL или задайте те же переменные через настройки окружения системы.",
|
||||
"keys.h.protoOff": "Протокол Anthropic (/v1/messages) выключен — Claude Code не подключится.",
|
||||
"keys.h.enableProto": "Включить",
|
||||
"keys.h.st.installed": "Установлен на машине шлюза",
|
||||
"keys.h.st.configured": "Уже настроен на этот шлюз",
|
||||
"keys.h.st.notFound": "Не найден на машине шлюза",
|
||||
"bm.degraded": "роутер деградировал",
|
||||
"bm.routerDegraded": "Роутер не успел ответить — запрос ушёл по резервной цепочке; классификация не засчитывается.",
|
||||
"bm.cutByCap": "бюджет ушёл на размышления — ответ не успел",
|
||||
"shell.uiStale": "Шлюз обновился — на этой странице старый интерфейс.",
|
||||
"shell.reload": "Перезагрузить",
|
||||
"keys.col.requests": "Запросы",
|
||||
"keys.col.tokens": "Токены вх/исх",
|
||||
"keys.col.costAll": "$ за всё время",
|
||||
"routing.tasks.tier": "Сложность",
|
||||
"routing.tasks.tierAuto": "Авто (по роутеру)",
|
||||
"routing.tasks.tierHint": "Какая модель обслуживает задачу на какой сложности; «любая» ведёт на всех уровнях. Порядок = порядок фолбэка. Вердикт high по-прежнему эскалирует поверх «любых».",
|
||||
"routing.tasks.tierAny": "любая сложность",
|
||||
"routing.tasks.addModel": "модель",
|
||||
"routing.bands.enable": "Мои границы сложности",
|
||||
"routing.bands.hint": "Тяните ручки: где кончается low и где начинается high — ручки толкают друг друга, цифры обновляются на лету. Непрерывный score роутера пересчитывается ВАШИМИ границами; галочка снята — действуют пороги самого роутера.",
|
||||
"tier.low": "лёгкий",
|
||||
"tier.medium": "средний",
|
||||
"tier.high": "сложный",
|
||||
"bm.scoreHint": "Непрерывный коэффициент сложности роутера (0..1) — ползунки границ на вкладке Роутинг нарезают его на уровни.",
|
||||
"settings.cache.noEmbed": "Кэш включён, но embedding-модели нет — он ничего не делает. Добавьте в пул модель с kind = \"embedding\" (например, OpenAI text-embedding) или выберите её ниже.",
|
||||
"settings.cache.entries": "записей",
|
||||
"settings.cache.hits": "попаданий",
|
||||
"settings.cache.saved": "сэкономлено",
|
||||
"keys.form.budget": "Бюджет в месяц, $ (0 = без лимита)",
|
||||
"keys.form.budgetHint": "Когда траты аккаунта за месяц превысят потолок, запросы отклоняются с 429 insufficient_quota.",
|
||||
"keys.budgetOf": "бюджет {b}/мес",
|
||||
"dash.kpi.orBalance": "Баланс OpenRouter",
|
||||
"keys.h.install": "Установить:",
|
||||
"keys.snip.keyInputPh": "вставьте ваш API-ключ",
|
||||
"keys.snip.apply": "Вставить ключ",
|
||||
};
|
||||
|
||||
Vendored
+370
@@ -280,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "Giriş TTL",
|
||||
"settings.cache.embed": "Gömme modeli",
|
||||
"settings.cache.note": "Anlamsal önbellek ayarları yeniden başlatmadan sonra geçerli olur.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Model içe aktar",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Günlükleri temizle",
|
||||
"dash.clearConfirm": "Kaydedilmiş tüm istatistikler ve günlükler silinsin mi? Bu işlem geri alınamaz.",
|
||||
"dash.cleared": "Günlükler temizlendi",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Bir id ya da model adı girin",
|
||||
"models.form.apiKey": "API anahtarı",
|
||||
"models.form.apiKeyPlaceholder": "API anahtarınızı yapıştırın",
|
||||
"models.form.apiKeyNotNeeded": "gerekmiyor (yerel sunucu)",
|
||||
"models.form.apiKeyHint": "Ağ geçidinde güvenle saklanır; bir daha gösterilmez.",
|
||||
"models.form.keyStored": "Kayıtlı bir anahtar var — korumak için boş bırakın ya da yenisini yapıştırın.",
|
||||
"models.form.keyFromEnv": "Ortam değişkenindeki anahtar kullanılıyor.",
|
||||
"models.form.paramsTitle": "Üretim ve düşünme parametreleri (O(1), MTP, sıcaklık)",
|
||||
"models.form.temperature": "Sıcaklık",
|
||||
"models.form.temperatureHint": "Üretim sıcaklığı (0.0 = belirlenimci, 1.0+ = yaratıcı).",
|
||||
"models.form.topP": "Top-P (nucleus örnekleme)",
|
||||
"models.form.topPHint": "Örnekleme eşiği (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Maks. token",
|
||||
"models.form.maxTokensHint": "Üretilecek en fazla token sayısı.",
|
||||
"models.form.thinkBudget": "Düşünme modu / düşünme bütçesi",
|
||||
"models.form.thinkBudgetHint": "Akıl yürütme token bütçesi (0 = kapalı, N > 0 = en fazla akıl yürütme tokeni).",
|
||||
"models.form.systemPrompt": "Varsayılan sistem istemi",
|
||||
"models.form.systemPromptHint": "İstekte sistem istemi yoksa otomatik olarak başa eklenir.",
|
||||
"models.form.systemPromptPlaceholder": "Yardımsever bir asistansın…",
|
||||
"models.form.testKey": "Test et ve modelleri yükle",
|
||||
"models.form.testing": "kontrol ediliyor…",
|
||||
"models.form.foundModels": "{n} model yüklendi",
|
||||
"models.form.modelPlaceholder": "aramak için yazın…",
|
||||
"models.form.modelHint": "Anahtarınızla listeyi çekmek için “Test et ve modelleri yükle”ye tıklayın, sonra birini seçin.",
|
||||
"models.managed": "yönetilen",
|
||||
"models.managedHint": "Yönetilen CMF sunucusu tarafından sağlanır (Ayarlar → Yerel modeller). Yapılandırmayı orada yapın.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Yönlendiriciyi kullan (kapalı = yalnızca yerel modeller)",
|
||||
"settings.cmf": "Yerel modeller (CMF)",
|
||||
"settings.cmf.localOnly": "Yalnızca yerel modelleri kullan",
|
||||
"settings.cmf.manage": "Yerel model sunucusu çalıştır",
|
||||
"settings.cmf.autoInstall": "cortiq'i otomatik kur (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq'i otomatik güncelle",
|
||||
"settings.cmf.model": "Yerel .cmf model yolu",
|
||||
"settings.cmf.port": "Yerel sunucu portu",
|
||||
"settings.cmf.modelId": "Havuzdaki model id'si",
|
||||
"settings.cmf.note": "Yerel bir .cmf dosyasını cortiq ile sunar (github.com/infosave2007/cmf). Yönlendirici erişilemez olsa bile çalışır.",
|
||||
"settings.cmf.threads": "İş parçacıkları",
|
||||
"settings.cmf.gpu": "GPU kullan (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "İş parçacıkları çözme aşamasındaki matris-vektör çarpımlarını paralelleştirir (~3× hızlı; 0 = çalışma zamanı varsayılanı). GPU'ya aktarım daha çok uzun istemlerin prefill aşamasına yarar — çözme bellek bant genişliğine bağlı olduğundan Apple Silicon'da etkisi yaklaşık nötrdür.",
|
||||
"settings.cmf.pick": "bir model seçin",
|
||||
"settings.cmf.models": "Yerel modeller",
|
||||
"settings.cmf.modelsHelp": "Her satır, ağ geçidinin çalıştırdığı (kendi portunda kendi cortiq serve süreci olan) ve havuza kaydettiği bir modeldir. Kaç .cmf dosyanız ve boş portunuz varsa o kadar ekleyebilirsiniz.",
|
||||
"settings.cmf.rowHint": "id · .cmf dosyası · port · iş parçacığı · GPU. Değişiklikler yeniden başlatınca geçerli olur; satırı silmek o sunucuyu durdurur.",
|
||||
"settings.cmf.addServer": "+ Yerel model ekle",
|
||||
"settings.cmf.checkPort": "Portu kontrol et",
|
||||
"settings.cmf.portFree": "port boş",
|
||||
"settings.cmf.portBusy": "port kullanımda",
|
||||
"settings.cmf.useSuggested": "{port} kullan",
|
||||
"settings.cmf.serving": "{model} :{port} üzerinde sunuluyor",
|
||||
"settings.cmf.intro": "Modelleri cortiq motoruyla yerelde çalıştırın — çevrimdışıyken ya da yönlendirici çöktüğünde bile çalışır.",
|
||||
"settings.cmf.runtime": "cortiq çalışma zamanı",
|
||||
"settings.cmf.runtimeHelp": ".cmf modellerini yükleyip çalıştıran `cortiq` komut satırı aracı. Yerel sunucuyu başlatmadan önce bir kez kurun.",
|
||||
"settings.cmf.install": "cortiq'i kur",
|
||||
"settings.cmf.reinstall": "Yeniden kur / güncelle",
|
||||
"settings.cmf.installing": "kuruluyor… (derleniyor, birkaç dakika)",
|
||||
"settings.cmf.installedOk": "cortiq kuruldu",
|
||||
"settings.cmf.notInstalled": "kurulu değil",
|
||||
"settings.cmf.addModel": "Model ekle",
|
||||
"settings.cmf.addModelHelp": "Modeller “Model içe aktar” sekmesinde indirilip dönüştürülür (HuggingFace → .cmf). Sonra aşağıdaki yerel sunucuyu o .cmf dosyasına yönlendirin.",
|
||||
"settings.cmf.importNote": "Henüz model yok mu? Bir tane içe aktarın:",
|
||||
"settings.cmf.importLink": "İçe aktarmayı aç",
|
||||
"settings.cmf.modelHelp": "Sunulacak .cmf dosyasının yolu (ör. models/qwen.cmf), içe aktarma sekmesinde üretilir. Farklı görevler için birden çok model mi? Hepsini içe aktarıp bu yolu değiştirin.",
|
||||
"settings.cmf.portHelp": "Yerel model sunucusunun dinlediği TCP portu (varsayılan 8081). Yalnızca başka bir servisle çakışırsa değiştirin.",
|
||||
"settings.cmf.threadsHelp": "Üretim için kaç CPU çekirdeği kullanılacağı. Daha fazlası = daha hızlı (tek çekirdeğe göre ~3×); 0 seçimi çalışma zamanına bırakır.",
|
||||
"settings.cmf.gpuHelp": "Matris hesaplarını GPU'ya aktarır (Metal/wgpu). Apple Silicon'da çoğunlukla uzun istemleri hızlandırır — çözme bellek bant genişliğine bağlıdır.",
|
||||
"settings.cmf.localOnlyHelp": "Her isteği yerel modele yönlendirir; yönlendiriciyi ve tüm bulut sağlayıcılarını yok sayar.",
|
||||
"settings.cmf.manageHelp": "Ağ geçidi açılışta aşağıdaki .cmf için `cortiq serve` başlatır ve istekleri oraya yönlendirir. cortiq'in kurulu olması gerekir (adım 1). Yeniden başlatınca geçerli olur.",
|
||||
"settings.cmf.autoInstallHelp": "Açılışta cortiq yoksa crates.io üzerinden otomatik kurar.",
|
||||
"settings.cmf.autoUpdateHelp": "Açılışta daha yeni bir sürüm varsa cortiq'i en son sürüme yükseltir.",
|
||||
"settings.router.enabledHelp": "Her isteği cortiq yönlendiricisi sınıflandırsın (görev türü + karmaşıklık) ve model seçsin. Kapalı = yalnızca yerel yönlendirme tablosunu kullan.",
|
||||
"settings.router.verifyTlsHelp": "Yönlendiricinin TLS sertifikasını doğrula. Yalnızca kendinden imzalı ya da IP'ye bağlı sertifikada kapatın.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus metriklerini GET /metrics adresinde yayınla.",
|
||||
"settings.cortiq.echoHelp": "Her yanıt gövdesine `cortiq` alanı ekle (görev, model, maliyet). Yanıt başlıkları her zaman gönderilir.",
|
||||
"settings.stats.enabledHelp": "Panoyu ve grafikleri beslemek için son istekleri kaydet.",
|
||||
"settings.cache.enabledHelp": "Anlamsal önbellek: yeni bir istem geçmiştekine çok benziyorsa kaydedilmiş yanıtı döndürür ve model çağrısını atlar. Havuzda bir gömme (embedding) modeli gerekir.",
|
||||
"settings.log.bodiesHelp": "İstek/yanıt mesaj gövdelerinin tamamını günlüğe yaz. Gizlilik için varsayılan olarak kapalı.",
|
||||
"settings.router.saveKey": "Anahtarı kaydet",
|
||||
"settings.router.keySaved": "Yönlendirici anahtarı kaydedildi",
|
||||
"settings.router.keyEmpty": "Önce anahtarı yapıştırın",
|
||||
"settings.router.secretHelp": "api.allaigate.com e-postanızdaki anahtarı yapıştırıp Anahtarı kaydet'e basın. Bu bir “API anahtarları” değeri DEĞİLDİR — onlar ağ geçidinizi çağıran istemciler içindir.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — anahtar kayıtlı (değiştirmek için yenisini yapıştırın)",
|
||||
"settings.route.strategyHelp": "Sınıflandırma için yönlendiriciye gönderilen metin: last_user = yalnızca son kullanıcı mesajı; last_user_plus_system = son kullanıcı mesajı + sistem istemi; concat_all = tüm konuşma.",
|
||||
"settings.route.maxCharsHelp": "Bu metnin yönlendiriciye gönderilen en fazla karakter sayısı (fazlası kırpılır). Daha fazlası = daha isabetli sınıflandırma, ama daha yavaş ve pahalı yönlendirici çağrıları.",
|
||||
"settings.route.cacheTtlHelp": "Aynı istek için yönlendirme kararının, yönlendiriciye yeniden sorulmadan önce ne kadar önbellekte tutulacağı (ör. 60s, 5m).",
|
||||
"settings.route.profileHelp": "İstek belirtmediğinde kullanılacak varsayılan yönlendirme profili: cost-saver (en ucuz), balanced ya da quality-first (en iyi model). Yönlendiricinin hangi modeli seçeceğini etkiler.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "HuggingFace'te arayın, yerel bir .cmf dosyasına dönüştürün ve sunun.",
|
||||
"import.searchPlaceholder": "HuggingFace modellerinde ara (ör. qwen3, llama)…",
|
||||
"import.searching": "Aranıyor…",
|
||||
"import.noResults": "Model bulunamadı.",
|
||||
"import.configure": "Dönüştürmeyi yapılandır",
|
||||
"import.outName": "Çıktı adı",
|
||||
"import.outNameHint": ".cmf dosya adı (harf, rakam, - ve _).",
|
||||
"import.quant": "Niceleme",
|
||||
"import.convert": "Dönüştür ve yükle",
|
||||
"import.converting": "Dönüştürülüyor",
|
||||
"import.started": "Dönüştürme başladı",
|
||||
"import.failed": "Dönüştürme başarısız — günlüğe bakın.",
|
||||
"import.register": "Yerel model olarak kaydet",
|
||||
"import.registered": "“{id}” olarak kaydedildi. Sunmak için cortiq-server'ı çalıştırın.",
|
||||
"import.jobs": "İndirmeler ve dönüştürmeler",
|
||||
"import.cancel": "İptal",
|
||||
"import.cancelling": "İptal ediliyor — yarım kalan dosya temizleniyor…",
|
||||
"import.delete": "Sil",
|
||||
"import.deleteConfirm": "Bu model ve dönüştürülmüş .cmf dosyası diskten silinsin mi?",
|
||||
"import.deleted": "Dosyalarıyla birlikte silindi",
|
||||
"import.state.running": "çalışıyor",
|
||||
"import.state.done": "tamam",
|
||||
"import.state.error": "hata",
|
||||
"import.state.cancelled": "iptal edildi",
|
||||
"import.featuredTitle": "Öne çıkan hazır .cmf modelleri",
|
||||
"import.featuredSub": "HuggingFace'teki infosave hesabından hazır .cmf modelleri — liste canlı okunur, yeni modeller kendiliğinden görünür.",
|
||||
"import.readyCmfBadge": "Hazır .cmf",
|
||||
"import.directDownload": "İndir",
|
||||
"import.q.q8_2f": "8 bit çift alan (𝒲×θ) — önerilen: en iyi kalite/boyut dengesi.",
|
||||
"import.q.q8_row": "Satır başına 8 bit — basit ve sağlam.",
|
||||
"import.q.q4": "4 bit blok — en küçük, kalite daha düşük.",
|
||||
"import.q.q4_tiled": "4 bit döşeli (Q4_TILED) — Metal GPU'da 1,66× daha hızlı.",
|
||||
"import.q.vbit": "Değişken 3–8 bit — uzman başına bit bütçesi.",
|
||||
"import.q.q1t": "1,58 bit üçlü (Q1T) — seyrek aykırı değerlerle BitNet b1.58.",
|
||||
"import.q.q1s": "1 bit seyrek (Q1S) — holografik aykırı değerlerle 1 bit taban.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — hata yayılımlı 1 bit.",
|
||||
"import.q.q1": "1 bit (Q1) — saf ikili biçim.",
|
||||
"import.q.f16": "16 bit — niceleme yok.",
|
||||
"import.q.f32": "32 bit — tam hassasiyet (büyük dosya).",
|
||||
"import.adv.title": "Gelişmiş ayarlar",
|
||||
"import.adv.auto": "otomatik",
|
||||
"import.adv.linearCore": "Doğrusal çekirdek",
|
||||
"import.adv.linearHint": "Doğrusal dikkat katmanlarının nasıl taşındığı/katlandığı.",
|
||||
"import.adv.nphase": "vmf_phase faz/başlık",
|
||||
"import.adv.vbitShape": "VBIT bütçe biçimi",
|
||||
"import.adv.meanBits": "VBIT ortalama bit",
|
||||
"import.adv.shard": "Parça üst sınırı (GB)",
|
||||
"import.adv.noShard": "parçalama yok",
|
||||
"import.adv.skipMtp": "MTP başlığını atla",
|
||||
"import.adv.o1": "O(1) dikkat",
|
||||
"import.adv.o1Hint": "Sabit bellekli akış dikkati: KV önbelleği prefill sonrası serbest bırakılır, böylece uzun bağlamlar belleği büyütmez. Ağırlıklar bayt bayt aynı kalır — yalnızca yükleme anına dair bir ipucu kaydedilir; istediğiniz zaman geri dönebilirsiniz (--o1 off ile sunun). Qwen3-0.6B üzerinde doğrulandı: tüm katmanlar ≈ ×1,18 şaşkınlık (perplexity).",
|
||||
"import.adv.o1Off": "kapalı — tam dikkat (varsayılan)",
|
||||
"import.adv.o1All": "all — her softmax katmanını dönüştür",
|
||||
"import.adv.o1Deep": "deep N — yalnızca en derin N katman",
|
||||
"import.adv.o1Custom": "özel — açık katman listesi",
|
||||
"import.adv.o1DeepN": "N derin katman",
|
||||
"import.adv.o1Layers": "Katmanlar (virgülle ayrılmış)",
|
||||
"import.adv.o1mHint": "landmark bütçesi (varsayılan 32)",
|
||||
"import.adv.o1windowHint": "tam pencere genişliği (varsayılan 128)",
|
||||
"import.adv.o1sinkHint": "kalıcı tam sink anahtarları (varsayılan 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "İlk kurulum",
|
||||
"onboarding.welcome": "Cortiq Gateway'e hoş geldiniz",
|
||||
"onboarding.welcomeSub": "Dilinizi seçin — üst çubuktan istediğiniz zaman değiştirebilirsiniz.",
|
||||
"onboarding.next": "İleri",
|
||||
"onboarding.back": "Geri",
|
||||
"onboarding.skip": "Şimdilik atla",
|
||||
"onboarding.skipSetup": "Atla",
|
||||
"onboarding.finish": "Kurulumu tamamla",
|
||||
"onboarding.featuredTitle": "Hazır modeller (tek tık)",
|
||||
"onboarding.featuredSub": "Bunlar HuggingFace'te infosave tarafından yayınlanan, önceden dönüştürülmüş .cmf modelleridir — sadece indirin, dönüştürme gerekmez. Hemen başlamak için bir ya da birkaçını seçin.",
|
||||
"onboarding.download": "İndir",
|
||||
"onboarding.hfTitle": "Herhangi bir HuggingFace modeli — otomatik dönüştürme",
|
||||
"onboarding.hfSub": "İstediğiniz modeli arayın, bir niceleme seçin; biz onu otomatik olarak .cmf biçimine dönüştürelim. Elle yapılandırma düzenlemesi yok.",
|
||||
"onboarding.noJobs": "Henüz indirme yok.",
|
||||
"onboarding.doneTitle": "Her şey hazır",
|
||||
"onboarding.doneSub": "İndirdiğiniz modeller otomatik olarak kaydedilir. Daha fazlasını İçe aktar ya da Modeller sekmesinden ekleyebilirsiniz.",
|
||||
"onboarding.doneHint": "“Kurulumu tamamla”, biten her indirmeyi yerel model olarak bağlar, .cmf sunucusunu başlatır ve yönlendirmeyi ona çevirir. Tümünü sonradan Modeller sekmesinden değiştirebilirsiniz.",
|
||||
"onboarding.willRegister": "Yerel model olarak bağlanacak",
|
||||
"onboarding.nothingToRegister": "Henüz tamamlanmış indirme yok — bağlanacak bir şey bulunmuyor. Bir adım geri dönüp model seçin.",
|
||||
"onboarding.registeredN": "{n} model bağlandı: {ids}",
|
||||
"onboarding.runtimeMissing": "cortiq çalışma zamanı kurulu değil — bunları sunabilmek için Ayarlar → Yerel modeller bölümünden kurun.",
|
||||
"onboarding.downloaded": "indirildi",
|
||||
"onboarding.update": "Güncelle",
|
||||
"onboarding.deleteFile": ".cmf dosyasını diskten sil",
|
||||
"onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.",
|
||||
"onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "Sohbet",
|
||||
"pg.tab.image": "Görsel",
|
||||
"pg.tab.video": "Video",
|
||||
"pg.generate": "Oluştur",
|
||||
"pg.generating": "Oluşturuluyor…",
|
||||
"pg.genFailed": "Oluşturma başarısız — günlüğe bakın.",
|
||||
"pg.result": "Sonuç",
|
||||
"pg.genMetrics": "Oluşturma metrikleri",
|
||||
"pg.imagePromptPlaceholder": "gün batımında karda oturan kızıl tilki, fotogerçekçi, ayrıntılı tüyler",
|
||||
"pg.videoPromptPlaceholder": "aşçı şapkalı bir corgi krep çeviriyor, cızırtı sesleri ve neşeli bir havlama",
|
||||
"pg.imageEmpty": "Oluşturulan görsel burada görünecek.",
|
||||
"pg.videoEmpty": "Oluşturulan klip (sesiyle) burada görünecek.",
|
||||
"pg.width": "Genişlik",
|
||||
"pg.height": "Yükseklik",
|
||||
"pg.steps": "Adım",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "Rastgele seed",
|
||||
"pg.cfgHint": "Guidance gücü; ≤ 1 CFG'yi kapatır ve işi yarıya indirir.",
|
||||
"pg.frames": "Kare",
|
||||
"pg.framesHint": "24 fps'te; modelin ızgarasına yuvarlanır.",
|
||||
"pg.quality": "JPEG kalitesi",
|
||||
"pg.videoSizeHint": "32'nin katları; eğitilen kısa kenar 768.",
|
||||
"pg.videoStepsHint": "Turbo LoRA 4 adım için eğitildi.",
|
||||
"pg.firstFrame": "İlk kare (isteğe bağlı)",
|
||||
"pg.lastFrame": "Son kare (isteğe bağlı)",
|
||||
"pg.keyframeHint": "Çalışmayı keyframe-to-video'ya çevirir.",
|
||||
"pg.resolution": "Çözünürlük",
|
||||
"pg.genTime": "Oluşturma süresi",
|
||||
"pg.timePerStep": "Adım başına",
|
||||
"pg.timePerFrame": "Kare başına",
|
||||
"pg.fileSize": "Dosya boyutu",
|
||||
"pg.duration": "Süre",
|
||||
"pg.play": "Oynat",
|
||||
"pg.noMediaModels": "Henüz yerel bir {kind} modeli yok. İçe aktarma sekmesinden bir tane indirin (ör. infosave koleksiyonundan):",
|
||||
"pg.tokSpeed": "Hız",
|
||||
"pg.ttft": "İlk token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "Günlükler",
|
||||
"logs.subtitle": "Ağ geçidinin son günlüğü — terminalde görecekleriniz.",
|
||||
"logs.filterPlaceholder": "filtrele…",
|
||||
"logs.pause": "Duraklat",
|
||||
"logs.resume": "Devam et",
|
||||
"logs.empty": "Eşleşen yok.",
|
||||
"notice.openAdmin": "Yönetim konsolu belirteçsiz açık ve diğer makinelerden erişilebilir. Bu makine özel değilse bir belirteç ayarlayın.",
|
||||
"notice.update": "Güncelleme mevcut: v{v}.",
|
||||
"dash.sys.title": "Sistem",
|
||||
"dash.sys.cores": "çekirdek",
|
||||
"dash.sys.disk": "Disk (modeller)",
|
||||
"dash.sys.free": "boş",
|
||||
"dash.sys.gateway": "ağ geçidi",
|
||||
"fit.disk": "disk alanı yetmez",
|
||||
"fit.diskTitle": "İndirme, model diskindeki boş alandan büyük.",
|
||||
"fit.ram": "RAM'e sığmayabilir",
|
||||
"fit.ramTitle": "Model bu makinenin RAM'ine yakın ya da ondan büyük — çalışmayabilir.",
|
||||
"pg.eta": "kaldı",
|
||||
"pg.min": "dk",
|
||||
"pg.queued": "Kuyrukta — {n}. sırada. Mevcut üretim biter bitmez başlar.",
|
||||
"pg.gallery": "Son üretimler",
|
||||
"pg.galDeleteConfirm": "Bu üretim ve dosyaları silinsin mi?",
|
||||
"pg.clearChat": "Temizle",
|
||||
"keys.snip.title": "Uygulamanı bağla",
|
||||
"keys.snip.hint": "Örneklerden birini projene yapıştır — yalnızca anahtar yer tutucusunu değiştirmen gerekir.",
|
||||
"keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
|
||||
"keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
|
||||
"keys.snip.keyPh": "API anahtarın",
|
||||
"settings.restart": "Tümünü yeniden başlat",
|
||||
"settings.restartConfirm": "Ağ geçidi ve tüm yerel model sunucuları yeniden başlatılsın mı? Devam eden istekler kesilecek.",
|
||||
"settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
|
||||
"settings.shutdown": "Tümünü durdur ve kapat",
|
||||
"settings.shutdownConfirm": "Tüm yerel model sunucuları durdurulup ağ geçidi kapatılsın mı? Ağ geçidini yeniden başlatana kadar API ve bu panel yanıt vermez.",
|
||||
"settings.shutdownDone": "Ağ geçidi kapalı.",
|
||||
"settings.shutdownHint": "Tüm yerel model sunucuları durduruldu ve süreç sonlandı. Paneli geri getirmek için ağ geçidini sunucuda yeniden başlatın.",
|
||||
"bench.title": "Yerel model hızı",
|
||||
"bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
|
||||
"bench.run": "Ölç",
|
||||
"bench.running": "ölçülüyor…",
|
||||
"bench.failed": "başarısız",
|
||||
"bench.none": "Henüz yerel .cmf sohbet modeli yok.",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "Sohbet",
|
||||
"task.code": "Kod",
|
||||
"task.creative-writing": "Yaratıcı yazarlık",
|
||||
"task.extraction": "Veri çıkarma",
|
||||
"task.math": "Matematik",
|
||||
"task.qa": "Soru-cevap",
|
||||
"task.summarization": "Özetleme",
|
||||
"task.translation": "Çeviri",
|
||||
"routing.tasks.title": "Göreve göre yönlendirme",
|
||||
"routing.tasks.hint": "Kuralın modelleri kendi görevi için önce gelir; karmaşıklık zinciri yedek olarak kalır. Boş = yalnızca karmaşıklığa göre.",
|
||||
"routing.tasks.inherit": "karmaşıklığa göre (kural yok)",
|
||||
"routing.tasks.share": "isteklerin %{p}'i (7 g)",
|
||||
"routing.tasks.fromRouter": "etiketler yönlendiriciden canlı",
|
||||
"routing.tasks.builtin": "yönlendiriciye ulaşılamıyor — yerleşik etiket seti",
|
||||
"routing.tasks.addCustom": "Görev ekle",
|
||||
"routing.tasks.addPlaceholder": "özel etiket…",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "Kıyaslama",
|
||||
"bm.title": "Kıyaslama",
|
||||
"bm.subtitle": "Canlı hat üzerinden kalite ve hız probları: yanıtlar anahtar kelimeyle doğrulanır, döngü tespit edilir, model başına tok/s ve ilk token süresi ölçülür.",
|
||||
"bm.models": "Test edilecek modeller",
|
||||
"bm.quickMode": "Hızlı test",
|
||||
"bm.quickHint": "Kısa yanıtlı hızlı alt küme — yerel model başına yaklaşık bir dakika.",
|
||||
"bm.suiteInfo": "Yerleşik küme: {q} hızlı / {f} tam test, artı kendi testleriniz.",
|
||||
"bm.run": "Çalıştır",
|
||||
"bm.pickModel": "En az bir model seçin.",
|
||||
"bm.testOf": "{n} testten {i}.",
|
||||
"bm.results": "Sonuçlar",
|
||||
"bm.quick": "hızlı",
|
||||
"bm.test": "Test",
|
||||
"bm.cellHint": "Modelin tam yanıtı için hücreye tıklayın.",
|
||||
"bm.history": "Önceki çalıştırmalar (bu tarayıcıda saklanır)",
|
||||
"bm.custom": "Özel testler",
|
||||
"bm.customHint": "İstem + yanıtın içermesi gereken virgülle ayrılmış anahtar kelimeler (| eşanlamlılar için). Bu tarayıcıda saklanır.",
|
||||
"bm.promptPh": "istem…",
|
||||
"bm.checkPh": "anahtar kelimeler, virgülle",
|
||||
"bm.addTest": "Test ekle",
|
||||
"bm.emptyAnswer": "boş yanıt",
|
||||
"bm.loopDetected": "döngü tespit edildi",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "Akıllı yönlendirme",
|
||||
"bm.mode.models": "Tek tek modeller",
|
||||
"bm.mode.routingDesc": "Ürünün temel özelliği testte: her prob cortiq-auto üzerinden gider — yönlendiricinin hangi görevi algıladığını, hangi modeli seçtiğini, bunun beklentiyle örtüşüp örtüşmediğini ve maliyetini görürsünüz.",
|
||||
"bm.mode.modelsDesc": "Aynı küme seçilen modellere sabitlenir — yönlendiriciyi atlayarak kalite, hız ve maliyeti doğrudan karşılaştırın.",
|
||||
"bm.answersOk": "yanıtlar doğru",
|
||||
"bm.classAcc": "görev sınıflandırması",
|
||||
"bm.modelDist": "İstekler şuraya gitti",
|
||||
"bm.expectedLbl": "beklenen:",
|
||||
"bm.answer": "Yanıt",
|
||||
"bm.liveRunning": "sürüyor…",
|
||||
"bm.spent": "harcanan",
|
||||
"bm.verdict": "🏁 Bitti: yanıtlar {p}, harcanan {s}",
|
||||
"bm.clearHistory": "Tümünü temizle",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — terminale yapıştırın (ağ geçidi Anthropic protokolünü doğal olarak konuşur):",
|
||||
"keys.h.codex": "Codex — sağlayıcıyı ~/.codex/config.toml dosyasına ekler ve başlatır:",
|
||||
"keys.h.opencode": "OpenCode — ~/.config/opencode/opencode.json dosyasını yazar (üzerine yazar!) ve başlatır:",
|
||||
"keys.h.continue": "Continue (VS Code / JetBrains) — modeli ~/.continue/config.yaml dosyasına ekler:",
|
||||
"keys.h.cline1": "Cline (VS Code) — eklenti ayarlarında şunları seçin:",
|
||||
"keys.h.cline2": "Ayarlar → API Provider → OpenAI Compatible, ardından yukarıdaki değerleri yapıştırın.",
|
||||
"keys.h.note": "cortiq-auto = akıllı yönlendirme: yönlendirici her istek için modeli kendisi seçer. Belirli bir model için tam adını yazın. Windows'ta betikleri Git Bash / WSL içinde çalıştırın veya aynı değişkenleri sistem ortamından ayarlayın.",
|
||||
"keys.h.protoOff": "Anthropic protokolü (/v1/messages) kapalı — Claude Code bağlanamaz.",
|
||||
"keys.h.enableProto": "Etkinleştir",
|
||||
"keys.h.st.installed": "Ağ geçidi makinesinde kurulu",
|
||||
"keys.h.st.configured": "Bu ağ geçidi için zaten yapılandırılmış",
|
||||
"keys.h.st.notFound": "Ağ geçidi makinesinde bulunamadı",
|
||||
"bm.degraded": "yönlendirici yedeğe düştü",
|
||||
"bm.routerDegraded": "Yönlendirici zamanında yanıt vermedi — istek yedek zincire gitti; sınıflandırma sayılmaz.",
|
||||
"bm.cutByCap": "bütçe düşünmeye gitti — yanıt gelemedi",
|
||||
"shell.uiStale": "Ağ geçidi güncellendi — bu sayfa eski arayüzü çalıştırıyor.",
|
||||
"shell.reload": "Yeniden yükle",
|
||||
"keys.col.requests": "İstekler",
|
||||
"keys.col.tokens": "Token giriş/çıkış",
|
||||
"keys.col.costAll": "$ toplam",
|
||||
"routing.tasks.tier": "Karmaşıklık",
|
||||
"routing.tasks.tierAuto": "Otomatik (yönlendirici)",
|
||||
"routing.tasks.tierHint": "Bu görevi hangi karmaşıklıkta hangi model karşılar; \"her\" tüm katmanlarda önde gider. Sıra = yedekleme sırası. High kararı \"her\" girdilerinin üzerine yine tırmanır.",
|
||||
"routing.tasks.tierAny": "her karmaşıklık",
|
||||
"routing.tasks.addModel": "model",
|
||||
"routing.bands.enable": "Kendi karmaşıklık sınırlarım",
|
||||
"routing.bands.hint": "Tutamaçları sürükleyin: low nerede biter, high nerede başlar — tutamaçlar birbirini iter, sayılar anlık güncellenir. Yönlendiricinin sürekli puanı SİZİN sınırlarınızla yeniden bölümlenir; işaret kaldırılırsa yönlendiricinin kendi bantları geçerli.",
|
||||
"tier.low": "kolay",
|
||||
"tier.medium": "orta",
|
||||
"tier.high": "zor",
|
||||
"bm.scoreHint": "Yönlendiricinin sürekli karmaşıklık puanı (0..1) — Yönlendirme sayfasındaki sınır kaydırıcıları onu katmanlara böler.",
|
||||
"settings.cache.noEmbed": "Önbellek açık ama çalışacağı bir embedding modeli YOK — hiçbir şey yapmıyor. Havuza kind = \"embedding\" olan bir model ekleyin (örn. OpenAI text-embedding) veya aşağıdan seçin.",
|
||||
"settings.cache.entries": "kayıt",
|
||||
"settings.cache.hits": "isabet",
|
||||
"settings.cache.saved": "tasarruf",
|
||||
"keys.form.budget": "Aylık bütçe, $ (0 = sınırsız)",
|
||||
"keys.form.budgetHint": "Hesabın aylık harcaması tavanı aşınca istekler 429 insufficient_quota ile reddedilir.",
|
||||
"keys.budgetOf": "bütçe {b}/ay",
|
||||
"dash.kpi.orBalance": "OpenRouter bakiyesi",
|
||||
"keys.h.install": "Kurulum:",
|
||||
"keys.snip.keyInputPh": "API anahtarınızı yapıştırın",
|
||||
"keys.snip.apply": "Anahtarı ekle",
|
||||
};
|
||||
|
||||
Vendored
+370
@@ -280,4 +280,374 @@ export default {
|
||||
"settings.cache.ttl": "条目 TTL",
|
||||
"settings.cache.embed": "嵌入模型",
|
||||
"settings.cache.note": "缓存设置将在重启后生效。",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "导入模型",
|
||||
|
||||
// dash
|
||||
"dash.clear": "清空日志",
|
||||
"dash.clearConfirm": "清空所有已记录的统计与日志?此操作无法撤销。",
|
||||
"dash.cleared": "日志已清空",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "请输入 id 或模型名称",
|
||||
"models.form.apiKey": "API 密钥",
|
||||
"models.form.apiKeyPlaceholder": "粘贴你的 API 密钥",
|
||||
"models.form.apiKeyNotNeeded": "无需密钥(本地服务器)",
|
||||
"models.form.apiKeyHint": "安全保存在网关中,之后不会再次显示。",
|
||||
"models.form.keyStored": "已保存一个密钥 — 留空表示保留,或粘贴新的密钥替换。",
|
||||
"models.form.keyFromEnv": "正在使用环境变量中的密钥。",
|
||||
"models.form.paramsTitle": "生成与思考参数(O(1)、MTP、温度)",
|
||||
"models.form.temperature": "温度",
|
||||
"models.form.temperatureHint": "生成温度(0.0 = 确定性,1.0+ = 更有创造性)。",
|
||||
"models.form.topP": "Top-P(核采样)",
|
||||
"models.form.topPHint": "采样阈值(0.0 .. 1.0)。",
|
||||
"models.form.maxTokens": "最大 token 数",
|
||||
"models.form.maxTokensHint": "生成内容的最大 token 数。",
|
||||
"models.form.thinkBudget": "思考模式 / 思考预算",
|
||||
"models.form.thinkBudgetHint": "推理 token 预算(0 = 关闭,N > 0 = 推理 token 上限)。",
|
||||
"models.form.systemPrompt": "默认系统提示词",
|
||||
"models.form.systemPromptHint": "当请求中没有系统提示词时自动添加到前面。",
|
||||
"models.form.systemPromptPlaceholder": "你是一个乐于助人的助手……",
|
||||
"models.form.testKey": "测试并加载模型",
|
||||
"models.form.testing": "检查中……",
|
||||
"models.form.foundModels": "已加载 {n} 个模型",
|
||||
"models.form.modelPlaceholder": "输入以搜索……",
|
||||
"models.form.modelHint": "点击“测试并加载模型”用你的密钥拉取列表,然后选择一个。",
|
||||
"models.managed": "托管",
|
||||
"models.managedHint": "由托管的 CMF 服务器提供(设置 → 本地模型)。请在那里配置。",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "使用路由器(关闭 = 仅本地模型)",
|
||||
"settings.cmf": "本地模型(CMF)",
|
||||
"settings.cmf.localOnly": "仅使用本地模型",
|
||||
"settings.cmf.manage": "运行本地模型服务器",
|
||||
"settings.cmf.autoInstall": "自动安装 cortiq(crates.io)",
|
||||
"settings.cmf.autoUpdate": "自动更新 cortiq",
|
||||
"settings.cmf.model": "本地 .cmf 模型路径",
|
||||
"settings.cmf.port": "本地服务器端口",
|
||||
"settings.cmf.modelId": "模型在池中的 id",
|
||||
"settings.cmf.note": "通过 cortiq 提供本地 .cmf 服务(github.com/infosave2007/cmf)。即使路由器不可用也能工作。",
|
||||
"settings.cmf.threads": "工作线程",
|
||||
"settings.cmf.gpu": "使用 GPU(Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "线程会并行化解码阶段的矩阵向量乘法(约快 3 倍;0 = 运行时默认值)。GPU 卸载主要加速长提示词的 prefill — 解码受内存带宽限制,因此在 Apple Silicon 上基本持平。",
|
||||
"settings.cmf.pick": "选择一个模型",
|
||||
"settings.cmf.models": "本地模型",
|
||||
"settings.cmf.modelsHelp": "每一行都是网关运行的一个模型(各自独立的 cortiq serve,各占一个端口)并注册到模型池。有多少 .cmf 文件和空闲端口就可以加多少行。",
|
||||
"settings.cmf.rowHint": "id · .cmf 文件 · 端口 · 线程 · GPU。改动在重启后生效;删除一行会停止对应的服务器。",
|
||||
"settings.cmf.addServer": "+ 添加本地模型",
|
||||
"settings.cmf.checkPort": "检查端口",
|
||||
"settings.cmf.portFree": "端口空闲",
|
||||
"settings.cmf.portBusy": "端口被占用",
|
||||
"settings.cmf.useSuggested": "使用 {port}",
|
||||
"settings.cmf.serving": "{model} 正在 :{port} 提供服务",
|
||||
"settings.cmf.intro": "用 cortiq 引擎在本地运行模型 — 即使离线或路由器故障也能工作。",
|
||||
"settings.cmf.runtime": "cortiq 运行时",
|
||||
"settings.cmf.runtimeHelp": "用于加载和运行 .cmf 模型的命令行工具 `cortiq`。启动本地服务器前先安装一次。",
|
||||
"settings.cmf.install": "安装 cortiq",
|
||||
"settings.cmf.reinstall": "重新安装 / 更新",
|
||||
"settings.cmf.installing": "安装中……(需要编译,几分钟)",
|
||||
"settings.cmf.installedOk": "cortiq 已安装",
|
||||
"settings.cmf.notInstalled": "未安装",
|
||||
"settings.cmf.addModel": "添加模型",
|
||||
"settings.cmf.addModelHelp": "模型在“导入模型”标签页下载并转换(HuggingFace → .cmf)。然后把下面的本地服务器指向该 .cmf 文件。",
|
||||
"settings.cmf.importNote": "还没有模型?去导入一个:",
|
||||
"settings.cmf.importLink": "打开导入",
|
||||
"settings.cmf.modelHelp": "要提供服务的 .cmf 文件路径(例如 models/qwen.cmf),由导入标签页生成。要为不同任务准备多个模型?导入它们并切换此路径。",
|
||||
"settings.cmf.portHelp": "本地模型服务器监听的 TCP 端口(默认 8081)。仅在与其他服务冲突时修改。",
|
||||
"settings.cmf.threadsHelp": "生成时使用多少个 CPU 核心。越多越快(相比单核约快 3 倍);0 表示交给运行时决定。",
|
||||
"settings.cmf.gpuHelp": "把矩阵运算卸载到 GPU(Metal/wgpu)。在 Apple Silicon 上主要加速长提示词 — 解码受内存带宽限制。",
|
||||
"settings.cmf.localOnlyHelp": "把每个请求都发往本地模型,忽略路由器和所有云端供应商。",
|
||||
"settings.cmf.manageHelp": "网关启动时会为下面的 .cmf 启动 `cortiq serve` 并把请求路由过去。需要已安装 cortiq(第 1 步)。重启后生效。",
|
||||
"settings.cmf.autoInstallHelp": "如果启动时缺少 cortiq,自动从 crates.io 安装。",
|
||||
"settings.cmf.autoUpdateHelp": "启动时如有更新版本,将 cortiq 升级到最新发布版。",
|
||||
"settings.router.enabledHelp": "让 cortiq 路由器对每个请求分类(任务类型 + 复杂度)并挑选模型。关闭 = 仅使用本地路由表。",
|
||||
"settings.router.verifyTlsHelp": "校验路由器的 TLS 证书。仅在使用自签名或绑定 IP 的证书时关闭。",
|
||||
"settings.telemetry.metricsHelp": "在 GET /metrics 暴露 Prometheus 指标。",
|
||||
"settings.cortiq.echoHelp": "在每个响应体中加入 `cortiq` 字段(任务、模型、成本)。响应头始终会发送。",
|
||||
"settings.stats.enabledHelp": "记录最近的请求,为仪表盘和图表提供数据。",
|
||||
"settings.cache.enabledHelp": "语义缓存:当新提示词与过去的某个非常相似时直接返回已保存的回答,跳过模型调用。需要池中有一个嵌入模型。",
|
||||
"settings.log.bodiesHelp": "记录请求/响应的完整消息体。出于隐私考虑默认关闭。",
|
||||
"settings.router.saveKey": "保存密钥",
|
||||
"settings.router.keySaved": "路由器密钥已保存",
|
||||
"settings.router.keyEmpty": "请先粘贴密钥",
|
||||
"settings.router.secretHelp": "粘贴 api.allaigate.com 邮件中的密钥,然后点击“保存密钥”。这不是“API 密钥”页中的值 — 那些是给调用你网关的客户端用的。",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — 密钥已保存(再次粘贴可替换)",
|
||||
"settings.route.strategyHelp": "发送给路由器用于分类的文本:last_user = 仅最后一条用户消息;last_user_plus_system = 最后一条用户消息 + 系统提示词;concat_all = 整段对话。",
|
||||
"settings.route.maxCharsHelp": "发送给路由器的文本最大字符数(超出会截断)。越多分类越准,但路由器调用更慢也更贵。",
|
||||
"settings.route.cacheTtlHelp": "相同请求的路由决策缓存多久后才再次询问路由器(例如 60s、5m)。",
|
||||
"settings.route.profileHelp": "请求未指定时使用的默认路由档位:cost-saver(最省钱)、balanced 或 quality-first(最好的模型)。会影响路由器挑选哪个模型。",
|
||||
|
||||
// import
|
||||
"import.subtitle": "在 HuggingFace 上搜索模型,转换为本地 .cmf 并提供服务。",
|
||||
"import.searchPlaceholder": "搜索 HuggingFace 模型(例如 qwen3、llama)……",
|
||||
"import.searching": "搜索中……",
|
||||
"import.noResults": "未找到模型。",
|
||||
"import.configure": "配置转换",
|
||||
"import.outName": "输出名称",
|
||||
"import.outNameHint": ".cmf 文件名(字母、数字、- 和 _)。",
|
||||
"import.quant": "量化",
|
||||
"import.convert": "转换并加载",
|
||||
"import.converting": "转换中",
|
||||
"import.started": "转换已开始",
|
||||
"import.failed": "转换失败 — 请查看日志。",
|
||||
"import.register": "注册为本地模型",
|
||||
"import.registered": "已注册为 “{id}”。运行 cortiq-server 即可提供服务。",
|
||||
"import.jobs": "下载与转换",
|
||||
"import.cancel": "取消",
|
||||
"import.cancelling": "正在取消 — 清理未完成的文件……",
|
||||
"import.delete": "删除",
|
||||
"import.deleteConfirm": "从磁盘删除该模型及其转换好的 .cmf 文件?",
|
||||
"import.deleted": "已连同文件一起删除",
|
||||
"import.state.running": "进行中",
|
||||
"import.state.done": "完成",
|
||||
"import.state.error": "错误",
|
||||
"import.state.cancelled": "已取消",
|
||||
"import.featuredTitle": "推荐的开箱即用 .cmf 模型",
|
||||
"import.featuredSub": "来自 HuggingFace 上 infosave 账号的预转换 .cmf 模型 — 列表实时读取,新模型会自动出现。",
|
||||
"import.readyCmfBadge": "现成 .cmf",
|
||||
"import.directDownload": "下载",
|
||||
"import.q.q8_2f": "8 位双场(𝒲×θ)— 推荐:质量与体积的最佳平衡。",
|
||||
"import.q.q8_row": "8 位按行 — 简单稳健。",
|
||||
"import.q.q4": "4 位分块 — 体积最小,质量较低。",
|
||||
"import.q.q4_tiled": "4 位分块平铺(Q4_TILED)— 在 Metal GPU 上快 1.66 倍。",
|
||||
"import.q.vbit": "可变 3–8 位 — 按专家分配位宽预算。",
|
||||
"import.q.q1t": "1.58 位三值(Q1T)— BitNet b1.58,带稀疏离群值。",
|
||||
"import.q.q1s": "1 位稀疏(Q1S)— 1 位基底,带全息离群值。",
|
||||
"import.q.q1p": "1 位 PTQ(Q1P)— 1 位,带误差扩散。",
|
||||
"import.q.q1": "1 位(Q1)— 纯二值格式。",
|
||||
"import.q.f16": "16 位 — 不量化。",
|
||||
"import.q.f32": "32 位 — 全精度(文件很大)。",
|
||||
"import.adv.title": "高级设置",
|
||||
"import.adv.auto": "自动",
|
||||
"import.adv.linearCore": "线性核",
|
||||
"import.adv.linearHint": "线性注意力层如何被保留或折叠。",
|
||||
"import.adv.nphase": "vmf_phase 每头相位数",
|
||||
"import.adv.vbitShape": "VBIT 预算形状",
|
||||
"import.adv.meanBits": "VBIT 平均位宽",
|
||||
"import.adv.shard": "分片上限(GB)",
|
||||
"import.adv.noShard": "不分片",
|
||||
"import.adv.skipMtp": "跳过 MTP 头",
|
||||
"import.adv.o1": "O(1) 注意力",
|
||||
"import.adv.o1Hint": "常量内存的流式注意力:prefill 之后释放 KV 缓存,因此长上下文不会持续增加内存。权重保持逐字节一致 — 只记录一个加载时的提示;随时可以切回(启动时加 --o1 off)。已在 Qwen3-0.6B 上验证:全部层约为 ×1.18 困惑度。",
|
||||
"import.adv.o1Off": "关闭 — 精确注意力(默认)",
|
||||
"import.adv.o1All": "all — 转换所有 softmax 层",
|
||||
"import.adv.o1Deep": "deep N — 仅最深的 N 层",
|
||||
"import.adv.o1Custom": "自定义 — 显式指定层列表",
|
||||
"import.adv.o1DeepN": "最深的 N 层",
|
||||
"import.adv.o1Layers": "层(用逗号分隔)",
|
||||
"import.adv.o1mHint": "landmark 预算(默认 32)",
|
||||
"import.adv.o1windowHint": "精确窗口宽度(默认 128)",
|
||||
"import.adv.o1sinkHint": "常驻精确 sink key 数(默认 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "初始设置",
|
||||
"onboarding.welcome": "欢迎使用 Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "选择你的语言 — 随时可以在顶栏更改。",
|
||||
"onboarding.next": "下一步",
|
||||
"onboarding.back": "上一步",
|
||||
"onboarding.skip": "暂时跳过",
|
||||
"onboarding.skipSetup": "跳过",
|
||||
"onboarding.finish": "完成设置",
|
||||
"onboarding.featuredTitle": "开箱即用的模型(一键)",
|
||||
"onboarding.featuredSub": "这些是 HuggingFace 上 infosave 发布的、已经转换好的 .cmf 模型 — 直接下载即可,无需转换。选择一个或多个即可立即开始。",
|
||||
"onboarding.download": "下载",
|
||||
"onboarding.hfTitle": "任意 HuggingFace 模型 — 自动转换",
|
||||
"onboarding.hfSub": "搜索任意模型,选择量化方式,我们会自动把它转换为 .cmf。无需手动修改配置。",
|
||||
"onboarding.noJobs": "还没有下载任务。",
|
||||
"onboarding.doneTitle": "全部就绪",
|
||||
"onboarding.doneSub": "下载好的模型会自动注册。之后可以在“导入模型”或“模型”中添加更多。",
|
||||
"onboarding.doneHint": "“完成设置”会把每个下载完成的模型接入为本地模型,启动它的 .cmf 服务器并把路由指向它。这些之后都可以在“模型”中修改。",
|
||||
"onboarding.willRegister": "将接入为本地模型",
|
||||
"onboarding.nothingToRegister": "还没有下载完成的模型 — 没有可接入的内容。返回上一步选一个模型。",
|
||||
"onboarding.registeredN": "已接入 {n} 个模型:{ids}",
|
||||
"onboarding.runtimeMissing": "cortiq 运行时尚未安装 — 请在 设置 → 本地模型 中安装后才能运行这些模型。",
|
||||
"onboarding.downloaded": "已下载",
|
||||
"onboarding.update": "更新",
|
||||
"onboarding.deleteFile": "从磁盘删除该 .cmf 文件",
|
||||
"onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。",
|
||||
"onboarding.unregistered": "同时已取消注册:{ids}",
|
||||
|
||||
// playground: media generation + streaming metrics
|
||||
"pg.tab.chat": "对话",
|
||||
"pg.tab.image": "图像",
|
||||
"pg.tab.video": "视频",
|
||||
"pg.generate": "生成",
|
||||
"pg.generating": "生成中……",
|
||||
"pg.genFailed": "生成失败 — 请查看日志。",
|
||||
"pg.result": "结果",
|
||||
"pg.genMetrics": "生成指标",
|
||||
"pg.imagePromptPlaceholder": "夕阳下雪地里的红狐狸,照片级真实感,毛发细节丰富",
|
||||
"pg.videoPromptPlaceholder": "戴厨师帽的柯基翻煎饼,滋滋声和欢快的犬吠",
|
||||
"pg.imageEmpty": "生成的图像将显示在这里。",
|
||||
"pg.videoEmpty": "生成的短片(含音频)将显示在这里。",
|
||||
"pg.width": "宽度",
|
||||
"pg.height": "高度",
|
||||
"pg.steps": "步数",
|
||||
"pg.seed": "Seed",
|
||||
"pg.randomSeed": "随机 seed",
|
||||
"pg.cfgHint": "引导强度;≤ 1 关闭 CFG,工作量减半。",
|
||||
"pg.frames": "帧数",
|
||||
"pg.framesHint": "24 fps;按模型网格取整。",
|
||||
"pg.quality": "JPEG 质量",
|
||||
"pg.videoSizeHint": "32 的倍数;训练短边为 768。",
|
||||
"pg.videoStepsHint": "Turbo LoRA 按 4 步训练。",
|
||||
"pg.firstFrame": "首帧(可选)",
|
||||
"pg.lastFrame": "末帧(可选)",
|
||||
"pg.keyframeHint": "使运行变为关键帧生视频。",
|
||||
"pg.resolution": "分辨率",
|
||||
"pg.genTime": "生成耗时",
|
||||
"pg.timePerStep": "每步",
|
||||
"pg.timePerFrame": "每帧",
|
||||
"pg.fileSize": "文件大小",
|
||||
"pg.duration": "时长",
|
||||
"pg.play": "播放",
|
||||
"pg.noMediaModels": "还没有本地{kind}模型。请在“导入模型”中下载一个(例如 infosave 合集):",
|
||||
"pg.tokSpeed": "速度",
|
||||
"pg.ttft": "首个 token",
|
||||
|
||||
// ops: logs, system, notices, gallery, chat, snippets, bench
|
||||
"nav.logs": "日志",
|
||||
"logs.subtitle": "网关最近的日志 — 终端里会看到的内容。",
|
||||
"logs.filterPlaceholder": "过滤……",
|
||||
"logs.pause": "暂停",
|
||||
"logs.resume": "继续",
|
||||
"logs.empty": "没有匹配内容。",
|
||||
"notice.openAdmin": "管理控制台未设置令牌即可从其他机器访问。若这台机器不是私有的,请设置令牌。",
|
||||
"notice.update": "有可用更新:v{v}。",
|
||||
"dash.sys.title": "系统",
|
||||
"dash.sys.cores": "核",
|
||||
"dash.sys.disk": "磁盘(模型)",
|
||||
"dash.sys.free": "可用",
|
||||
"dash.sys.gateway": "网关",
|
||||
"fit.disk": "磁盘空间不足",
|
||||
"fit.diskTitle": "下载大小超过模型磁盘的可用空间。",
|
||||
"fit.ram": "可能超出内存",
|
||||
"fit.ramTitle": "模型接近或超过本机内存 — 可能无法运行。",
|
||||
"pg.eta": "剩余",
|
||||
"pg.min": "分钟",
|
||||
"pg.queued": "排队中 — 第 {n} 位。当前生成结束后立即开始。",
|
||||
"pg.gallery": "最近的生成",
|
||||
"pg.galDeleteConfirm": "删除这次生成及其文件?",
|
||||
"pg.clearChat": "清空",
|
||||
"keys.snip.title": "接入你的应用",
|
||||
"keys.snip.hint": "把示例粘贴到你的项目里 — 只需替换密钥占位符。",
|
||||
"keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
|
||||
"keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
|
||||
"keys.snip.keyPh": "你的 API 密钥",
|
||||
"settings.restart": "全部重启",
|
||||
"settings.restartConfirm": "重启网关和所有本地模型服务器?正在处理的请求将被中断。",
|
||||
"settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
|
||||
"settings.shutdown": "全部停止并关机",
|
||||
"settings.shutdownConfirm": "停止所有本地模型服务器并关闭网关?在你重新启动网关之前,API 和此面板都将无法响应。",
|
||||
"settings.shutdownDone": "网关已关闭。",
|
||||
"settings.shutdownHint": "所有本地模型服务器已停止,进程已退出。请在服务器上重新启动网关以恢复面板。",
|
||||
"bench.title": "本地模型速度",
|
||||
"bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s,一次一个模型。",
|
||||
"bench.run": "测速",
|
||||
"bench.running": "测速中……",
|
||||
"bench.failed": "失败",
|
||||
"bench.none": "还没有本地 .cmf 聊天模型。",
|
||||
|
||||
// task-based routing
|
||||
"task.chitchat": "闲聊",
|
||||
"task.code": "代码",
|
||||
"task.creative-writing": "创意写作",
|
||||
"task.extraction": "数据提取",
|
||||
"task.math": "数学",
|
||||
"task.qa": "问答",
|
||||
"task.summarization": "摘要",
|
||||
"task.translation": "翻译",
|
||||
"routing.tasks.title": "按任务路由",
|
||||
"routing.tasks.hint": "规则中的模型对该任务优先;下方的复杂度链仍作为回退。留空 = 仅按复杂度。",
|
||||
"routing.tasks.inherit": "按复杂度(无规则)",
|
||||
"routing.tasks.share": "{p}% 的请求(7 天)",
|
||||
"routing.tasks.fromRouter": "标签实时来自路由器",
|
||||
"routing.tasks.builtin": "路由器不可达 — 使用内置标签集",
|
||||
"routing.tasks.addCustom": "添加任务",
|
||||
"routing.tasks.addPlaceholder": "自定义标签……",
|
||||
|
||||
// benchmark view
|
||||
"nav.benchmark": "基准测试",
|
||||
"bm.title": "基准测试",
|
||||
"bm.subtitle": "通过实时管线做质量与速度探测:按关键词核对答案、检测循环输出,并按模型测量 tok/s 与首个 token 时间。",
|
||||
"bm.models": "要测试的模型",
|
||||
"bm.quickMode": "快速测试",
|
||||
"bm.quickHint": "答案简短的快速子集 — 每个本地模型约一分钟。",
|
||||
"bm.suiteInfo": "内置套件:{q} 个快速 / {f} 个完整测试,另加你自己的。",
|
||||
"bm.run": "运行",
|
||||
"bm.pickModel": "请至少选择一个模型。",
|
||||
"bm.testOf": "第 {i}/{n} 个测试",
|
||||
"bm.results": "结果",
|
||||
"bm.quick": "快速",
|
||||
"bm.test": "测试",
|
||||
"bm.cellHint": "点击单元格查看模型的完整回答。",
|
||||
"bm.history": "历史运行(保存在本浏览器)",
|
||||
"bm.custom": "自定义测试",
|
||||
"bm.customHint": "提示词 + 逗号分隔的关键词,回答必须包含它们(| 表示同义词)。保存在本浏览器。",
|
||||
"bm.promptPh": "提示词……",
|
||||
"bm.checkPh": "关键词,逗号分隔",
|
||||
"bm.addTest": "添加测试",
|
||||
"bm.emptyAnswer": "回答为空",
|
||||
"bm.loopDetected": "检测到循环",
|
||||
|
||||
// benchmark: routing mode, cost, live
|
||||
"bm.mode.routing": "智能路由",
|
||||
"bm.mode.models": "单独模型",
|
||||
"bm.mode.routingDesc": "测试产品核心能力:每个探测都经过 cortiq-auto — 可以看到路由器识别的任务、选中的模型、是否符合预期,以及花费了多少。",
|
||||
"bm.mode.modelsDesc": "同一套测试固定在所选模型上 — 绕过路由器,直接对比它们的质量、速度与成本。",
|
||||
"bm.answersOk": "回答正确",
|
||||
"bm.classAcc": "任务分类",
|
||||
"bm.modelDist": "请求流向",
|
||||
"bm.expectedLbl": "预期:",
|
||||
"bm.answer": "回答",
|
||||
"bm.liveRunning": "进行中……",
|
||||
"bm.spent": "花费",
|
||||
"bm.verdict": "🏁 完成:回答 {p},花费 {s}",
|
||||
"bm.clearHistory": "全部清除",
|
||||
|
||||
// harness auto-setup (API keys view)
|
||||
"keys.h.claude": "Claude Code — 粘贴到终端(网关原生支持 Anthropic 协议):",
|
||||
"keys.h.codex": "Codex — 将提供商追加到 ~/.codex/config.toml 并启动:",
|
||||
"keys.h.opencode": "OpenCode — 写入 ~/.config/opencode/opencode.json(会覆盖!)并启动:",
|
||||
"keys.h.continue": "Continue(VS Code / JetBrains)— 将模型追加到 ~/.continue/config.yaml:",
|
||||
"keys.h.cline1": "Cline(VS Code)— 在扩展设置中选择:",
|
||||
"keys.h.cline2": "设置 → API Provider → OpenAI Compatible,然后粘贴上面的值。",
|
||||
"keys.h.note": "cortiq-auto = 智能路由:路由器为每个请求选择模型。要固定某个模型,请填写其精确名称。Windows 上请在 Git Bash / WSL 中运行脚本,或通过系统环境变量设置相同的值。",
|
||||
"keys.h.protoOff": "Anthropic 协议(/v1/messages)已关闭 — Claude Code 无法连接。",
|
||||
"keys.h.enableProto": "启用",
|
||||
"keys.h.st.installed": "已安装在网关所在机器上",
|
||||
"keys.h.st.configured": "已配置为使用此网关",
|
||||
"keys.h.st.notFound": "网关所在机器上未找到",
|
||||
"bm.degraded": "路由器降级",
|
||||
"bm.routerDegraded": "路由器未及时响应 — 请求走了备用链路;分类不计入统计。",
|
||||
"bm.cutByCap": "预算全用于思考——未能给出答案",
|
||||
"shell.uiStale": "网关已更新 — 此页面运行的是旧界面。",
|
||||
"shell.reload": "重新加载",
|
||||
"keys.col.requests": "请求数",
|
||||
"keys.col.tokens": "Token 入/出",
|
||||
"keys.col.costAll": "$ 累计",
|
||||
"routing.tasks.tier": "复杂度",
|
||||
"routing.tasks.tierAuto": "自动(路由器)",
|
||||
"routing.tasks.tierHint": "哪个模型在哪个复杂度上处理该任务;「任意」在所有层级领先。顺序即回退顺序。high 判定仍会越过「任意」条目升级。",
|
||||
"routing.tasks.tierAny": "任意复杂度",
|
||||
"routing.tasks.addModel": "模型",
|
||||
"routing.bands.enable": "自定义复杂度边界",
|
||||
"routing.bands.hint": "拖动手柄设定 low 到哪里结束、high 从哪里开始 — 手柄会相互推动,数字实时更新。路由器的连续得分将按你的边界重新划分;取消勾选则使用路由器自身的边界。",
|
||||
"tier.low": "简单",
|
||||
"tier.medium": "中等",
|
||||
"tier.high": "困难",
|
||||
"bm.scoreHint": "路由器的连续复杂度得分(0..1)— 路由页面上的边界滑块将其划分为层级。",
|
||||
"settings.cache.noEmbed": "缓存已启用,但没有可用的 embedding 模型 — 它没有任何作用。请向模型池添加 kind = \"embedding\" 的模型(如 OpenAI text-embedding),或在下方选择。",
|
||||
"settings.cache.entries": "条目",
|
||||
"settings.cache.hits": "命中",
|
||||
"settings.cache.saved": "已节省",
|
||||
"keys.form.budget": "每月预算,$(0 = 不限)",
|
||||
"keys.form.budgetHint": "当账户当月支出超过上限时,请求将被拒绝并返回 429 insufficient_quota。",
|
||||
"keys.budgetOf": "预算 {b}/月",
|
||||
"dash.kpi.orBalance": "OpenRouter 余额",
|
||||
"keys.h.install": "安装:",
|
||||
"keys.snip.keyInputPh": "粘贴你的 API 密钥",
|
||||
"keys.snip.apply": "插入密钥",
|
||||
};
|
||||
|
||||
Vendored
+167
@@ -0,0 +1,167 @@
|
||||
// Browser-side codecs for cortiq's dependency-free media outputs:
|
||||
// - P6 PPM → canvas (cortiq imagine links no image library)
|
||||
// - MJPEG+PCM AVI → frame list + canvas player synced to the .wav
|
||||
// (cortiq animate never touches ffmpeg; neither do we)
|
||||
|
||||
/// Parse binary P6 PPM into {width, height, rgb: Uint8Array}.
|
||||
export function parsePpm(buf) {
|
||||
const b = new Uint8Array(buf);
|
||||
// header: "P6" <ws> width <ws> height <ws> maxval <single ws> raster
|
||||
let pos = 0;
|
||||
const isWs = (c) => c === 32 || c === 9 || c === 10 || c === 13;
|
||||
function token() {
|
||||
while (pos < b.length && (isWs(b[pos]) || b[pos] === 35 /* # comment */)) {
|
||||
if (b[pos] === 35) while (pos < b.length && b[pos] !== 10) pos++;
|
||||
else pos++;
|
||||
}
|
||||
let start = pos;
|
||||
while (pos < b.length && !isWs(b[pos])) pos++;
|
||||
return String.fromCharCode(...b.slice(start, pos));
|
||||
}
|
||||
if (token() !== "P6") throw new Error("not a P6 PPM");
|
||||
const width = parseInt(token());
|
||||
const height = parseInt(token());
|
||||
const maxval = parseInt(token());
|
||||
pos++; // single whitespace after maxval
|
||||
if (!width || !height || !maxval) throw new Error("bad PPM header");
|
||||
return { width, height, rgb: b.subarray(pos, pos + width * height * 3) };
|
||||
}
|
||||
|
||||
/// Draw a parsed PPM onto a (new or given) canvas. Returns the canvas.
|
||||
export function ppmToCanvas(buf, canvas) {
|
||||
const { width, height, rgb } = parsePpm(buf);
|
||||
const c = canvas || document.createElement("canvas");
|
||||
c.width = width;
|
||||
c.height = height;
|
||||
const ctx = c.getContext("2d");
|
||||
const img = ctx.createImageData(width, height);
|
||||
for (let i = 0, j = 0; i < width * height; i++, j += 3) {
|
||||
img.data[i * 4] = rgb[j];
|
||||
img.data[i * 4 + 1] = rgb[j + 1];
|
||||
img.data[i * 4 + 2] = rgb[j + 2];
|
||||
img.data[i * 4 + 3] = 255;
|
||||
}
|
||||
ctx.putImageData(img, 0, 0);
|
||||
return c;
|
||||
}
|
||||
|
||||
/// Rasterize an uploaded image file to WxH (stretched, as the engine does with
|
||||
/// the first frame) and return it as base64-encoded binary P6 PPM.
|
||||
export async function fileToPpmB64(file, width, height) {
|
||||
const bmp = await createImageBitmap(file);
|
||||
const c = document.createElement("canvas");
|
||||
c.width = width;
|
||||
c.height = height;
|
||||
const ctx = c.getContext("2d");
|
||||
ctx.drawImage(bmp, 0, 0, width, height);
|
||||
const img = ctx.getImageData(0, 0, width, height).data;
|
||||
const header = `P6\n${width} ${height}\n255\n`;
|
||||
const out = new Uint8Array(header.length + width * height * 3);
|
||||
for (let i = 0; i < header.length; i++) out[i] = header.charCodeAt(i);
|
||||
let p = header.length;
|
||||
for (let i = 0; i < width * height; i++) {
|
||||
out[p++] = img[i * 4];
|
||||
out[p++] = img[i * 4 + 1];
|
||||
out[p++] = img[i * 4 + 2];
|
||||
}
|
||||
// chunked btoa — the frame can be a few MB
|
||||
let bin = "";
|
||||
for (let i = 0; i < out.length; i += 0x8000)
|
||||
bin += String.fromCharCode.apply(null, out.subarray(i, i + 0x8000));
|
||||
return btoa(bin);
|
||||
}
|
||||
|
||||
/// Extract JPEG frames from an MJPEG AVI (RIFF): walk chunks from 'movi',
|
||||
/// collect '00dc'/'00dj' payloads as Blobs.
|
||||
export function aviJpegFrames(buf) {
|
||||
const b = new Uint8Array(buf);
|
||||
const dv = new DataView(buf);
|
||||
const fourcc = (o) => String.fromCharCode(b[o], b[o + 1], b[o + 2], b[o + 3]);
|
||||
if (fourcc(0) !== "RIFF" || fourcc(8) !== "AVI ") throw new Error("not an AVI");
|
||||
// find the 'movi' LIST
|
||||
let movi = -1;
|
||||
for (let o = 12; o + 12 <= b.length; ) {
|
||||
const id = fourcc(o);
|
||||
const size = dv.getUint32(o + 4, true);
|
||||
if (id === "LIST" && fourcc(o + 8) === "movi") {
|
||||
movi = o + 12;
|
||||
break;
|
||||
}
|
||||
o += 8 + size + (size & 1);
|
||||
}
|
||||
if (movi < 0) throw new Error("no movi list");
|
||||
const frames = [];
|
||||
for (let o = movi; o + 8 <= b.length; ) {
|
||||
const id = fourcc(o);
|
||||
const size = dv.getUint32(o + 4, true);
|
||||
if (id === "00dc" || id === "00dj") {
|
||||
if (size > 0) frames.push(new Blob([b.subarray(o + 8, o + 8 + size)], { type: "image/jpeg" }));
|
||||
} else if (id === "LIST") {
|
||||
o += 12; // descend (rec lists)
|
||||
continue;
|
||||
} else if (id === "idx1") {
|
||||
break;
|
||||
}
|
||||
o += 8 + size + (size & 1);
|
||||
}
|
||||
return frames;
|
||||
}
|
||||
|
||||
/// Canvas player for extracted MJPEG frames at a fixed fps, synced to an
|
||||
/// optional <audio> element (the .wav). Returns {canvas, play, pause, seek0, playing}.
|
||||
export function mjpegPlayer(frames, fps, audio) {
|
||||
const canvas = document.createElement("canvas");
|
||||
const ctx = canvas.getContext("2d");
|
||||
let bitmaps = null; // decoded lazily on first play
|
||||
let raf = null;
|
||||
let t0 = 0;
|
||||
let startFrame = 0;
|
||||
const state = { playing: false };
|
||||
|
||||
async function decodeAll() {
|
||||
if (bitmaps) return;
|
||||
bitmaps = await Promise.all(frames.map((f) => createImageBitmap(f)));
|
||||
if (bitmaps.length) {
|
||||
canvas.width = bitmaps[0].width;
|
||||
canvas.height = bitmaps[0].height;
|
||||
ctx.drawImage(bitmaps[0], 0, 0);
|
||||
}
|
||||
}
|
||||
function drawAt(i) {
|
||||
if (bitmaps && bitmaps[i]) ctx.drawImage(bitmaps[i], 0, 0);
|
||||
}
|
||||
function tick() {
|
||||
// audio clock when available (keeps A/V in sync), wall clock otherwise
|
||||
const t = audio && !audio.paused ? audio.currentTime : (performance.now() - t0) / 1000;
|
||||
const i = Math.min(frames.length - 1, startFrame + Math.floor(t * fps));
|
||||
drawAt(i);
|
||||
if (i >= frames.length - 1) {
|
||||
pause();
|
||||
return;
|
||||
}
|
||||
raf = requestAnimationFrame(tick);
|
||||
}
|
||||
async function play() {
|
||||
await decodeAll();
|
||||
state.playing = true;
|
||||
startFrame = 0;
|
||||
t0 = performance.now();
|
||||
if (audio) {
|
||||
audio.currentTime = 0;
|
||||
audio.play().catch(() => {});
|
||||
}
|
||||
raf = requestAnimationFrame(tick);
|
||||
}
|
||||
function pause() {
|
||||
state.playing = false;
|
||||
if (raf) cancelAnimationFrame(raf);
|
||||
raf = null;
|
||||
if (audio) audio.pause();
|
||||
}
|
||||
async function seek0() {
|
||||
pause();
|
||||
await decodeAll();
|
||||
drawAt(0);
|
||||
}
|
||||
return { canvas, play, pause, seek0, state, decodeAll };
|
||||
}
|
||||
Vendored
+33
-4
@@ -23,9 +23,15 @@ export function clear(node) {
|
||||
while (node && node.firstChild) node.removeChild(node.firstChild);
|
||||
}
|
||||
|
||||
export function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
export function mount(node, ...children) {
|
||||
clear(node);
|
||||
for (const c of children.flat()) if (c != null && c !== false) node.appendChild(c);
|
||||
for (const c of children.flat())
|
||||
if (c != null && c !== false)
|
||||
node.appendChild(typeof c === "object" ? c : document.createTextNode(String(c)));
|
||||
return node;
|
||||
}
|
||||
|
||||
@@ -44,7 +50,14 @@ export function toast(msg, kind = "info", ms = 3200) {
|
||||
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
|
||||
export function modal(title, body, onSave, saveLabel) {
|
||||
const backdrop = h("div", { class: "modal-backdrop" });
|
||||
const close = () => backdrop.remove();
|
||||
const onKeydown = (e) => {
|
||||
if (e.key === "Escape") close();
|
||||
};
|
||||
const close = () => {
|
||||
backdrop.remove();
|
||||
document.body.classList.remove("modal-open");
|
||||
document.removeEventListener("keydown", onKeydown);
|
||||
};
|
||||
backdrop.addEventListener("click", (e) => {
|
||||
if (e.target === backdrop) close();
|
||||
});
|
||||
@@ -71,6 +84,8 @@ export function modal(title, body, onSave, saveLabel) {
|
||||
)
|
||||
);
|
||||
backdrop.appendChild(card);
|
||||
document.body.classList.add("modal-open");
|
||||
document.addEventListener("keydown", onKeydown);
|
||||
document.body.appendChild(backdrop);
|
||||
return { close };
|
||||
}
|
||||
@@ -80,11 +95,25 @@ export function confirmDialog(message) {
|
||||
}
|
||||
|
||||
// number formatting
|
||||
// Money as a DOM node: 4 decimals, the last two rendered smaller — tiny
|
||||
// per-request prices stay readable without shouting six zeros.
|
||||
export function moneyEl(n) {
|
||||
if (n == null) return document.createTextNode("—");
|
||||
if (n === 0) return document.createTextNode("$0");
|
||||
// sub-0.0001 costs (cheap flash models) would collapse to $0.0000 — grow
|
||||
// precision until two significant digits survive
|
||||
const s = Math.abs(n) >= 0.0001 ? n.toFixed(4) : n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
|
||||
return h("span", { class: "money" }, "$" + s.slice(0, -2), h("small", { class: "money-tail" }, s.slice(-2)));
|
||||
}
|
||||
|
||||
export function money(n) {
|
||||
if (n == null) return "—";
|
||||
if (n === 0) return "$0";
|
||||
if (Math.abs(n) < 0.01) return "$" + n.toFixed(6);
|
||||
return "$" + n.toFixed(4);
|
||||
if (Math.abs(n) > 0 && Math.abs(n) < 0.0001) return "$" + n.toFixed(Math.min(8, 1 - Math.floor(Math.log10(Math.abs(n)))));
|
||||
// up to 4 decimals, trailing zeros trimmed (but keep money-like 2 minimum)
|
||||
let s = n.toFixed(4).replace(/0+$/, "").replace(/\.$/, "");
|
||||
if (/\.\d$/.test(s)) s += "0";
|
||||
return "$" + s;
|
||||
}
|
||||
export function ms(n) {
|
||||
return n == null ? "—" : `${Math.round(Number(n))} ms`;
|
||||
|
||||
Vendored
+530
@@ -0,0 +1,530 @@
|
||||
// Benchmark — two modes over the live pipeline:
|
||||
// 🧭 Smart routing (default): the product's core promise under test — every
|
||||
// probe goes through cortiq-auto and the router's task/tier decision and
|
||||
// chosen model are judged alongside the answer itself.
|
||||
// 🎯 Individual models: the same suite pinned to selected models — compare
|
||||
// quality and speed head-to-head.
|
||||
// A redesign of the old standalone benchmark.html: same scoring ideas
|
||||
// (keyword checks with | synonyms, think-tag stripping, anti-loop detection,
|
||||
// pass/partial/fail), now embedded, localized and CDN-free.
|
||||
import { h, mount, toast, ms, money, moneyEl } from "../ui.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api, testStream } from "../api.js";
|
||||
|
||||
// ── built-in suite ──────────────────────────────────────────────────────────
|
||||
// Checks are language-neutral where possible (digits, identifiers, emails);
|
||||
// `check` entries may hold |-separated synonyms — one hit per entry counts.
|
||||
// `quick: true` marks the fast subset.
|
||||
// `label`: router task labels accepted as a correct classification (null = not
|
||||
// judged — the prompt has no natural label in the taxonomy).
|
||||
const SUITE = [
|
||||
{ id: "math-mult", cat: "math", label: ["math"], quick: true, max: 96, prompt: "What is 17 * 23? Answer with the number only.", check: ["391"] },
|
||||
{ id: "math-eq", cat: "math", label: ["math"], max: 128, prompt: "Solve 2x + 6 = 14. Answer with the value of x only.", check: ["4"] },
|
||||
{ id: "logic-syll", cat: "logic", label: ["qa", "math"], max: 96, prompt: "If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? Answer yes or no.", check: ["yes|да|oui|ja|sí|si|evet|是"] },
|
||||
{ id: "count-letters", cat: "logic", label: ["qa", "math"], quick: true, max: 64, prompt: "How many letters are in the word 'cat'? Answer with a digit.", check: ["3"] },
|
||||
{ id: "apples", cat: "logic", label: ["math", "qa"], quick: true, max: 96, prompt: "John has 3 apples, buys 2 more, then eats 1. How many apples does he have? Answer with a digit.", check: ["4"] },
|
||||
{ id: "code-fact", cat: "code", label: ["code"], max: 256, prompt: "Write a Python function factorial(n) that returns n!.", check: ["def", "factorial", "return"] },
|
||||
{ id: "qa-capital", cat: "qa", label: ["qa"], quick: true, max: 48, prompt: "What is the capital of France? One word.", check: ["paris|париж"] },
|
||||
{ id: "primes", cat: "qa", label: ["math", "qa"], max: 96, prompt: "List the first five prime numbers, comma-separated.", check: ["2", "3", "5", "7", "11"] },
|
||||
{ id: "instr-ok", cat: "instruction", label: null, quick: true, max: 32, prompt: "Reply with exactly the word OK and nothing else.", check: ["ok"] },
|
||||
{ id: "extract-email", cat: "extraction", label: ["extraction"], quick: true, max: 64, prompt: "Extract the email address from: 'Contact us at info@example.com today.' Reply with the address only.", check: ["info@example.com"] },
|
||||
{ id: "translate", cat: "translation", label: ["translation"], max: 64, prompt: "Translate to English: «Доброе утро». Reply with the translation only.", check: ["good morning"] },
|
||||
{ id: "format-json", cat: "instruction", label: null, max: 96, prompt: 'Return a JSON object with a single key "n" whose value is 7. JSON only.', check: ['"n"', "7"] },
|
||||
{ id: "summarize", cat: "summarization", label: ["summarization"], max: 96, prompt: "Summarize in one short sentence: 'The cat sat on the warm mat by the window. After a while it curled up and fell asleep in the sun.'", check: ["cat|кот|кошк"] },
|
||||
{ id: "creative", cat: "creative-writing", label: ["creative-writing"], max: 512, prompt: "Write a two-line poem about the sea.", check: ["sea|море|ocean"] },
|
||||
{ id: "chitchat", cat: "chitchat", label: ["chitchat"], quick: true, max: 48, prompt: "Hi! How are you today?", check: [] },
|
||||
// hard probes: dense enough that the router's complexity model rates them
|
||||
// high tier — this is where tier-based routing actually earns its keep.
|
||||
// Prompts are GENERATED fresh each run (domain + numbers rotate): the
|
||||
// router deliberately scores familiar text as less novel, so any fixed
|
||||
// wording would decay to medium/low over repeated runs.
|
||||
// label: null — ambiguity between task labels is exactly what makes these high
|
||||
{ id: "hard-science", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "science" },
|
||||
{ id: "hard-ops", cat: "hard", label: null, judge: "route", max: 8, check: [], gen: "ops" },
|
||||
];
|
||||
|
||||
// Fresh hard prompts per run. Two findings drive the design (measured against
|
||||
// the live router): ambiguity carries the biggest weight in the complexity
|
||||
// score, so INTERDISCIPLINARY prompts (science + engineering + planning) rate
|
||||
// high reliably, while single-domain ones classify too confidently and drop
|
||||
// to medium; and familiar text loses novelty on every repeat, so domains and
|
||||
// numbers rotate to keep each run lexically fresh.
|
||||
function hardPrompt(kind) {
|
||||
const pick = (a) => a[(Math.random() * a.length) | 0];
|
||||
const n = (lo, hi) => lo + ((Math.random() * (hi - lo)) | 0);
|
||||
const science = [
|
||||
() => `Architect a protein-folding inference pipeline for a wet lab: ensemble MSA generation with taxonomy-aware subsampling, template selection under a ${n(30, 60)} percent identity ceiling, distributed attention across ${n(4, 16)} GPUs with activation checkpointing, confidence calibration against experimental B-factors, and an active-learning loop that decides which of ${n(300, 2000)} candidate mutants to synthesize next week.`,
|
||||
() => `Schedule observations for a network of ${n(8, 30)} robotic telescopes chasing gravitational-wave counterparts: probability skymaps updated every ${n(5, 20)} minutes, slew-time-aware target assignment with decaying rewards, cloud-cover forecasts folded in as stochastic availability, spectroscopy triggered only above a ${n(40, 85)} percent kilonova likelihood, and a plan to validate the scheduler against the one historical multi-messenger event.`,
|
||||
() => `Run a closed-loop materials-discovery campaign for solid-state electrolytes: DFT screening of ${n(2000, 9000)} candidates, Bayesian optimization driving ${n(3, 12)} parallel synthesis robots, CNN-based XRD phase identification, an ionic-conductivity target of ${n(2, 25)} mS/cm, and an explicit decision policy for when simulation is trusted over synthesis.`,
|
||||
() => `Design a surface-code error-correction pipeline for a ${n(400, 3000)}-qubit superconducting processor: choose a code distance for a logical error target of 1e-${n(9, 14)} at physical error rate 1e-3, syndrome extraction avoiding hook errors, a real-time decoder under a ${n(1, 4)} microsecond budget, and the classical co-processor layout with feedback for lattice surgery.`,
|
||||
() => `Design a monitoring-reporting-verification pipeline for soil-carbon credits across ${n(500, 5000)} farms: satellite spectral indices calibrated against ${n(800, 4000)} physical core samples, hierarchical Bayesian estimation with shrinkage, additionality argued against county-level counterfactuals, uncertainty discounts priced into issuance, and an audit sampling plan catching over-reporting with ${n(90, 99)} percent power.`,
|
||||
() => `Plan a cryo-EM structure-determination campaign for a ${n(300, 900)} kDa membrane complex: grid-screening strategy across ${n(4, 12)} detergent conditions, particle picking with a distilled transformer at ${n(2, 9)} million particles, ab-initio reconstruction ambiguity resolved by tilt-pair validation, local-resolution-aware model building, and a decision rule for when to stop collecting and publish.`,
|
||||
];
|
||||
const ops = [
|
||||
() => `Model a dengue outbreak across ${n(20, 90)} districts by coupling an SEIR compartment model to vaccine cold-chain optimization: transmission parameters estimated from noisy case counts with a particle filter, uncertainty propagated into weekly demand forecasts, refrigerated routing under a ${n(120, 400)}-dose-per-crate constraint, and a defended ethical weighting for which ${n(3, 12)} districts get priority when supply covers ${n(20, 60)} percent of demand.`,
|
||||
() => `Co-optimize a regional power grid's day-ahead market with a fleet of ${n(200, 2000)} grid-scale batteries: bid curves learned from ${n(2, 6)} years of price data, unit commitment as mixed-integer programming with ramp constraints, degradation amortized into marginal costs, N-1 contingency screening after every dispatch, and a mechanism-design argument that the bidding rules cannot be gamed by a coalition controlling ${n(10, 30)} percent of storage.`,
|
||||
() => `Couple regional climate downscaling at ${n(2, 12)} km resolution to catastrophe-bond pricing: bias-correct ${n(10, 40)} CMIP ensemble members, generate stochastic hurricane tracks, propagate landfall distributions into a portfolio of ${n(30, 200)} bonds, price the attachment probabilities, and defend the discount rate applied to century-scale tail risk.`,
|
||||
() => `Co-design a hospital emergency department's triage policy with its nurse staffing schedule: arrivals as a time-varying Poisson process fitted to ${n(2, 6)} years of data, acuity drift as a hidden Markov chain, staffing as integer programming over ${n(8, 12)}-hour shifts, a queueing bound on door-to-doctor time at the ${n(90, 99)}th percentile, and a fairness constraint preventing systematic delays for low-acuity elderly patients.`,
|
||||
() => `Optimize a container port's berth allocation jointly with tidal windows and rail departures: vessel ETAs with heavy-tailed uncertainty, berth-crane assignment as a mixed-integer program re-solved hourly, tide-constrained draft limits for the ${n(3, 8)} deepest berths, a queueing model of the truck gate coupled to city traffic, and a robustness argument for surviving a ${n(24, 72)}-hour crane outage.`,
|
||||
() => `Schedule observations for a fleet of ${n(6, 40)} agricultural drones across ${n(200, 2000)} fields: multispectral revisit deadlines driven by crop growth-stage models, battery-swap logistics as a vehicle-routing problem with time windows, rain forecasts as stochastic field availability, disease-outbreak alerts pre-empting the plan, and a proof the replanner never starves a field past its agronomic deadline.`,
|
||||
];
|
||||
return pick(kind === "science" ? science : ops)();
|
||||
}
|
||||
|
||||
const LS_CUSTOM = "allaigate_bench_custom";
|
||||
const LS_HISTORY = "allaigate_bench_history";
|
||||
|
||||
function customTests() {
|
||||
try {
|
||||
return JSON.parse(localStorage.getItem(LS_CUSTOM) || "[]");
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
function saveCustom(list) {
|
||||
localStorage.setItem(LS_CUSTOM, JSON.stringify(list));
|
||||
}
|
||||
function history() {
|
||||
try {
|
||||
return JSON.parse(localStorage.getItem(LS_HISTORY) || "[]");
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
function pushHistory(run) {
|
||||
const l = history();
|
||||
l.unshift(run);
|
||||
localStorage.setItem(LS_HISTORY, JSON.stringify(l.slice(0, 12)));
|
||||
}
|
||||
function saveHistory(l) {
|
||||
localStorage.setItem(LS_HISTORY, JSON.stringify(l));
|
||||
}
|
||||
|
||||
// ── evaluation (ported from the old benchmark, trimmed) ─────────────────────
|
||||
function stripThink(text) {
|
||||
return text.replace(/<(?:\|)?(?:think|thought)(?:\|)?>[\s\S]*?(?:<\/(?:\|)?(?:think|thought)(?:\|)?>|$)/gi, "").trim();
|
||||
}
|
||||
// tail loop: the same 12+ char chunk repeated 3+ times back to back
|
||||
function looksLooped(text) {
|
||||
const tail = text.slice(-360);
|
||||
for (let len = 12; len <= 60; len += 8) {
|
||||
const probe = tail.slice(-len);
|
||||
if (probe && tail.split(probe).length - 1 >= 3) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
function evaluate(raw, test) {
|
||||
let text = stripThink(raw);
|
||||
if (!text && raw) text = raw; // max_tokens cut inside <think> — judge the raw
|
||||
const resp = text.toLowerCase();
|
||||
if (!resp.trim()) return { status: "fail", score: 0, note: t("bm.emptyAnswer") };
|
||||
if (looksLooped(resp)) return { status: "fail", score: 0, note: t("bm.loopDetected") };
|
||||
if (!test.check.length) return { status: "pass", score: 1, note: "✓" };
|
||||
let hit = 0;
|
||||
for (const cond of test.check) {
|
||||
const opts = cond.toLowerCase().split("|").map((s) => s.trim());
|
||||
if (opts.some((o) => o && resp.includes(o))) hit++;
|
||||
}
|
||||
const frac = hit / test.check.length;
|
||||
const status = frac >= 1 ? "pass" : frac >= 0.5 ? "partial" : "fail";
|
||||
return { status, score: frac, note: `${hit}/${test.check.length}` };
|
||||
}
|
||||
|
||||
export async function renderBenchmark() {
|
||||
const modelsData = await api.listModels();
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
const root = h("div");
|
||||
let running = false;
|
||||
let stopFlag = false;
|
||||
|
||||
let mode = "routing"; // routing (cortiq-auto, the product's feature) | models
|
||||
const modelCbs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox", checked: ids.length === 1 ? true : null }) }));
|
||||
const quickCb = h("input", { type: "checkbox", checked: true });
|
||||
const resultsHost = h("div", {});
|
||||
const progressHost = h("div", {});
|
||||
const historyHost = h("div", {});
|
||||
|
||||
const modeDesc = h("p", { class: "muted", style: "margin:8px 0 0" }, t("bm.mode.routingDesc"));
|
||||
const modelsPick = h("div", { class: "flex wrap", style: "gap:10px; margin-top:12px; display:none" },
|
||||
...modelCbs.map(({ id, cb }) => h("label", { class: "check" }, cb, h("span", { class: "mono" }, id))));
|
||||
const modeSeg = h("div", { class: "seg" },
|
||||
h("button", { class: "active" }, "🧭 " + t("bm.mode.routing")),
|
||||
h("button", {}, "🎯 " + t("bm.mode.models")));
|
||||
modeSeg.querySelectorAll("button").forEach((b, i) =>
|
||||
b.addEventListener("click", () => {
|
||||
mode = i === 0 ? "routing" : "models";
|
||||
modeSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === i));
|
||||
modelsPick.style.display = mode === "models" ? "" : "none";
|
||||
modeDesc.textContent = t(mode === "routing" ? "bm.mode.routingDesc" : "bm.mode.modelsDesc");
|
||||
}));
|
||||
|
||||
function suite() {
|
||||
const base = SUITE.concat(customTests());
|
||||
return quickCb.checked ? base.filter((x) => x.quick || x.custom) : base;
|
||||
}
|
||||
|
||||
const runBtn = h("button", { class: "btn primary" }, "▶ " + t("bm.run"));
|
||||
const stopBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
|
||||
stopBtn.addEventListener("click", () => {
|
||||
stopFlag = true;
|
||||
stopBtn.disabled = true;
|
||||
});
|
||||
|
||||
runBtn.addEventListener("click", async () => {
|
||||
const models = mode === "routing" ? ["cortiq-auto"] : modelCbs.filter((m) => m.cb.checked).map((m) => m.id);
|
||||
if (!models.length) {
|
||||
toast(t("bm.pickModel"), "warn");
|
||||
return;
|
||||
}
|
||||
running = true;
|
||||
stopFlag = false;
|
||||
runBtn.disabled = true;
|
||||
stopBtn.style.display = "";
|
||||
stopBtn.disabled = false;
|
||||
const tests = suite().filter((x) => mode === "routing" || x.judge !== "route");
|
||||
const run = { ts: Date.now(), mode, quick: quickCb.checked, models: {}, nTests: tests.length };
|
||||
|
||||
for (const model of models) {
|
||||
const rec = { results: [], score: 0, tokS: [], ttft: [], failed: false };
|
||||
run.models[model] = rec;
|
||||
for (let i = 0; i < tests.length; i++) {
|
||||
if (stopFlag) break;
|
||||
const test = tests[i];
|
||||
const prompt = test.gen ? hardPrompt(test.gen) : test.prompt;
|
||||
mount(
|
||||
progressHost,
|
||||
h("div", { class: "card" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "spinner" }), " ",
|
||||
h("b", { class: "mono" }, mode === "routing" ? t("bm.mode.routing") : model), " · ",
|
||||
t("bm.testOf", { i: i + 1, n: tests.length }), " · ",
|
||||
h("span", { class: "muted" }, test.id)),
|
||||
h("div", { class: "muted small", style: "margin-top:6px; font-style:italic" }, "«" + prompt.slice(0, 110) + (prompt.length > 110 ? "…»" : "»")),
|
||||
h("div", { class: "pbar", style: "margin-top:8px" },
|
||||
h("div", { class: "pbar-fill det", style: `width:${Math.round((i / tests.length) * 100)}%` })))
|
||||
);
|
||||
const started = performance.now();
|
||||
let text = "";
|
||||
try {
|
||||
const info = await testStream(
|
||||
// +256: headroom for reasoning models — a tight cap cuts them
|
||||
// mid-<think> and turns a correct model into a false ❌
|
||||
{ model, messages: [{ role: "user", content: prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
|
||||
(d) => (text += d)
|
||||
);
|
||||
const latency = performance.now() - started;
|
||||
let ev;
|
||||
if (test.judge === "route") {
|
||||
// the test asserts the ROUTER's decision: did it rate this high?
|
||||
// Two honest retries. (1) The SAME prompt after a pause: the
|
||||
// router judges difficulty in the background — the first-ever
|
||||
// occurrence routes by prior, the judgement lands in the cache a
|
||||
// couple of seconds later, so the second occurrence is the real
|
||||
// steady-state answer. (2) A fresh template, in case this one's
|
||||
// wording clustered into a burnt region.
|
||||
if (info.tier !== "high" && test.gen) {
|
||||
const attempts = [
|
||||
{ prompt, delayMs: 4000 },
|
||||
{ prompt: hardPrompt(test.gen), delayMs: 0 },
|
||||
];
|
||||
for (const a of attempts) {
|
||||
if (info.tier === "high") break;
|
||||
if (a.delayMs) await new Promise((r) => setTimeout(r, a.delayMs));
|
||||
try {
|
||||
const again = await testStream(
|
||||
{ model, messages: [{ role: "user", content: a.prompt }], temperature: 0, max_tokens: (test.max || 128) + 256 },
|
||||
() => {}
|
||||
);
|
||||
info.cost_usd = (info.cost_usd || 0) + (again.cost_usd || 0);
|
||||
if (again.tier === "high") {
|
||||
info.tier = again.tier;
|
||||
info.score = again.score;
|
||||
info.selected_model = again.selected_model;
|
||||
info.task_label = again.task_label;
|
||||
info.route_source = again.route_source;
|
||||
}
|
||||
} catch { /* keep previous attempt */ }
|
||||
}
|
||||
}
|
||||
ev = info.tier === "high"
|
||||
? { status: "pass", score: 1, note: "high ✓" }
|
||||
: { status: "fail", score: 0, note: (info.tier || "—") + " ≠ high" };
|
||||
} else {
|
||||
ev = evaluate(text, test);
|
||||
}
|
||||
const compl = info.usage?.completion_tokens;
|
||||
if (test.judge !== "route" && ev.status === "fail" && !stripThink(text).trim() && compl >= (test.max || 128) + 250) {
|
||||
ev.note = t("bm.cutByCap"); // the whole budget went into <think>
|
||||
}
|
||||
if (compl && latency > 0) {
|
||||
const gen = info.ttft_ms != null ? Math.max(1, latency - info.ttft_ms) : latency;
|
||||
rec.tokS.push(compl / (gen / 1000));
|
||||
}
|
||||
if (info.ttft_ms != null) rec.ttft.push(info.ttft_ms);
|
||||
const row = { id: test.id, cat: test.cat, ...ev, latency: Math.round(latency), cost: info.cost_usd || 0, answer: stripThink(text).slice(0, 300) };
|
||||
if (mode === "routing") {
|
||||
// router timed out / fell back — there was no classification to
|
||||
// judge, so it must not count against accuracy
|
||||
row.degraded = !info.task_label
|
||||
|| info.task_label === "degraded"
|
||||
|| String(info.route_source || "").includes("degraded");
|
||||
row.task = info.task_label || "—";
|
||||
row.tier = info.tier || "—";
|
||||
row.cscore = info.score || 0; // непрерывный коэффициент сложности
|
||||
// (row.score занят долей правильности ответа из evaluate())
|
||||
row.chosen = info.selected_model || "—";
|
||||
row.source = info.route_source || "—";
|
||||
row.expected = test.label; // null = classification not judged
|
||||
row.taskOk = test.label && !row.degraded ? test.label.includes(info.task_label) : null;
|
||||
}
|
||||
rec.results.push(row);
|
||||
} catch (e) {
|
||||
rec.results.push({ id: test.id, cat: test.cat, status: "fail", score: 0, note: String(e.message || e).slice(0, 80), latency: 0, answer: "" });
|
||||
rec.failed = true;
|
||||
}
|
||||
renderResults(run, true); // live: строка появляется сразу после теста
|
||||
}
|
||||
if (stopFlag) break;
|
||||
}
|
||||
|
||||
mount(progressHost);
|
||||
running = false;
|
||||
runBtn.disabled = false;
|
||||
stopBtn.style.display = "none";
|
||||
if (!stopFlag) {
|
||||
pushHistory(run);
|
||||
// финальный аккорд: короткий вердикт с процентами и деньгами
|
||||
const first = run.models[Object.keys(run.models)[0]];
|
||||
if (first) {
|
||||
const st = stats(first);
|
||||
const judged = first.results.filter((r) => r.taskOk != null);
|
||||
const cls = judged.length ? ` · ${judged.filter((r) => r.taskOk).length}/${judged.length}` : "";
|
||||
const total = Object.values(run.models).reduce((a, rec) => a + stats(rec).cost, 0);
|
||||
toast(t("bm.verdict", { p: Math.round(st.score * 100) + "%" + cls, s: money(total) }), "ok", 6000);
|
||||
}
|
||||
}
|
||||
renderResults(run);
|
||||
renderHistory();
|
||||
});
|
||||
|
||||
function statusIcon(s) {
|
||||
return s === "pass" ? "✅" : s === "partial" ? "◐" : "❌";
|
||||
}
|
||||
|
||||
function stats(rec) {
|
||||
const n = rec.results.length || 1;
|
||||
return {
|
||||
score: rec.results.reduce((a, r) => a + r.score, 0) / n,
|
||||
tokS: rec.tokS.length ? rec.tokS.reduce((a, b) => a + b, 0) / rec.tokS.length : null,
|
||||
ttft: rec.ttft.length ? rec.ttft.reduce((a, b) => a + b, 0) / rec.ttft.length : null,
|
||||
cost: rec.results.reduce((a, r) => a + (r.cost || 0), 0),
|
||||
};
|
||||
}
|
||||
|
||||
function renderResults(run, live) {
|
||||
const models = Object.keys(run.models);
|
||||
if (!models.length) {
|
||||
mount(resultsHost);
|
||||
return;
|
||||
}
|
||||
const routing = run.mode === "routing";
|
||||
const detail = h("div", {});
|
||||
|
||||
let summary;
|
||||
if (routing) {
|
||||
const rec = run.models[models[0]];
|
||||
const st = stats(rec);
|
||||
const judged = rec.results.filter((r) => r.taskOk != null);
|
||||
const okCount = judged.filter((r) => r.taskOk).length;
|
||||
// какие модели реально выбирал роутер
|
||||
const dist = {};
|
||||
for (const r of rec.results) {
|
||||
if (!r.chosen) continue;
|
||||
dist[r.chosen] = dist[r.chosen] || { n: 0, cost: 0 };
|
||||
dist[r.chosen].n++;
|
||||
dist[r.chosen].cost += r.cost || 0;
|
||||
}
|
||||
const pct = Math.round(st.score * 100);
|
||||
summary = h("div", {},
|
||||
h("div", { class: "kpis", style: "grid-template-columns: repeat(auto-fit, minmax(150px, 1fr))" },
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, pct + "%"), h("div", { class: "lbl" }, t("bm.answersOk"))),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, judged.length ? `${okCount}/${judged.length}` : "—"), h("div", { class: "lbl" }, t("bm.classAcc"))),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, st.ttft != null ? ms(Math.round(st.ttft)) : "—"), h("div", { class: "lbl" }, t("pg.ttft"))),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, st.tokS != null ? st.tokS.toFixed(1) : "—"), h("div", { class: "lbl" }, "tok/s")),
|
||||
h("div", { class: "kpi" }, h("div", { class: "big" }, moneyEl(st.cost)), h("div", { class: "lbl" }, t("bm.spent")))),
|
||||
h("div", { class: "flex wrap", style: "gap:6px; margin-top:4px" },
|
||||
h("span", { class: "muted small" }, t("bm.modelDist") + ":"),
|
||||
...Object.entries(dist).map(([m, d]) =>
|
||||
h("span", { class: "badge" }, `${m} × ${d.n}`, d.cost > 0 ? " · " : "", d.cost > 0 ? moneyEl(d.cost) : ""))));
|
||||
} else {
|
||||
const sorted = models.slice().sort((a, b) => stats(run.models[b]).score - stats(run.models[a]).score);
|
||||
summary = h("div", {},
|
||||
...sorted.map((m) => {
|
||||
const st = stats(run.models[m]);
|
||||
const pct = Math.round(st.score * 100);
|
||||
return h("div", { class: "bm-sum" },
|
||||
h("div", { class: "flex" },
|
||||
h("b", { class: "mono grow" }, m),
|
||||
h("span", { class: "badge " + (pct >= 80 ? "ok" : pct >= 50 ? "" : "bad") }, pct + "%"),
|
||||
st.tokS != null ? h("span", { class: "muted small", style: "margin-left:8px" }, st.tokS.toFixed(1) + " tok/s") : null,
|
||||
st.ttft != null ? h("span", { class: "muted small", style: "margin-left:8px" }, "TTFT " + ms(Math.round(st.ttft))) : null,
|
||||
st.cost > 0 ? h("span", { class: "muted small", style: "margin-left:8px" }, t("bm.spent") + " ", moneyEl(st.cost)) : null),
|
||||
h("div", { class: "sysbar" }, h("i", { class: pct < 50 ? "bad" : "", style: `width:${pct}%` })));
|
||||
}));
|
||||
}
|
||||
|
||||
function answerCell(r, m) {
|
||||
const cell = h("td", { class: "bm-cell", title: r.answer || r.note },
|
||||
statusIcon(r.status) + " ",
|
||||
h("span", { class: "muted small" }, r.note + (r.latency ? ` · ${(r.latency / 1000).toFixed(1)}s` : "")));
|
||||
cell.addEventListener("click", () => {
|
||||
mount(detail, h("div", { class: "card" },
|
||||
h("div", { class: "card-title" }, `${r.id} · ${m} · ${statusIcon(r.status)}`),
|
||||
h("pre", { class: "conv-log", style: "max-height:200px" }, r.answer || "—")));
|
||||
});
|
||||
return cell;
|
||||
}
|
||||
|
||||
let header, rows;
|
||||
if (routing) {
|
||||
const rec = run.models[models[0]];
|
||||
header = h("tr", {},
|
||||
h("th", {}, t("bm.test")),
|
||||
h("th", {}, t("pg.task")),
|
||||
h("th", {}, t("pg.tier")),
|
||||
h("th", {}, t("pg.model")),
|
||||
h("th", {}, t("bm.answer")));
|
||||
rows = rec.results.map((r) =>
|
||||
h("tr", {},
|
||||
h("td", {}, h("div", {}, r.id), h("div", { class: "muted small" }, r.cat)),
|
||||
h("td", {},
|
||||
r.degraded
|
||||
? h("span", { class: "muted", title: t("bm.routerDegraded") }, "⚠ " + t("bm.degraded"))
|
||||
: r.taskOk == null
|
||||
? h("span", { class: "muted" }, r.task || "—")
|
||||
: h("span", { class: r.taskOk ? "" : "err" },
|
||||
(r.taskOk ? "✓ " : "✗ ") + r.task,
|
||||
r.taskOk ? "" : h("div", { class: "muted small" }, t("bm.expectedLbl") + " " + (r.expected || []).join("/")))),
|
||||
h("td", {},
|
||||
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||
r.cscore > 0 ? h("div", { class: "muted small mono", title: t("bm.scoreHint") }, r.cscore.toFixed(2)) : null),
|
||||
h("td", {},
|
||||
h("span", { class: "mono small" }, r.chosen || "—"),
|
||||
r.cost > 0 ? h("div", { class: "muted small" }, moneyEl(r.cost)) : null,
|
||||
r.source && r.source.includes("task-rule") ? h("div", { class: "badge", style: "margin-top:2px" }, "task-rule") : null),
|
||||
answerCell(r, r.chosen || "cortiq-auto")));
|
||||
} else {
|
||||
header = h("tr", {}, h("th", {}, t("bm.test")), ...models.map((m) => h("th", { class: "mono" }, m)));
|
||||
rows = (run.models[models[0]].results || []).map((_, i) =>
|
||||
h("tr", {},
|
||||
h("td", {},
|
||||
h("div", {}, run.models[models[0]].results[i].id),
|
||||
h("div", { class: "muted small" }, run.models[models[0]].results[i].cat)),
|
||||
...models.map((m) => {
|
||||
const r = run.models[m].results[i];
|
||||
return r ? answerCell(r, m) : h("td", {}, "—");
|
||||
})));
|
||||
}
|
||||
|
||||
mount(resultsHost,
|
||||
h("div", { class: "card" },
|
||||
h("div", { class: "card-head" },
|
||||
h("h3", {}, (routing ? "🧭 " : "🎯 ") + t("bm.results")),
|
||||
h("span", { class: "card-head sub" },
|
||||
(live ? t("bm.liveRunning") + " · " : "") + new Date(run.ts).toLocaleString() + (run.quick ? " · " + t("bm.quick") : ""))),
|
||||
summary,
|
||||
h("div", { class: "table-wrap", style: "margin-top:14px" }, h("table", {}, h("thead", {}, header), h("tbody", {}, ...rows))),
|
||||
h("div", { class: "muted small", style: "margin-top:6px" }, t("bm.cellHint"))),
|
||||
detail);
|
||||
}
|
||||
|
||||
function renderHistory() {
|
||||
const l = history();
|
||||
if (!l.length) {
|
||||
mount(historyHost);
|
||||
return;
|
||||
}
|
||||
const clearAll = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("bm.clearHistory"));
|
||||
clearAll.addEventListener("click", () => {
|
||||
saveHistory([]);
|
||||
renderHistory();
|
||||
});
|
||||
mount(historyHost, h("div", { class: "card" },
|
||||
h("div", { class: "card-head" },
|
||||
h("h3", {}, t("bm.history")),
|
||||
h("div", { class: "right" }, clearAll)),
|
||||
...l.map((run, idx) => {
|
||||
const models = Object.keys(run.models);
|
||||
const scoreOf = (m) => {
|
||||
const rs = run.models[m].results || [];
|
||||
return rs.length ? rs.reduce((a, r) => a + r.score, 0) / rs.length : 0;
|
||||
};
|
||||
const best = models.slice().sort((a, b) => scoreOf(b) - scoreOf(a))[0];
|
||||
const row = h("div", { class: "flex bench-row", style: "cursor:pointer" },
|
||||
h("span", {}, run.mode === "routing" ? "🧭" : "🎯"),
|
||||
h("span", { class: "muted small" }, new Date(run.ts).toLocaleString()),
|
||||
h("span", { class: "grow mono small" }, run.mode === "routing" ? t("bm.mode.routing") : models.join(", ")),
|
||||
run.quick ? h("span", { class: "badge" }, t("bm.quick")) : null,
|
||||
best ? h("b", {}, Math.round(scoreOf(best) * 100) + "%") : null,
|
||||
h("button", { class: "btn sm ghost danger", title: t("common.delete"), onClick: (e) => {
|
||||
e.stopPropagation();
|
||||
l.splice(idx, 1);
|
||||
saveHistory(l);
|
||||
renderHistory();
|
||||
} }, "×"));
|
||||
row.addEventListener("click", () => renderResults(run));
|
||||
return row;
|
||||
})));
|
||||
}
|
||||
|
||||
// ── custom tests (stored in the browser) ──
|
||||
function customCard() {
|
||||
const host = h("div", { class: "card" });
|
||||
function paint() {
|
||||
const list = customTests();
|
||||
const promptIn = h("input", { placeholder: t("bm.promptPh") });
|
||||
const checkIn = h("input", { placeholder: t("bm.checkPh") });
|
||||
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("bm.addTest"));
|
||||
addBtn.addEventListener("click", () => {
|
||||
const prompt = promptIn.value.trim();
|
||||
const check = checkIn.value.split(",").map((s) => s.trim()).filter(Boolean);
|
||||
if (!prompt || !check.length) return;
|
||||
list.push({ id: "custom-" + (list.length + 1), cat: "custom", custom: true, max: 128, prompt, check });
|
||||
saveCustom(list);
|
||||
paint();
|
||||
});
|
||||
mount(host,
|
||||
h("div", { class: "card-title" }, t("bm.custom")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bm.customHint")),
|
||||
...list.map((tst, i) =>
|
||||
h("div", { class: "flex bench-row" },
|
||||
h("span", { class: "grow small" }, tst.prompt.slice(0, 90)),
|
||||
h("span", { class: "muted small mono" }, tst.check.join(", ").slice(0, 40)),
|
||||
h("button", { class: "btn sm ghost danger", onClick: () => { list.splice(i, 1); saveCustom(list); paint(); } }, "×"))),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-top:8px" }, promptIn, checkIn, addBtn));
|
||||
}
|
||||
paint();
|
||||
return host;
|
||||
}
|
||||
|
||||
mount(root,
|
||||
h("div", { class: "page-head" },
|
||||
h("h2", {}, t("bm.title")),
|
||||
h("p", {}, t("bm.subtitle"))),
|
||||
ids.length === 0 ? h("div", { class: "callout" }, t("routing.empty")) : null,
|
||||
h("div", { class: "card" },
|
||||
modeSeg,
|
||||
modeDesc,
|
||||
modelsPick,
|
||||
h("div", { class: "flex", style: "justify-content:space-between; margin-top:14px" },
|
||||
h("label", { class: "check", style: "margin:0", title: t("bm.quickHint") }, quickCb, "⚡ " + t("bm.quickMode")),
|
||||
h("div", { class: "flex", style: "gap:8px" }, stopBtn, runBtn)),
|
||||
h("div", { class: "muted small", style: "margin-top:6px" },
|
||||
t("bm.suiteInfo", { q: SUITE.filter((x) => x.quick).length, f: SUITE.length }))),
|
||||
progressHost,
|
||||
resultsHost,
|
||||
customCard(),
|
||||
historyHost);
|
||||
renderHistory();
|
||||
return root;
|
||||
}
|
||||
Vendored
+115
-10
@@ -1,6 +1,6 @@
|
||||
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
|
||||
import { h, mount, money, num, ms, pct, timeAgo } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { h, mount, money, moneyEl, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { lineChart, barList } from "../charts.js";
|
||||
import { SITE_URL } from "../app.js";
|
||||
@@ -27,14 +27,96 @@ export async function renderDashboard() {
|
||||
let groupby = "model";
|
||||
const root = h("div");
|
||||
|
||||
// ── host metrics: CPU sparkline + RAM / disk bars, refreshed every 2 s ──
|
||||
const sysHost = h("div", {});
|
||||
const cpuHistory = []; // last ~60 samples (2 min)
|
||||
function fmtG(n) {
|
||||
return (n / 1e9).toFixed(1) + " GB";
|
||||
}
|
||||
function bar(frac, cls) {
|
||||
return h("div", { class: "sysbar" }, h("i", { class: cls || "", style: `width:${Math.min(100, frac * 100)}%` }));
|
||||
}
|
||||
function sparkline(values, w, hgt) {
|
||||
const c = document.createElement("canvas");
|
||||
c.width = w;
|
||||
c.height = hgt;
|
||||
const ctx = c.getContext("2d");
|
||||
const accent = getComputedStyle(document.documentElement).getPropertyValue("--accent").trim() || "#5b8def";
|
||||
ctx.strokeStyle = accent;
|
||||
ctx.lineWidth = 1.5;
|
||||
ctx.beginPath();
|
||||
const n = Math.max(values.length, 2);
|
||||
values.forEach((v, i) => {
|
||||
const x = (i / (n - 1)) * (w - 2) + 1;
|
||||
const y = hgt - 2 - (Math.min(v, 100) / 100) * (hgt - 4);
|
||||
if (i === 0) ctx.moveTo(x, y);
|
||||
else ctx.lineTo(x, y);
|
||||
});
|
||||
ctx.stroke();
|
||||
return c;
|
||||
}
|
||||
async function refreshSystem() {
|
||||
let sys;
|
||||
try {
|
||||
sys = await api.system();
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!sys.ts) return;
|
||||
cpuHistory.push(sys.cpu_percent || 0);
|
||||
if (cpuHistory.length > 60) cpuHistory.shift();
|
||||
const ramFrac = sys.ram_total ? sys.ram_used / sys.ram_total : 0;
|
||||
const diskFrac = sys.disk_total ? (sys.disk_total - sys.disk_free) / sys.disk_total : 0;
|
||||
mount(
|
||||
sysHost,
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-title" }, t("dash.sys.title")),
|
||||
h(
|
||||
"div",
|
||||
{ class: "sysgrid" },
|
||||
h("div", { class: "sysitem" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "k grow" }, `CPU · ${sys.cores} ${t("dash.sys.cores")}`),
|
||||
h("span", { class: "v mono" }, (sys.cpu_percent || 0).toFixed(0) + "%")),
|
||||
sparkline(cpuHistory, 220, 36),
|
||||
h("div", { class: "muted small" }, `load ${Number(sys.load1 || 0).toFixed(1)}`)),
|
||||
h("div", { class: "sysitem" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "k grow" }, "RAM"),
|
||||
h("span", { class: "v mono" }, `${fmtG(sys.ram_used)} / ${fmtG(sys.ram_total)}`)),
|
||||
bar(ramFrac, ramFrac > 0.9 ? "bad" : ""),
|
||||
h("div", { class: "muted small" }, t("dash.sys.gateway") + " " + (sys.process_rss / 1e6).toFixed(0) + " MB")),
|
||||
h("div", { class: "sysitem" },
|
||||
h("div", { class: "flex" },
|
||||
h("span", { class: "k grow" }, t("dash.sys.disk")),
|
||||
h("span", { class: "v mono" }, t("dash.sys.free") + " " + fmtG(sys.disk_free)),
|
||||
),
|
||||
bar(diskFrac, diskFrac > 0.92 ? "bad" : ""),
|
||||
h("div", { class: "muted small" }, `${fmtG(sys.disk_total - sys.disk_free)} / ${fmtG(sys.disk_total)}`))
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
const sysTimer = setInterval(() => {
|
||||
if (!root.isConnected) {
|
||||
clearInterval(sysTimer);
|
||||
return;
|
||||
}
|
||||
refreshSystem();
|
||||
}, 2000);
|
||||
refreshSystem();
|
||||
|
||||
async function load() {
|
||||
const rangeSecs = range;
|
||||
let stats, recent, health;
|
||||
let stats, recent, health, balance;
|
||||
try {
|
||||
[stats, recent, health] = await Promise.all([
|
||||
[stats, recent, health, balance] = await Promise.all([
|
||||
api.stats(rangeSecs, groupby),
|
||||
api.requests(40, 0),
|
||||
api.health().catch(() => null),
|
||||
api.balance().catch(() => null),
|
||||
]);
|
||||
} catch (e) {
|
||||
mount(root, h("div", { class: "callout" }, String(e.message || e)));
|
||||
@@ -53,10 +135,24 @@ export async function renderDashboard() {
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
h("div", { class: "grow" }, h("h2", {}, t("dash.title")), h("p", {}, t("dash.subtitle"))),
|
||||
seg(RANGES, range, (r) => {
|
||||
range = r;
|
||||
load();
|
||||
}, (r) => t("dash.range." + r))
|
||||
h("div", { class: "flex", style: "gap:8px;align-items:center" },
|
||||
seg(RANGES, range, (r) => {
|
||||
range = r;
|
||||
load();
|
||||
}, (r) => t("dash.range." + r)),
|
||||
h("button", {
|
||||
class: "btn sm danger",
|
||||
onClick: async () => {
|
||||
if (!(await confirmDialog(t("dash.clearConfirm")))) return;
|
||||
try {
|
||||
await api.clearStats();
|
||||
toast(t("dash.cleared"), "good");
|
||||
await load();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
},
|
||||
}, t("dash.clear")))
|
||||
)
|
||||
),
|
||||
|
||||
@@ -66,12 +162,19 @@ export async function renderDashboard() {
|
||||
{ class: "kpis" },
|
||||
kpi(num(tot.requests), t("dash.kpi.requests")),
|
||||
kpi(num(tot.total_tokens), t("dash.kpi.tokens")),
|
||||
kpi(money(tot.cost_usd), t("dash.kpi.cost")),
|
||||
kpi(moneyEl(tot.cost_usd), t("dash.kpi.cost")),
|
||||
balance && balance.openrouter_usd != null
|
||||
? kpi(
|
||||
h("span", { class: balance.openrouter_usd < 5 ? "err" : "" }, "$" + balance.openrouter_usd.toFixed(2)),
|
||||
t("dash.kpi.orBalance"))
|
||||
: null,
|
||||
kpi(ms(tot.avg_latency_ms), t("dash.kpi.latency")),
|
||||
kpi(pct(tot.success_rate), t("dash.kpi.success")),
|
||||
kpi(num(tot.failovers), t("dash.kpi.failovers"))
|
||||
),
|
||||
|
||||
sysHost,
|
||||
|
||||
h(
|
||||
"div",
|
||||
{ class: "grid cols-2" },
|
||||
@@ -174,7 +277,9 @@ function recentTable(rows) {
|
||||
{},
|
||||
h("td", { class: "mono" }, timeAgo(r.ts)),
|
||||
h("td", {}, r.model_id || "—", r.failover ? h("span", { class: "badge warn", style: "margin-left:6px" }, "fo") : null),
|
||||
h("td", {}, r.tier ? h("span", { class: "badge tier-" + r.tier }, r.tier) : "—"),
|
||||
h("td", {},
|
||||
r.tier ? h("span", { class: "badge tier-" + r.tier }, trTier(r.tier)) : "—",
|
||||
r.score > 0 ? h("span", { class: "muted small mono", style: "margin-left:6px" }, r.score.toFixed(2)) : null),
|
||||
h("td", {}, r.task_label || "—"),
|
||||
h("td", { class: "mono" }, `${r.prompt_tokens}/${r.completion_tokens}`),
|
||||
h("td", { class: "mono" }, money(r.cost_usd)),
|
||||
|
||||
Vendored
+205
@@ -0,0 +1,205 @@
|
||||
// Ready-to-run .cmf models, read live from the infosave HuggingFace account via
|
||||
// /setup/featured — so a newly published model shows up without a code change.
|
||||
// Shared by the first-run wizard and the Import view: same cards, same download
|
||||
// / progress / delete behaviour in both places.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export async function loadFeatured() {
|
||||
try {
|
||||
const r = await api.setupFeatured();
|
||||
return r.models || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export function quantFromVariant(name) {
|
||||
const s = (name || "").toLowerCase();
|
||||
if (s.includes("q1t")) return "Q1T";
|
||||
if (s.includes("q2tp")) return "Q2TP";
|
||||
if (s.includes("q4tp")) return "Q4TP";
|
||||
if (s.includes("q4t")) return "Q4T";
|
||||
if (s.includes("q1")) return "Q1";
|
||||
if (s.includes("q4")) return "Q4";
|
||||
if (s.includes("q8")) return "Q8";
|
||||
return "";
|
||||
}
|
||||
|
||||
// Same thresholds as the backend's format_bytes, so card and job agree.
|
||||
export function fmtBytes(n) {
|
||||
if (!n && n !== 0) return "";
|
||||
if (n >= 1e9) return (n / 1e9).toFixed(1) + " GB";
|
||||
if (n >= 1e6) return Math.round(n / 1e6) + " MB";
|
||||
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
|
||||
return n + " B";
|
||||
}
|
||||
|
||||
export function jobPct(j) {
|
||||
if (j.progress == null) return null;
|
||||
return Math.max(0, Math.min(100, Math.round(j.progress * 100)));
|
||||
}
|
||||
|
||||
/// "downloading · 42% · 1.4 / 3.2 GB" — whatever of it the job actually knows.
|
||||
/// A settled job shows its result, not the phase it stopped in.
|
||||
export function jobStatusText(j) {
|
||||
const parts = [];
|
||||
if (j.state === "running") {
|
||||
if (j.phase) parts.push(j.phase);
|
||||
const pct = jobPct(j);
|
||||
if (pct != null) parts.push(pct + "%");
|
||||
if (j.total_bytes) parts.push(`${fmtBytes(j.done_bytes || 0)} / ${fmtBytes(j.total_bytes)}`);
|
||||
} else if (j.state === "done" && j.size_bytes) {
|
||||
parts.push(fmtBytes(j.size_bytes));
|
||||
}
|
||||
return parts.join(" · ");
|
||||
}
|
||||
|
||||
export function jobBar(j) {
|
||||
const pct = jobPct(j);
|
||||
if (j.state === "done") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill ok", style: "width:100%" }));
|
||||
if (j.state !== "running") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill err", style: "width:100%" }));
|
||||
// known percentage → determinate fill; unknown (listing files) → animated slider
|
||||
return h("div", { class: "pbar" },
|
||||
pct != null
|
||||
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
|
||||
: h("div", { class: "pbar-fill run" })
|
||||
);
|
||||
}
|
||||
|
||||
/// HF gives `desc` as a bare pipeline tag ("text-generation") for live entries and
|
||||
/// a real sentence for curated ones. A one-word tag belongs in the badge row, not
|
||||
/// on a line of its own under every card.
|
||||
function descBadge(desc) {
|
||||
return desc && !desc.includes(" ") ? h("span", { class: "badge" }, desc) : null;
|
||||
}
|
||||
function descLine(desc) {
|
||||
return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null;
|
||||
}
|
||||
|
||||
/// Disk/RAM sanity for a model of `bytes` size against the live system
|
||||
/// snapshot (ctx.sys): running a model needs it in RAM; downloading needs disk.
|
||||
function fitWarnings(bytes, sys) {
|
||||
const out = [];
|
||||
if (!bytes || !sys) return out;
|
||||
if (sys.disk_free && bytes > sys.disk_free)
|
||||
out.push(h("span", { class: "badge bad", title: t("fit.diskTitle") }, "⚠ " + t("fit.disk")));
|
||||
else if (sys.ram_total && bytes > sys.ram_total * 0.75)
|
||||
out.push(h("span", { class: "badge warn", title: t("fit.ramTitle") }, "⚠ " + t("fit.ram")));
|
||||
return out;
|
||||
}
|
||||
|
||||
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v), sys? }.
|
||||
/// Live progress lands in the per-variant host so a poll can repaint it in place,
|
||||
/// without re-rendering the grid under the user's cursor.
|
||||
export function featuredCard(m, ctx) {
|
||||
const badge = m.badge || "CMF";
|
||||
const label = m.label || m.id;
|
||||
const desc = m.desc || "";
|
||||
const variants = m.variants && m.variants.length ? m.variants : null;
|
||||
const size = m.size || (m.size_bytes ? fmtBytes(m.size_bytes) : "");
|
||||
const isDownloaded = !!m.downloaded;
|
||||
const progressHost = (v) => {
|
||||
const host = h("div", { class: "card-prog" });
|
||||
const jid = (v && v.job) || m.job;
|
||||
if (jid) ctx.hosts.set(jid, host);
|
||||
return host;
|
||||
};
|
||||
// multi-variant (e.g. DeepSeek with parts-q2tp / parts-q4tp) → separate buttons per variant
|
||||
if (variants && variants.length > 1) {
|
||||
return h("div", { class: "hf-card featured-card" + (isDownloaded ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, isDownloaded ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
descBadge(desc),
|
||||
...fitWarnings(m.size_bytes, ctx.sys),
|
||||
isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; flex-direction:column; gap:6px; margin-top:8px" },
|
||||
...variants.map((v) => {
|
||||
const q = quantFromVariant(v.name) || badge;
|
||||
const done = !!v.downloaded;
|
||||
const upd = !!v.update_available;
|
||||
const lab = done
|
||||
? (upd ? `↻ ${t("onboarding.update")} ${q} · ${v.size}` : `✓ ${q} · ${v.size}`)
|
||||
: `⬇ ${q} · ${v.size}`;
|
||||
const btn = h("button", {
|
||||
class: "btn sm " + (done && !upd ? "ghost" : "primary"),
|
||||
disabled: done && !upd ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v),
|
||||
title: v.name + (done ? " — " + t("onboarding.downloaded") : ""),
|
||||
}, lab);
|
||||
const row = h("div", { style: "display:flex; gap:6px; align-items:center; flex-wrap:wrap" }, btn);
|
||||
if (done) {
|
||||
const del = h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑");
|
||||
del.addEventListener("click", () => ctx.onDelete(m, v));
|
||||
row.appendChild(del);
|
||||
}
|
||||
return h("div", {}, row, progressHost(v));
|
||||
})
|
||||
)
|
||||
);
|
||||
}
|
||||
const v0 = variants && variants[0];
|
||||
const done0 = !!(v0 && v0.downloaded) || isDownloaded;
|
||||
const upd0 = !!(v0 && v0.update_available) || !!m.update_available;
|
||||
const q0 = quantFromVariant((v0 && v0.name) || "") || badge;
|
||||
const size0 = (v0 && v0.size) || size;
|
||||
const btn0 = h("button", {
|
||||
class: "btn sm " + (done0 && !upd0 ? "ghost" : "primary"),
|
||||
disabled: done0 && !upd0 ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v0),
|
||||
title: done0 && !upd0 ? t("onboarding.downloaded") : ""
|
||||
}, done0
|
||||
? (upd0 ? `↻ ${t("onboarding.update")} ${q0} · ${size0}` : `✓ ${q0} · ${size0}`)
|
||||
: `⬇ ${q0}${size0 ? " · " + size0 : ""}`);
|
||||
const del0 = done0 ? h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete")) : null;
|
||||
if (del0) del0.addEventListener("click", () => ctx.onDelete(m, v0));
|
||||
return h("div", { class: "hf-card featured-card" + (done0 ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, done0 ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null,
|
||||
descBadge(desc),
|
||||
...(done0 ? [] : fitWarnings((v0 && v0.size_bytes) || m.size_bytes, ctx.sys))
|
||||
),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; gap:6px; margin-top:8px; align-items:center; flex-wrap:wrap" }, btn0, del0),
|
||||
progressHost(v0)
|
||||
);
|
||||
}
|
||||
|
||||
/// Repaint the per-variant progress hosts from the current job list.
|
||||
export function paintFeaturedHosts(hosts, jobs, onCancel) {
|
||||
const byId = new Map(jobs.map((j) => [j.id, j]));
|
||||
for (const [id, host] of hosts) {
|
||||
const j = byId.get(id);
|
||||
if (!j || j.state !== "running") { mount(host); continue; }
|
||||
const cancel = h("button", { class: "btn sm ghost", style: "margin-left:auto", title: t("import.cancel") }, "✕");
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await onCancel(j); } catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
mount(host,
|
||||
jobBar(j),
|
||||
h("div", { style: "display:flex; align-items:center; gap:8px" },
|
||||
h("span", { class: "muted small" }, jobStatusText(j)),
|
||||
cancel
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Start the download for a featured variant. `local_name` is the stem the
|
||||
/// gateway itself derived, sent back verbatim so download / progress / delete
|
||||
/// can never disagree about which file this is.
|
||||
export async function startFeaturedDownload(m, v) {
|
||||
const fallbackBase = m.id.split("/").pop().toLowerCase().replace(/[^a-z0-9_-]/g, "-");
|
||||
const name = (v && v.local_name) || m.local_name || fallbackBase;
|
||||
const variant = v && v.name ? v.name : undefined;
|
||||
await api.startImport({ repo: m.id, quant: "Q1", name, variant });
|
||||
return variant;
|
||||
}
|
||||
Vendored
+132
-5
@@ -2,26 +2,36 @@
|
||||
// live progress view. Search HF, pick quantization + advanced params, watch
|
||||
// the conversion (multiple in parallel, resumable across tab switches), then
|
||||
// register the result as a local model.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { h, mount, toast, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobStatusText,
|
||||
} from "./featured.js";
|
||||
|
||||
const QUANTS = [
|
||||
["Q8_2F", "import.q.q8_2f"],
|
||||
["Q8_ROW", "import.q.q8_row"],
|
||||
["Q4_BLOCK", "import.q.q4"],
|
||||
["Q4_TILED", "import.q.q4_tiled"],
|
||||
["VBIT", "import.q.vbit"],
|
||||
["Q1T", "import.q.q1t"],
|
||||
["Q1S", "import.q.q1s"],
|
||||
["Q1P", "import.q.q1p"],
|
||||
["Q1", "import.q.q1"],
|
||||
["F16", "import.q.f16"],
|
||||
["F32", "import.q.f32"],
|
||||
];
|
||||
|
||||
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control,
|
||||
hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
function fmtNum(n) {
|
||||
if (n == null) return "—";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
|
||||
@@ -44,6 +54,10 @@ export function renderImport() {
|
||||
let selected = null;
|
||||
let lastModels = []; // cache of the last search, so re-highlighting needs no refetch
|
||||
let jobsTimer = null;
|
||||
let featured = []; // ready-made .cmf models from the infosave HF account
|
||||
let featuredHosts = new Map(); // job id → live progress slot inside a featured card
|
||||
let lastJobs = [];
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
|
||||
const searchIn = h("input", {
|
||||
class: "hf-search",
|
||||
@@ -125,6 +139,32 @@ export function renderImport() {
|
||||
const meanBitsIn = h("input", { type: "number", step: "0.1", placeholder: "5.0" });
|
||||
const shardIn = h("input", { type: "number", step: "0.5", placeholder: t("import.adv.noShard") });
|
||||
const skipMtp = h("input", { type: "checkbox" });
|
||||
// O(1) attention (cortiq ≥ 0.2.0): constant-memory streaming attention hint,
|
||||
// weights byte-identical — the runtime applies it at load.
|
||||
const o1Sel = h("select", {},
|
||||
opt("", t("import.adv.o1Off")),
|
||||
opt("all", t("import.adv.o1All")),
|
||||
opt("deep", t("import.adv.o1Deep")),
|
||||
opt("custom", t("import.adv.o1Custom")));
|
||||
const o1DeepIn = h("input", { type: "number", min: "1", placeholder: "12" });
|
||||
const o1CustomIn = h("input", { placeholder: "0,4,8" });
|
||||
const o1M = h("input", { type: "number", min: "1", placeholder: "32" });
|
||||
const o1Window = h("input", { type: "number", min: "0", placeholder: "128" });
|
||||
const o1Sink = h("input", { type: "number", min: "0", placeholder: "4" });
|
||||
const o1SpecRow = h("div", { class: "row", style: "display:none" },
|
||||
field(t("import.adv.o1DeepN"), o1DeepIn),
|
||||
field(t("import.adv.o1Layers"), o1CustomIn));
|
||||
const o1Knobs = h("div", { class: "row", style: "display:none" },
|
||||
field("--o1-m", o1M, t("import.adv.o1mHint")),
|
||||
field("--o1-window", o1Window, t("import.adv.o1windowHint")),
|
||||
field("--o1-sink", o1Sink, t("import.adv.o1sinkHint")));
|
||||
o1Sel.addEventListener("change", () => {
|
||||
const on = !!o1Sel.value;
|
||||
o1Knobs.style.display = on ? "" : "none";
|
||||
o1SpecRow.style.display = o1Sel.value === "deep" || o1Sel.value === "custom" ? "" : "none";
|
||||
o1DeepIn.parentElement.style.display = o1Sel.value === "deep" ? "" : "none";
|
||||
o1CustomIn.parentElement.style.display = o1Sel.value === "custom" ? "" : "none";
|
||||
});
|
||||
const advBody = h(
|
||||
"div",
|
||||
{ class: "adv-body", style: "display:none" },
|
||||
@@ -137,6 +177,9 @@ export function renderImport() {
|
||||
h("div", { class: "row" },
|
||||
field(t("import.adv.shard"), shardIn),
|
||||
field("MTP", h("label", { class: "check" }, skipMtp, t("import.adv.skipMtp")))),
|
||||
field(t("import.adv.o1"), o1Sel, t("import.adv.o1Hint")),
|
||||
o1SpecRow,
|
||||
o1Knobs,
|
||||
);
|
||||
const advToggle = h("button", { class: "btn ghost sm" }, "⚙ " + t("import.adv.title"));
|
||||
advToggle.addEventListener("click", () => {
|
||||
@@ -156,6 +199,19 @@ export function renderImport() {
|
||||
if (vbitSel.value) params.vbit_shape = vbitSel.value;
|
||||
if (meanBitsIn.value) params.mean_bits = parseFloat(meanBitsIn.value);
|
||||
if (shardIn.value) params.shard_max_gb = parseFloat(shardIn.value);
|
||||
if (o1Sel.value) {
|
||||
// spec: all | deepN | explicit layer list
|
||||
params.o1 =
|
||||
o1Sel.value === "deep" ? "deep" + (parseInt(o1DeepIn.value) || 12)
|
||||
: o1Sel.value === "custom" ? o1CustomIn.value.trim()
|
||||
: "all";
|
||||
if (!params.o1) delete params.o1;
|
||||
else {
|
||||
if (o1M.value) params.o1_m = parseInt(o1M.value);
|
||||
if (o1Window.value) params.o1_window = parseInt(o1Window.value);
|
||||
if (o1Sink.value) params.o1_sink = parseInt(o1Sink.value);
|
||||
}
|
||||
}
|
||||
goBtn.disabled = true;
|
||||
try {
|
||||
await api.startImport(params);
|
||||
@@ -203,6 +259,12 @@ export function renderImport() {
|
||||
}
|
||||
// Load + render all jobs. Runs on mount (before `root` is attached — so no
|
||||
// isConnected guard here) and on each poll tick.
|
||||
async function cancelJob(j) {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "warn");
|
||||
await refreshJobs();
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
let jobs = [];
|
||||
try {
|
||||
@@ -211,7 +273,14 @@ export function renderImport() {
|
||||
return;
|
||||
}
|
||||
jobs = jobs.filter((j) => j.state !== "cancelled"); // hide user-cancelled jobs
|
||||
lastJobs = jobs;
|
||||
renderJobs(jobs);
|
||||
// A settled download changes what the featured cards offer (✓ + delete), so
|
||||
// re-read the list on that transition — not on every poll tick.
|
||||
const settled = jobs.some((j) => jobStates.get(j.id) === "running" && j.state !== "running");
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) await renderFeatured();
|
||||
else paintFeaturedHosts(featuredHosts, jobs, cancelJob);
|
||||
if (jobs.some((j) => j.state === "running")) ensurePolling();
|
||||
else stopPolling();
|
||||
}
|
||||
@@ -238,7 +307,7 @@ export function renderImport() {
|
||||
(running && pct != null ? " · " + pct + "%" : "") +
|
||||
(j.size_bytes ? " · " + fmtBytes(j.size_bytes) : "")
|
||||
);
|
||||
const phase = h("div", { class: "muted small" }, j.phase || "");
|
||||
const phase = h("div", { class: "muted small" }, jobStatusText(j));
|
||||
const logEl = h("pre", { class: "conv-log" }, (j.log || []).slice(-10).join("\n"));
|
||||
|
||||
const actions = [];
|
||||
@@ -316,6 +385,61 @@ export function renderImport() {
|
||||
);
|
||||
}
|
||||
|
||||
// ── featured: the infosave account, read live so new releases just show up ──
|
||||
const featuredGrid = h("div", { class: "hf-grid" }, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
const reloadBtn = h("button", { class: "btn ghost sm" }, "↻ " + t("common.refresh"));
|
||||
const featuredHost = h(
|
||||
"div",
|
||||
{ class: "card featured-section" },
|
||||
h("div", { class: "card-head", style: "display:flex; align-items:center; gap:10px" },
|
||||
h("div", { class: "card-title", style: "margin:0" }, "⚡ " + t("import.featuredTitle")),
|
||||
h("div", { style: "flex:1" }),
|
||||
reloadBtn),
|
||||
h("div", { class: "muted small", style: "margin:2px 0 10px" }, t("import.featuredSub")),
|
||||
featuredGrid
|
||||
);
|
||||
|
||||
async function startFeatured(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function deleteFeatured(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!window.confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await renderFeatured();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function renderFeatured(reload = true) {
|
||||
if (reload) featured = await loadFeatured();
|
||||
let sys = null;
|
||||
try { sys = await api.system(); } catch { /* metrics optional */ }
|
||||
featuredHosts = new Map();
|
||||
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured, sys };
|
||||
mount(featuredGrid, ...(featured.length
|
||||
? featured.map((m) => featuredCard(m, ctx))
|
||||
: [h("div", { class: "muted pad" }, t("import.noResults"))]));
|
||||
paintFeaturedHosts(featuredHosts, lastJobs, cancelJob);
|
||||
}
|
||||
reloadBtn.addEventListener("click", async () => {
|
||||
reloadBtn.disabled = true;
|
||||
mount(featuredGrid, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
await renderFeatured();
|
||||
reloadBtn.disabled = false;
|
||||
});
|
||||
|
||||
const root = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -323,10 +447,13 @@ export function renderImport() {
|
||||
h("h2", {}, t("nav.import")),
|
||||
h("div", { class: "muted" }, t("import.subtitle"))),
|
||||
jobsHost, // progress panel on top, above the search
|
||||
featuredHost, // featured ready-to-run models
|
||||
h("div", { class: "card" }, searchIn, results),
|
||||
formHost
|
||||
);
|
||||
doSearch(); // initial trending list (mounts into `results`)
|
||||
renderFeatured(); // ready-made .cmf models, fetched live
|
||||
refreshJobs(); // resume: show any running/recent conversions immediately
|
||||
return root;
|
||||
}
|
||||
|
||||
|
||||
Vendored
+329
-14
@@ -1,30 +1,23 @@
|
||||
// API keys — list, create (with copy-once), revoke.
|
||||
import { h, mount, modal, toast, confirmDialog } from "../ui.js";
|
||||
import { h, mount, modal, toast, confirmDialog, moneyEl, num } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export async function renderKeys() {
|
||||
const modelsData = await api.listModels();
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
const root = h("div");
|
||||
let usageRange = "7d";
|
||||
|
||||
function openCreate(reload) {
|
||||
const accountIn = h("input", { placeholder: "team-a" });
|
||||
const rateIn = h("input", { type: "number", value: 600 });
|
||||
const budgetIn = h("input", { type: "number", step: "1", min: "0", placeholder: "0" });
|
||||
const customIn = h("input", { placeholder: "sk-gw-…" });
|
||||
const allowInputs = ids.map((id) => ({ id, cb: h("input", { type: "checkbox" }) }));
|
||||
const body = h(
|
||||
"div",
|
||||
{},
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.account")), accountIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.rate")), rateIn),
|
||||
h(
|
||||
"label",
|
||||
{ class: "field" },
|
||||
h("span", {}, t("keys.form.allow")),
|
||||
h("div", { class: "flex wrap" }, ...allowInputs.map(({ id, cb }) => h("label", { class: "check" }, cb, id))),
|
||||
h("div", { class: "hint" }, t("keys.form.allowHint"))
|
||||
),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.budget")), budgetIn, h("div", { class: "hint" }, t("keys.form.budgetHint"))),
|
||||
h("label", { class: "field" }, h("span", {}, t("keys.form.custom")), customIn, h("div", { class: "hint" }, t("keys.form.customHint")))
|
||||
);
|
||||
modal(t("keys.form.title"), body, async () => {
|
||||
@@ -33,7 +26,8 @@ export async function renderKeys() {
|
||||
key: customIn.value.trim(),
|
||||
account: accountIn.value.trim() || "default",
|
||||
rate_per_min: parseInt(rateIn.value) || 0,
|
||||
allow_models: allowInputs.filter(({ cb }) => cb.checked).map(({ id }) => id),
|
||||
monthly_budget_usd: parseFloat(budgetIn.value) > 0 ? parseFloat(budgetIn.value) : null,
|
||||
allow_models: [],
|
||||
});
|
||||
reload();
|
||||
showCreated(r.key);
|
||||
@@ -48,11 +42,310 @@ export async function renderKeys() {
|
||||
const input = h("input", { value: key, readonly: true, class: "mono" });
|
||||
const copyBtn = h("button", { class: "btn", onClick: () => { input.select(); navigator.clipboard?.writeText(key); toast(t("common.copied"), "good"); } }, t("common.copy"));
|
||||
modal(t("common.ok"), h("div", {}, h("p", {}, t("keys.created")), h("div", { class: "flex" }, input, copyBtn)), null);
|
||||
lastCreatedKey = key;
|
||||
reload();
|
||||
}
|
||||
|
||||
let lastCreatedKey = null;
|
||||
|
||||
// "Connect your app" — copy-paste snippets with this gateway's real URL.
|
||||
// Keys are stored hashed-masked in the list, so the key itself is a
|
||||
// placeholder unless one was just created on this page.
|
||||
function snippetsCard(openMode) {
|
||||
const origin = location.origin;
|
||||
const base = origin + "/v1";
|
||||
let key = lastCreatedKey || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
// Every scripty snippet exists in two OS flavors: bash heredocs for
|
||||
// Linux/macOS and PowerShell here-strings for Windows. SDK snippets
|
||||
// (python/js) and GUI instructions are OS-independent.
|
||||
let os = /win/i.test(navigator.platform || "") ? "ps" : "sh";
|
||||
// Harness auto-setup scripts first (the gateway speaks OpenAI at /v1/* and
|
||||
// Anthropic at /v1/messages, so Claude Code connects natively), SDKs after.
|
||||
const buildSnips = (osKind) => osKind === "ps" ? {
|
||||
"Claude Code": `$env:ANTHROPIC_BASE_URL = "${origin}"
|
||||
$env:ANTHROPIC_AUTH_TOKEN = "${key}"
|
||||
$env:ANTHROPIC_MODEL = "cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `New-Item -ItemType Directory -Force "$HOME\\.codex" | Out-Null
|
||||
Add-Content "$HOME\\.codex\\config.toml" @'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
[model_providers.cortiq]
|
||||
name = "Cortiq Gateway"
|
||||
base_url = "${base}"
|
||||
env_key = "CORTIQ_API_KEY"
|
||||
wire_api = "chat"
|
||||
'@
|
||||
$env:CORTIQ_API_KEY = "${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `New-Item -ItemType Directory -Force "$HOME\\.config\\opencode" | Out-Null
|
||||
Set-Content "$HOME\\.config\\opencode\\opencode.json" @'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"cortiq": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Cortiq Gateway",
|
||||
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
opencode`,
|
||||
Continue: `New-Item -ItemType Directory -Force "$HOME\\.continue" | Out-Null
|
||||
Add-Content "$HOME\\.continue\\config.yaml" @'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
model: cortiq-auto
|
||||
apiBase: ${base}
|
||||
apiKey: ${key}
|
||||
'@`,
|
||||
Cline: `${t("keys.h.cline1")}
|
||||
API Provider: OpenAI Compatible
|
||||
Base URL: ${base}
|
||||
API Key: ${key}
|
||||
Model ID: cortiq-auto
|
||||
|
||||
${t("keys.h.cline2")}`,
|
||||
curl: `curl.exe ${base}/chat/completions \`
|
||||
-H "Authorization: Bearer ${key}" \`
|
||||
-H "Content-Type: application/json" \`
|
||||
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||
} : {
|
||||
"Claude Code": `export ANTHROPIC_BASE_URL="${origin}"
|
||||
export ANTHROPIC_AUTH_TOKEN="${key}"
|
||||
export ANTHROPIC_MODEL="cortiq-auto"
|
||||
|
||||
claude`,
|
||||
Codex: `mkdir -p ~/.codex && cat >> ~/.codex/config.toml <<'EOF'
|
||||
model = "cortiq-auto"
|
||||
model_provider = "cortiq"
|
||||
|
||||
[model_providers.cortiq]
|
||||
name = "Cortiq Gateway"
|
||||
base_url = "${base}"
|
||||
env_key = "CORTIQ_API_KEY"
|
||||
wire_api = "chat"
|
||||
EOF
|
||||
export CORTIQ_API_KEY="${key}"
|
||||
|
||||
codex`,
|
||||
OpenCode: `mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json <<'EOF'
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"cortiq": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Cortiq Gateway",
|
||||
"options": { "baseURL": "${base}", "apiKey": "${key}" },
|
||||
"models": { "cortiq-auto": { "name": "Cortiq Auto (smart routing)" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
opencode`,
|
||||
Continue: `mkdir -p ~/.continue && cat >> ~/.continue/config.yaml <<'EOF'
|
||||
models:
|
||||
- name: Cortiq Auto
|
||||
provider: openai
|
||||
model: cortiq-auto
|
||||
apiBase: ${base}
|
||||
apiKey: ${key}
|
||||
EOF`,
|
||||
Cline: `${t("keys.h.cline1")}
|
||||
API Provider: OpenAI Compatible
|
||||
Base URL: ${base}
|
||||
API Key: ${key}
|
||||
Model ID: cortiq-auto
|
||||
|
||||
${t("keys.h.cline2")}`,
|
||||
curl: `curl ${base}/chat/completions \\
|
||||
-H "Authorization: Bearer ${key}" \\
|
||||
-H "Content-Type: application/json" \\
|
||||
-d '{"model":"cortiq-auto","messages":[{"role":"user","content":"Привет!"}]}'`,
|
||||
python: `from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="${base}", api_key="${key}")
|
||||
resp = client.chat.completions.create(
|
||||
model="cortiq-auto", # авто-роутинг; или точное имя модели
|
||||
messages=[{"role": "user", "content": "Привет!"}],
|
||||
)
|
||||
print(resp.choices[0].message.content)`,
|
||||
javascript: `import OpenAI from "openai";
|
||||
|
||||
const client = new OpenAI({ baseURL: "${base}", apiKey: "${key}" });
|
||||
const resp = await client.chat.completions.create({
|
||||
model: "cortiq-auto",
|
||||
messages: [{ role: "user", content: "Привет!" }],
|
||||
});
|
||||
console.log(resp.choices[0].message.content);`,
|
||||
};
|
||||
let shSnips, psSnips, snips;
|
||||
function rebuildSnips() {
|
||||
shSnips = buildSnips("sh");
|
||||
psSnips = buildSnips("ps");
|
||||
// python/js/Cline одинаковы — дополняем ps-набор из sh
|
||||
for (const k of Object.keys(shSnips)) if (!psSnips[k]) psSnips[k] = shSnips[k];
|
||||
snips = os === "ps" ? psSnips : shSnips;
|
||||
}
|
||||
rebuildSnips();
|
||||
let cur = "Claude Code";
|
||||
// per-harness hint lives OUTSIDE the copyable text: a leading "# comment"
|
||||
// in a pasted snippet is not a comment in default interactive zsh, and a
|
||||
// "!" inside it triggers history expansion ("event not found")
|
||||
const HINTS = {
|
||||
"Claude Code": () => t("keys.h.claude"),
|
||||
Codex: () => t("keys.h.codex"),
|
||||
OpenCode: () => t("keys.h.opencode"),
|
||||
Continue: () => t("keys.h.continue"),
|
||||
};
|
||||
const hintHost = h("div", { class: "muted small", style: "margin-bottom:6px" });
|
||||
const paintHint = () => { hintHost.textContent = HINTS[cur] ? HINTS[cur]() : ""; };
|
||||
const pre = h("pre", { class: "conv-log snippet" }, snips[cur]);
|
||||
paintHint();
|
||||
const seg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
...Object.keys(snips).map((k, i) =>
|
||||
h("button", { class: (i === 0 ? "active" : "") + (["curl", "python", "javascript"].includes(k) ? " sdk" : ""), "data-snip": k, onClick: (e) => {
|
||||
cur = k;
|
||||
pre.textContent = snips[k];
|
||||
paintHint();
|
||||
protoWarn.style.display = k === "Claude Code" && protoWarn.childNodes.length ? "" : "none";
|
||||
paintStatus();
|
||||
seg.querySelectorAll("button").forEach((b) => b.classList.remove("active"));
|
||||
e.currentTarget.classList.add("active");
|
||||
} }, k)
|
||||
)
|
||||
);
|
||||
// Claude Code needs the anthropic_messages protocol — offer a one-click
|
||||
// enable when it is off, right where the user will hit the problem.
|
||||
const protoWarn = h("div", { style: "display:none; margin-top:8px" });
|
||||
api.getProtocols().then((prot) => {
|
||||
if (prot.anthropic_messages) return;
|
||||
const fixBtn = h("button", { class: "btn sm primary" }, t("keys.h.enableProto"));
|
||||
fixBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.putProtocols({ ...prot, anthropic_messages: true });
|
||||
protoWarn.style.display = "none";
|
||||
toast(t("common.saved"), "ok");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
});
|
||||
mount(protoWarn, h("div", { class: "callout warn" }, h("span", {}, "⚠ " + t("keys.h.protoOff")), " ", fixBtn));
|
||||
if (cur === "Claude Code") protoWarn.style.display = "";
|
||||
}).catch(() => {});
|
||||
|
||||
// Detected harnesses: ✓ on the tab when installed on the gateway machine,
|
||||
// status line under the snippet ("installed" / "already configured") —
|
||||
// and a one-line INSTALL command (per OS) when it's missing.
|
||||
const INSTALL = {
|
||||
"Claude Code": { sh: "curl -fsSL https://claude.ai/install.sh | bash", ps: "irm https://claude.ai/install.ps1 | iex" },
|
||||
Codex: { sh: "npm install -g @openai/codex", ps: "npm install -g @openai/codex" },
|
||||
OpenCode: { sh: "curl -fsSL https://opencode.ai/install | bash", ps: "npm install -g opencode-ai" },
|
||||
Continue: { sh: "code --install-extension Continue.continue", ps: "code --install-extension Continue.continue" },
|
||||
Cline: { sh: "code --install-extension saoudrizwan.claude-dev", ps: "code --install-extension saoudrizwan.claude-dev" },
|
||||
};
|
||||
let detected = {};
|
||||
const statusHost = h("div", { class: "small", style: "margin-top:6px" });
|
||||
function paintStatus() {
|
||||
const d = detected[cur];
|
||||
if (!d) {
|
||||
statusHost.textContent = "";
|
||||
return;
|
||||
}
|
||||
statusHost.innerHTML = "";
|
||||
if (d.configured) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.configured")));
|
||||
else if (d.installed) mount(statusHost, h("span", { class: "ok" }, "✓ " + t("keys.h.st.installed")));
|
||||
else {
|
||||
const cmd = (INSTALL[cur] || {})[os];
|
||||
const copyInstall = h("button", { class: "btn ghost sm", onClick: async () => {
|
||||
await navigator.clipboard.writeText(cmd);
|
||||
toast(t("common.copied"), "good");
|
||||
} }, "⧉");
|
||||
mount(statusHost,
|
||||
h("span", { class: "muted" }, "○ " + t("keys.h.st.notFound")),
|
||||
cmd ? h("div", { class: "flex", style: "gap:6px; align-items:center; margin-top:4px" },
|
||||
h("span", { class: "muted" }, t("keys.h.install")),
|
||||
h("code", { class: "mono small cmd-chip" }, cmd),
|
||||
copyInstall) : null);
|
||||
}
|
||||
}
|
||||
api.harnesses().then((d) => {
|
||||
detected = d || {};
|
||||
seg.querySelectorAll("button").forEach((b) => {
|
||||
const st = detected[b.dataset.snip];
|
||||
if (st && st.installed) b.append(" ✓");
|
||||
});
|
||||
paintStatus();
|
||||
}).catch(() => {});
|
||||
|
||||
// OS switch: bash for Linux/macOS, PowerShell for Windows (auto-detected)
|
||||
const osSeg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
h("button", { class: os === "sh" ? "active" : "" }, "🐧🍎 Linux / macOS"),
|
||||
h("button", { class: os === "ps" ? "active" : "" }, "🪟 Windows")
|
||||
);
|
||||
osSeg.querySelectorAll("button").forEach((b, i) =>
|
||||
b.addEventListener("click", () => {
|
||||
os = i === 0 ? "sh" : "ps";
|
||||
rebuildSnips();
|
||||
osSeg.querySelectorAll("button").forEach((x, j) => x.classList.toggle("active", j === (os === "sh" ? 0 : 1)));
|
||||
pre.textContent = snips[cur];
|
||||
paintStatus();
|
||||
}));
|
||||
|
||||
// paste-your-key: substitutes the key into every snippet immediately
|
||||
const keyIn = h("input", { class: "mono", placeholder: t("keys.snip.keyInputPh"), style: "max-width:230px; font-size:12px", value: lastCreatedKey || "" });
|
||||
const applyKey = () => {
|
||||
key = keyIn.value.trim() || (openMode ? "sk-anything" : "<" + t("keys.snip.keyPh") + ">");
|
||||
rebuildSnips();
|
||||
pre.textContent = snips[cur];
|
||||
};
|
||||
const applyBtn = h("button", { class: "btn ghost sm", onClick: applyKey }, "⤵ " + t("keys.snip.apply"));
|
||||
keyIn.addEventListener("keydown", (e) => { if (e.key === "Enter") applyKey(); });
|
||||
|
||||
const copy = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||
copy.addEventListener("click", async () => {
|
||||
await navigator.clipboard.writeText(pre.textContent);
|
||||
toast(t("common.copied"), "ok");
|
||||
});
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-title" }, "🔌 " + t("keys.snip.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" },
|
||||
lastCreatedKey ? t("keys.snip.withKey") : openMode ? t("keys.snip.openMode") : t("keys.snip.hint")),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-bottom:8px; flex-wrap:wrap" }, seg, h("div", { class: "grow" }), keyIn, applyBtn, osSeg, copy),
|
||||
hintHost,
|
||||
pre,
|
||||
statusHost,
|
||||
protoWarn,
|
||||
h("div", { class: "muted small", style: "margin-top:6px" }, t("keys.h.note"))
|
||||
);
|
||||
}
|
||||
|
||||
async function reload() {
|
||||
const data = await api.listKeys();
|
||||
const keys = data.keys || [];
|
||||
// per-account usage: window breakdown + cumulative-forever totals — the
|
||||
// "who spends what" view for departments/developers sharing one gateway
|
||||
let usage = {};
|
||||
let alltime = {};
|
||||
try {
|
||||
const st = await api.stats(usageRange, "account");
|
||||
for (const b of st.breakdown || []) usage[b.key] = b;
|
||||
alltime = st.alltime_by_account || {};
|
||||
} catch { /* stats disabled — columns show dashes */ }
|
||||
mount(
|
||||
root,
|
||||
h(
|
||||
@@ -62,6 +355,9 @@ export async function renderKeys() {
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
h("div", { class: "grow" }, h("h2", {}, t("keys.title")), h("p", {}, t("keys.subtitle"))),
|
||||
h("div", { class: "seg", style: "margin-right:10px" },
|
||||
...["24h", "7d", "30d"].map((r) =>
|
||||
h("button", { class: r === usageRange ? "active" : "", onClick: () => { usageRange = r; reload(); } }, r))),
|
||||
h("button", { class: "btn primary", onClick: () => openCreate(reload) }, "+ " + t("keys.add"))
|
||||
)
|
||||
),
|
||||
@@ -80,7 +376,16 @@ export async function renderKeys() {
|
||||
h(
|
||||
"thead",
|
||||
{},
|
||||
h("tr", {}, h("th", {}, t("keys.col.key")), h("th", {}, t("keys.col.account")), h("th", {}, t("keys.col.rate")), h("th", {}, t("keys.col.models")), h("th", { class: "right-align" }, t("common.actions")))
|
||||
h("tr", {},
|
||||
h("th", {}, t("keys.col.key")),
|
||||
h("th", {}, t("keys.col.account")),
|
||||
h("th", {}, t("keys.col.rate")),
|
||||
h("th", {}, t("keys.col.models")),
|
||||
h("th", { class: "right-align" }, t("keys.col.requests") + " (" + usageRange + ")"),
|
||||
h("th", { class: "right-align" }, t("keys.col.tokens") + " (" + usageRange + ")"),
|
||||
h("th", { class: "right-align" }, "$ (" + usageRange + ")"),
|
||||
h("th", { class: "right-align" }, t("keys.col.costAll")),
|
||||
h("th", { class: "right-align" }, t("common.actions")))
|
||||
),
|
||||
h(
|
||||
"tbody",
|
||||
@@ -93,6 +398,15 @@ export async function renderKeys() {
|
||||
h("td", {}, k.account),
|
||||
h("td", { class: "mono" }, k.rate_per_min || "—"),
|
||||
h("td", {}, (k.allow_models && k.allow_models.length) ? k.allow_models.join(", ") : h("span", { class: "muted" }, t("keys.allowAll"))),
|
||||
h("td", { class: "right-align mono" }, String((usage[k.account] || {}).requests ?? "—")),
|
||||
h("td", { class: "right-align mono small" }, usage[k.account] ? num(usage[k.account].prompt_tokens) + " / " + num(usage[k.account].completion_tokens) : "—"),
|
||||
h("td", { class: "right-align" }, usage[k.account] ? moneyEl(usage[k.account].cost_usd) : "—"),
|
||||
h("td", { class: "right-align" },
|
||||
alltime[k.account] ? moneyEl(alltime[k.account].cost_usd) : "—",
|
||||
k.monthly_budget_usd
|
||||
? h("div", { class: "small " + (((usage[k.account] || {}).cost_usd || 0) >= k.monthly_budget_usd ? "err" : "muted") },
|
||||
t("keys.budgetOf", { b: "$" + k.monthly_budget_usd }))
|
||||
: null),
|
||||
h(
|
||||
"td",
|
||||
{ class: "right-align" },
|
||||
@@ -119,7 +433,8 @@ export async function renderKeys() {
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
),
|
||||
snippetsCard(!!data.open_mode)
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
Vendored
+95
@@ -0,0 +1,95 @@
|
||||
// Gateway log viewer — the in-memory ring served by /admin/api/logs, for
|
||||
// everyone who never sees the terminal (Docker, autostart, double-clicked exe).
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export function renderLogs() {
|
||||
const root = h("div");
|
||||
let timer = null;
|
||||
let paused = false;
|
||||
let filter = "";
|
||||
let level = "";
|
||||
|
||||
const pre = h("pre", { class: "conv-log logs-view" }, t("common.loading"));
|
||||
const filterIn = h("input", { placeholder: t("logs.filterPlaceholder"), style: "max-width:260px" });
|
||||
const levelSel = h(
|
||||
"select",
|
||||
{ style: "max-width:130px" },
|
||||
h("option", { value: "" }, t("common.all")),
|
||||
...["ERROR", "WARN", "INFO", "DEBUG"].map((l) => h("option", { value: l }, l))
|
||||
);
|
||||
const pauseBtn = h("button", { class: "btn ghost sm" }, "⏸ " + t("logs.pause"));
|
||||
const copyBtn = h("button", { class: "btn ghost sm" }, "⧉ " + t("common.copy"));
|
||||
const clearBtn = h("button", { class: "btn ghost sm danger" }, "🗑 " + t("dash.clear"));
|
||||
clearBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.clearLogs();
|
||||
lines = [];
|
||||
paint();
|
||||
toast(t("dash.cleared"), "ok");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
});
|
||||
|
||||
let lines = [];
|
||||
function paint() {
|
||||
let shown = lines;
|
||||
if (level) shown = shown.filter((l) => l.includes(` ${level} `) || l.includes(`${level} `));
|
||||
if (filter) shown = shown.filter((l) => l.toLowerCase().includes(filter));
|
||||
pre.textContent = shown.length ? shown.join("\n") : t("logs.empty");
|
||||
if (!paused) pre.scrollTop = pre.scrollHeight;
|
||||
}
|
||||
async function refresh() {
|
||||
if (paused) return;
|
||||
try {
|
||||
lines = (await api.logs(500)).lines || [];
|
||||
paint();
|
||||
} catch {
|
||||
/* transient */
|
||||
}
|
||||
}
|
||||
filterIn.addEventListener("input", () => {
|
||||
filter = filterIn.value.trim().toLowerCase();
|
||||
paint();
|
||||
});
|
||||
levelSel.addEventListener("change", () => {
|
||||
level = levelSel.value;
|
||||
paint();
|
||||
});
|
||||
pauseBtn.addEventListener("click", () => {
|
||||
paused = !paused;
|
||||
pauseBtn.textContent = (paused ? "▶ " : "⏸ ") + t(paused ? "logs.resume" : "logs.pause");
|
||||
if (!paused) refresh();
|
||||
});
|
||||
copyBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await navigator.clipboard.writeText(pre.textContent);
|
||||
toast(t("common.copied"), "ok");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
});
|
||||
|
||||
timer = setInterval(() => {
|
||||
if (!root.isConnected) {
|
||||
clearInterval(timer);
|
||||
return;
|
||||
}
|
||||
refresh();
|
||||
}, 2000);
|
||||
refresh();
|
||||
|
||||
mount(
|
||||
root,
|
||||
h("div", { class: "view-head" }, h("h2", {}, t("nav.logs")), h("div", { class: "muted" }, t("logs.subtitle"))),
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "flex", style: "gap:8px; flex-wrap:wrap; margin-bottom:10px" }, filterIn, levelSel, pauseBtn, copyBtn, clearBtn),
|
||||
pre
|
||||
)
|
||||
);
|
||||
return root;
|
||||
}
|
||||
Vendored
+173
-18
@@ -1,5 +1,5 @@
|
||||
// Models — pool table with add/edit modal, probe, and per-model secret entry.
|
||||
import { h, mount, modal, toast, confirmDialog, money } from "../ui.js";
|
||||
import { h, mount, modal, toast, confirmDialog, money, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
@@ -7,9 +7,6 @@ import { appState } from "../app.js";
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
function keyBadge(src) {
|
||||
const map = { store: "store", env: "env", missing: "bad", none: "" };
|
||||
@@ -26,6 +23,13 @@ const PROVIDER_DEFAULTS = {
|
||||
http: { base: "", keyEnv: "", needsKey: false, tier: "cheap", caps: ["tools"] },
|
||||
};
|
||||
|
||||
// Derive a clean model id from a model name (last path segment, slugified).
|
||||
// e.g. "qwen/qwen3.7-max" → "qwen3-7-max".
|
||||
function slugId(name) {
|
||||
const base = (name || "").split("/").pop() || "";
|
||||
return base.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "");
|
||||
}
|
||||
|
||||
function modelForm(meta, existing) {
|
||||
const m = existing || { provider: "openai", kind: "chat", cost_tier: "mid", caps: ["tools"], price_in: 0, price_out: 0 };
|
||||
const idIn = h("input", { value: m.id || "", placeholder: "my-model", disabled: existing ? true : null });
|
||||
@@ -47,7 +51,16 @@ function modelForm(meta, existing) {
|
||||
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
|
||||
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
|
||||
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
|
||||
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
|
||||
const CAP_EMOJIS = {
|
||||
tools: "🔧 tools",
|
||||
vision: "👁️ vision",
|
||||
audio: "🎙️ audio",
|
||||
reasoning: "🧠 reasoning",
|
||||
json: "📄 json",
|
||||
caching: "⚡ caching"
|
||||
};
|
||||
|
||||
const capInputs = (meta.caps || Object.keys(CAP_EMOJIS)).map((c) => {
|
||||
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
|
||||
return { c, cb };
|
||||
});
|
||||
@@ -66,14 +79,55 @@ function modelForm(meta, existing) {
|
||||
capInputs.forEach(({ c, cb }) => { cb.checked = d.caps.includes(c); });
|
||||
keyIn.placeholder = d.needsKey ? t("models.form.apiKeyPlaceholder") : t("models.form.apiKeyNotNeeded");
|
||||
mount(keyStatus, "");
|
||||
// Automatically pre-fetch models (especially useful for OpenRouter which doesn't need a key)
|
||||
testKey(true);
|
||||
}
|
||||
providerSel.addEventListener("change", applyDefaults);
|
||||
if (!existing) applyDefaults();
|
||||
|
||||
async function testKey() {
|
||||
if (!baseUrlIn.value.trim()) { mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?")); return; }
|
||||
let cachedModels = [];
|
||||
|
||||
const autofillData = () => {
|
||||
console.log("autofillData triggered. modelIn.value:", modelIn.value);
|
||||
console.log("cachedModels length:", cachedModels.length);
|
||||
if (!existing && (!idIn.value || idIn._auto)) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
console.log("auto-filled idIn:", idIn.value);
|
||||
}
|
||||
const found = cachedModels.find((m) => (m.id || m) === modelIn.value);
|
||||
console.log("found model:", found);
|
||||
if (found) {
|
||||
if (found.price_in !== undefined) {
|
||||
priceInIn.value = parseFloat(Number(found.price_in).toFixed(4));
|
||||
priceOutIn.value = parseFloat(Number(found.price_out).toFixed(4));
|
||||
console.log("Updated prices:", priceInIn.value, priceOutIn.value);
|
||||
}
|
||||
if (found.caps && Array.isArray(found.caps)) {
|
||||
capInputs.forEach(({ c, cb }) => {
|
||||
cb.checked = found.caps.includes(c);
|
||||
});
|
||||
console.log("Updated caps:", found.caps);
|
||||
}
|
||||
}
|
||||
};
|
||||
modelIn.addEventListener("input", autofillData);
|
||||
modelIn.addEventListener("change", autofillData);
|
||||
|
||||
if (!existing) {
|
||||
idIn.addEventListener("input", () => { idIn._auto = false; });
|
||||
} else {
|
||||
console.log("Editing existing model, fetching testKey(true)");
|
||||
testKey(true);
|
||||
}
|
||||
|
||||
async function testKey(silent = false) {
|
||||
if (!baseUrlIn.value.trim()) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?"));
|
||||
return;
|
||||
}
|
||||
testBtn.disabled = true;
|
||||
mount(keyStatus, t("models.form.testing"));
|
||||
if (!silent) mount(keyStatus, t("models.form.testing"));
|
||||
try {
|
||||
const r = await api.providerModels({
|
||||
provider: providerSel.value,
|
||||
@@ -82,24 +136,61 @@ function modelForm(meta, existing) {
|
||||
api_key_env: envFor(),
|
||||
});
|
||||
if (r.ok) {
|
||||
dl.replaceChildren(...(r.models || []).map((mm) => h("option", { value: mm })));
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, "✓ " + t("models.form.foundModels", { n: r.count })));
|
||||
modelIn.focus();
|
||||
cachedModels = r.models || [];
|
||||
dl.replaceChildren(...cachedModels.map((mm) => h("option", { value: mm.id || mm })));
|
||||
|
||||
let msg = "✓ " + t("models.form.foundModels", { n: r.count });
|
||||
if (r.balance !== undefined) {
|
||||
msg += ` (Balance: $${r.balance.toFixed(4)})`;
|
||||
}
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, msg));
|
||||
if (!silent) modelIn.focus();
|
||||
} else {
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
}
|
||||
} catch (e) {
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
}
|
||||
testBtn.disabled = false;
|
||||
}
|
||||
testBtn.addEventListener("click", testKey);
|
||||
testBtn.addEventListener("click", () => testKey(false));
|
||||
keyIn.addEventListener("change", () => testKey(true));
|
||||
|
||||
const keyHint =
|
||||
existing && existing.key_source === "store" ? t("models.form.keyStored")
|
||||
: existing && existing.key_source === "env" ? t("models.form.keyFromEnv")
|
||||
: t("models.form.apiKeyHint");
|
||||
|
||||
// ⚙ Generation & Thinking Parameters
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: m.temperature ?? "" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: m.top_p ?? "" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: m.max_tokens ?? "" });
|
||||
const thinkIn = h("input", { type: "number", min: "0", placeholder: "1024", value: m.think_budget ?? "" });
|
||||
const o1Sel = h("select", {},
|
||||
h("option", { value: "" }, t("import.adv.auto")),
|
||||
h("option", { value: "all" }, "all — O(1) Attention"),
|
||||
h("option", { value: "deep" }, "deep — O(1) Deep"),
|
||||
h("option", { value: "off" }, "off — Standard"));
|
||||
if (m.o1) o1Sel.value = m.o1;
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: m.skip_mtp ? true : null });
|
||||
const sysPromptIn = h("textarea", { rows: 2, placeholder: t("models.form.systemPromptPlaceholder"), value: m.system_prompt || "" });
|
||||
|
||||
const advParams = h(
|
||||
"details",
|
||||
{ class: "adv-params", style: "margin-top:12px;padding:10px;border:1px solid var(--border);border-radius:6px;" },
|
||||
h("summary", { style: "font-weight:600;cursor:pointer;" }, "⚙ " + t("models.form.paramsTitle")),
|
||||
h("div", { class: "row", style: "margin-top:8px" },
|
||||
field(t("models.form.temperature"), tempIn, t("models.form.temperatureHint")),
|
||||
field(t("models.form.topP"), topPIn, t("models.form.topPHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("models.form.maxTokens"), maxTokIn, t("models.form.maxTokensHint")),
|
||||
field(t("models.form.thinkBudget"), thinkIn, t("models.form.thinkBudgetHint"))),
|
||||
h("div", { class: "row" },
|
||||
field("O(1) Attention", o1Sel, t("import.adv.o1Hint")),
|
||||
field("MTP", h("label", { class: "check" }, skipMtpCb, t("import.adv.skipMtp")))),
|
||||
field(t("models.form.systemPrompt"), sysPromptIn, t("models.form.systemPromptHint"))
|
||||
);
|
||||
|
||||
const node = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -110,7 +201,8 @@ function modelForm(meta, existing) {
|
||||
dl,
|
||||
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
|
||||
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c))))
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap", style: "gap:10px" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check", style: "margin:0" }, cb, CAP_EMOJIS[c] || c)))),
|
||||
advParams
|
||||
);
|
||||
|
||||
const getValue = () => {
|
||||
@@ -129,6 +221,13 @@ function modelForm(meta, existing) {
|
||||
price_out: parseFloat(priceOutIn.value) || 0,
|
||||
api_key_env: hasKey ? keyEnv : null,
|
||||
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: thinkIn.value !== "" ? parseInt(thinkIn.value) : null,
|
||||
o1: o1Sel.value || null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysPromptIn.value.trim() || null,
|
||||
},
|
||||
secret: typedKey,
|
||||
keyEnv,
|
||||
@@ -142,8 +241,11 @@ async function openModal(meta, existing, reload) {
|
||||
const { node, getValue } = modelForm(meta, existing);
|
||||
modal(existing ? t("models.form.editTitle") : t("models.form.addTitle"), node, async () => {
|
||||
const { model, secret, keyEnv } = getValue();
|
||||
// Forgot the id? Derive it from the model name so saving never errors on a
|
||||
// blank id.
|
||||
if (!model.id) model.id = slugId(model.model);
|
||||
if (!model.id) {
|
||||
toast(t("models.form.id") + "?", "bad");
|
||||
toast(t("models.form.idOrModel"), "bad");
|
||||
return false;
|
||||
}
|
||||
if (!model.api_key_env) delete model.api_key_env; // keep serde field optional
|
||||
@@ -213,10 +315,63 @@ export async function renderModels() {
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
),
|
||||
benchCard()
|
||||
);
|
||||
}
|
||||
|
||||
// ── one-click benchmark of local .cmf files (cortiq bench --json) ──
|
||||
function benchCard() {
|
||||
const host = h("div", { class: "card" }, h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
|
||||
h("div", { class: "empty" }, t("common.loading")));
|
||||
let timer = null;
|
||||
async function paint() {
|
||||
let files = [];
|
||||
let runs = {};
|
||||
try {
|
||||
files = (await api.mediaModels()).models || [];
|
||||
for (const r of (await api.benchList()).runs || []) runs[r.model] = r;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const chat = files.filter((m) => m.modality === "chat");
|
||||
const anyRunning = Object.values(runs).some((r) => r.state === "running");
|
||||
const rows = chat.map((m) => {
|
||||
const r = runs[m.name];
|
||||
const go = h("button", { class: "btn sm ghost", disabled: anyRunning ? true : null }, "⏱ " + t("bench.run"));
|
||||
go.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.benchStart(m.name);
|
||||
paint();
|
||||
if (!timer) timer = setInterval(() => {
|
||||
if (!host.isConnected) { clearInterval(timer); return; }
|
||||
paint();
|
||||
}, 3000);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
});
|
||||
let result = h("span", { class: "muted" }, "—");
|
||||
if (r?.state === "running") result = h("span", { class: "flex" }, h("span", { class: "spinner" }), " ", t("bench.running"));
|
||||
else if (r?.state === "done") result = h("b", { class: "mono" }, (r.tok_s != null ? r.tok_s.toFixed(1) : "?") + " tok/s");
|
||||
else if (r?.state === "error") result = h("span", { class: "err", title: (r.log || []).slice(-2).join(" · ") }, t("bench.failed"));
|
||||
return h("div", { class: "flex bench-row" },
|
||||
h("span", { class: "mono grow" }, m.name),
|
||||
h("span", { class: "muted small" }, (m.size_bytes / 1e9).toFixed(2) + " GB"),
|
||||
result,
|
||||
go);
|
||||
});
|
||||
mount(host,
|
||||
h("div", { class: "card-title" }, "⏱ " + t("bench.title")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:8px" }, t("bench.hint")),
|
||||
...(rows.length ? rows : [h("div", { class: "empty" }, t("bench.none"))]));
|
||||
if (!anyRunning && timer) { clearInterval(timer); timer = null; }
|
||||
}
|
||||
paint();
|
||||
return host;
|
||||
}
|
||||
|
||||
await reload();
|
||||
return root;
|
||||
}
|
||||
@@ -264,7 +419,7 @@ function row(m, meta, reload) {
|
||||
try {
|
||||
await api.deleteModel(m.id);
|
||||
toast(t("toast.deleted"), "good");
|
||||
reload();
|
||||
await reload();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
|
||||
Vendored
+335
@@ -0,0 +1,335 @@
|
||||
// First-run onboarding wizard: language → featured .cmf models (infosave) → HF search+convert → done.
|
||||
// No manual TOML edits: every action goes through the admin API.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t, getLang, setLang, LANGS } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured as fetchFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobBar,
|
||||
jobStatusText,
|
||||
fmtBytes,
|
||||
} from "./featured.js";
|
||||
|
||||
let step = 0; // 0 lang, 1 featured, 2 hf convert, 3 done
|
||||
let featured = [];
|
||||
let featuredLoaded = false;
|
||||
|
||||
function langStep(onNext) {
|
||||
const cards = LANGS.map((l) =>
|
||||
h("button", {
|
||||
class: "onb-lang" + (getLang() === l.code ? " sel" : ""),
|
||||
onClick: () => {
|
||||
setLang(l.code);
|
||||
cards.forEach((c) => c.classList.toggle("sel", c.dataset.code === l.code));
|
||||
toast(l.flag + " " + l.label, "ok");
|
||||
},
|
||||
"data-code": l.code,
|
||||
title: l.label,
|
||||
"aria-label": l.label,
|
||||
}, h("span", { class: "flag" }, l.flag), h("b", {}, l.label))
|
||||
);
|
||||
return h("div", { class: "onb-step" },
|
||||
h("h2", {}, t("onboarding.welcome")),
|
||||
h("p", { class: "muted" }, t("onboarding.welcomeSub")),
|
||||
h("div", { class: "onb-langs" }, ...cards),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn primary", onClick: onNext }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
export function renderOnboarding() {
|
||||
const root = h("div", { class: "onb-wrap" });
|
||||
let jobsPoll = null;
|
||||
let jobs = [];
|
||||
let sysSnapshot = null; // disk/RAM для плашек "не влезет" на карточках
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
let hosts = new Map(); // job id → card progress host (rebuilt on every render)
|
||||
let jobsHost = h("div", {});
|
||||
|
||||
// Job rows: bar, live status, and the actions that finish or undo a download.
|
||||
function jobsMini(list) {
|
||||
if (!list.length) return h("div", { class: "muted small" }, t("onboarding.noJobs"));
|
||||
return h("div", {},
|
||||
...list.map((j) => {
|
||||
const act = [];
|
||||
if (j.state === "done") {
|
||||
const btn = h("button", { class: "btn primary sm" }, "✓ " + t("import.register"));
|
||||
btn.addEventListener("click", async () => {
|
||||
btn.disabled = true;
|
||||
try {
|
||||
const r = await api.registerImport(j.id);
|
||||
toast(t("import.registered", { id: r.model_id }), "ok");
|
||||
} catch (e) { toast(e.message, "err"); btn.disabled = false; }
|
||||
});
|
||||
act.push(btn);
|
||||
}
|
||||
if (j.state === "running") {
|
||||
const cancel = h("button", { class: "btn ghost sm" }, "✕ " + t("import.cancel"));
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await api.cancelImport(j.id); toast(t("import.cancelling"), "ok"); await refreshJobs(); }
|
||||
catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
act.push(cancel);
|
||||
} else {
|
||||
// done / error / cancelled — remove the .cmf from disk and redo the step
|
||||
const del = h("button", { class: "btn ghost sm danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete"));
|
||||
del.addEventListener("click", () => deleteJob(j));
|
||||
act.push(del);
|
||||
}
|
||||
const known = ["running", "done", "error", "cancelled"].includes(j.state);
|
||||
return h("div", { class: "job", style: "margin:8px 0" },
|
||||
h("div", { class: "job-head" },
|
||||
h("b", {}, j.name || j.repo || j.id),
|
||||
h("span", { class: "badge" + (j.state === "done" ? " ok" : j.state === "running" ? "" : " bad") }, known ? t("import.state." + j.state) : j.state)
|
||||
),
|
||||
jobBar(j),
|
||||
h("div", { class: "muted small" }, jobStatusText(j)),
|
||||
h("pre", { class: "conv-log", style: "max-height:80px" }, (j.log || []).slice(-6).join("\n")),
|
||||
act.length ? h("div", { class: "actions" }, ...act) : null
|
||||
);
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
// Repaint only the live bits (card hosts + job list) — cheap enough for 1.5s.
|
||||
function paintJobs() {
|
||||
mount(jobsHost, jobs.length
|
||||
? h("div", { class: "card" }, h("div", { class: "card-title" }, t("import.jobs")), jobsMini(jobs))
|
||||
: h("div", {}));
|
||||
paintFeaturedHosts(hosts, jobs, async (j) => {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "ok");
|
||||
await refreshJobs();
|
||||
});
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
try { jobs = (await api.listImports()).jobs || []; } catch { jobs = []; }
|
||||
// A job that just settled changes what the cards offer (✓ + delete), so the
|
||||
// featured list has to be re-read — but only on the transition, not per poll.
|
||||
const settled = jobs.some((j) => {
|
||||
const prev = jobStates.get(j.id);
|
||||
return prev === "running" && j.state !== "running";
|
||||
});
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) {
|
||||
await loadFeatured();
|
||||
render(); // rebuilds the cards (✓ + delete) and repaints the job rows
|
||||
} else {
|
||||
paintJobs();
|
||||
}
|
||||
if (jobs.some((j) => j.state === "running")) {
|
||||
if (!jobsPoll) jobsPoll = setInterval(refreshJobs, 1500);
|
||||
} else {
|
||||
if (jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; }
|
||||
}
|
||||
}
|
||||
|
||||
async function loadFeatured() {
|
||||
featured = await fetchFeatured();
|
||||
try { sysSnapshot = await api.system(); } catch { /* optional */ }
|
||||
featuredLoaded = true;
|
||||
}
|
||||
|
||||
function topNav(prev, next, extra) {
|
||||
return h("div", { class: "actions", style: "margin-bottom:12px; position:sticky; top:0; background:var(--bg); z-index:2; padding:8px 0; border-bottom:1px solid var(--border-soft); backdrop-filter:blur(4px)" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
extra ? extra : h("div", { style: "flex:1" }),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
}
|
||||
|
||||
async function startDownload(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function handleDelete(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
// Delete straight from a job row: the gateway knows that job's output path.
|
||||
async function deleteJob(j) {
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: j.name || j.repo }))) return;
|
||||
try {
|
||||
await api.deleteImport(j.id);
|
||||
toast(t("import.deleted"), "ok");
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
function render() {
|
||||
const dots = [0, 1, 2, 3].map((i) => h("span", { class: "dot" + (i === step ? " on" : i < step ? " done" : "") }));
|
||||
const header = h("div", { class: "onb-header" },
|
||||
h("div", { class: "onb-dots" }, ...dots),
|
||||
h("div", { class: "onb-title" }, t("onboarding.title") + ` — ${step + 1}/4`)
|
||||
);
|
||||
|
||||
let body;
|
||||
const next = () => { step = Math.min(3, step + 1); render(); };
|
||||
const prev = () => { step = Math.max(0, step - 1); render(); };
|
||||
hosts = new Map(); // rebuilt by featuredCard below
|
||||
|
||||
if (step === 0) {
|
||||
const top = h("div", { class: "actions", style: "margin-bottom:12px; justify-content:flex-end" },
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
body = h("div", {},
|
||||
top,
|
||||
langStep(next)
|
||||
);
|
||||
} else if (step === 1) {
|
||||
const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip")));
|
||||
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete, sys: sysSnapshot };
|
||||
// the list is fetched from HuggingFace — say so instead of showing a blank grid
|
||||
const grid = featured.length
|
||||
? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx)))
|
||||
: h("div", { class: "muted pad" }, featuredLoaded ? t("import.noResults") : t("import.searching"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
top,
|
||||
h("h2", {}, "📦 " + t("onboarding.featuredTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.featuredSub")),
|
||||
grid,
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn", onClick: async () => { await loadFeatured(); render(); await refreshJobs(); } }, t("common.refresh")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →"),
|
||||
h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip"))
|
||||
)
|
||||
);
|
||||
} else if (step === 2) {
|
||||
const top2 = topNav(prev, next);
|
||||
// lightweight HF search inline
|
||||
const inp = h("input", { placeholder: t("import.searchPlaceholder"), style: "width:100%" });
|
||||
const res = h("div", { class: "hf-grid" });
|
||||
const quantSel = h("select", {},
|
||||
h("option", { value: "Q8_2F" }, "Q8_2F — " + t("import.q.q8_2f")),
|
||||
h("option", { value: "Q4_BLOCK" }, "Q4 — " + t("import.q.q4")),
|
||||
h("option", { value: "Q1T" }, "Q1T — " + t("import.q.q1t")),
|
||||
h("option", { value: "Q1" }, "Q1 — " + t("import.q.q1")),
|
||||
);
|
||||
let selected = null;
|
||||
let debounce = null;
|
||||
async function doSearch() {
|
||||
const q = inp.value.trim();
|
||||
mount(res, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
try {
|
||||
const r = await api.hfSearch(q, 16);
|
||||
const models = r.models || [];
|
||||
if (!models.length) { mount(res, h("div", { class: "muted pad" }, t("import.noResults"))); return; }
|
||||
mount(res, ...models.map((m) => {
|
||||
const card = h("div", { class: "hf-card" + (selected && selected.id === m.id ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, m.id),
|
||||
h("div", { class: "hf-meta" }, h("span", { class: "badge" }, m.pipeline_tag || "model"), h("span", { class: "muted" }, "♥ " + (m.likes ?? 0)))
|
||||
);
|
||||
card.addEventListener("click", () => { selected = m; doSearch(); });
|
||||
return card;
|
||||
}));
|
||||
} catch (e) { mount(res, h("div", { class: "err" }, String(e.message))); }
|
||||
}
|
||||
inp.addEventListener("input", () => { clearTimeout(debounce); debounce = setTimeout(doSearch, 300); });
|
||||
const go = h("button", { class: "btn primary" }, "⬇ " + t("import.convert"));
|
||||
go.addEventListener("click", async () => {
|
||||
if (!selected) { toast(t("import.noResults"), "warn"); return; }
|
||||
go.disabled = true;
|
||||
try {
|
||||
await api.startImport({ repo: selected.id, quant: quantSel.value, name: selected.id.split("/").pop().toLowerCase() });
|
||||
toast(t("import.started"), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
go.disabled = false;
|
||||
});
|
||||
// auto initial search
|
||||
setTimeout(doSearch, 100);
|
||||
body = h("div", { class: "onb-step" },
|
||||
top2,
|
||||
h("h2", {}, "🔍 " + t("onboarding.hfTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.hfSub")),
|
||||
h("div", { class: "card" }, inp, res),
|
||||
h("div", { class: "card", style: "margin-top:12px" },
|
||||
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("import.quant")), quantSel)),
|
||||
h("div", { class: "actions" }, go)
|
||||
),
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
} else {
|
||||
// done — finishing connects every completed download as a local model
|
||||
const ready = jobs.filter((j) => j.state === "done");
|
||||
const doneBtn = h("button", { class: "btn primary" }, t("onboarding.finish"));
|
||||
doneBtn.addEventListener("click", async () => {
|
||||
doneBtn.disabled = true;
|
||||
let r = null;
|
||||
try { r = await api.setupComplete(getLang(), true); } catch (e) { toast(e.message, "err"); }
|
||||
if (r && r.registered && r.registered.length) {
|
||||
toast(t("onboarding.registeredN", { n: r.registered.length, ids: r.registered.join(", ") }), "ok");
|
||||
if (!r.runtime_installed) toast(t("onboarding.runtimeMissing"), "warn");
|
||||
// let the toast be readable before the reload swaps the page
|
||||
await new Promise((res) => setTimeout(res, 900));
|
||||
}
|
||||
// land where the reward is: try the model right away
|
||||
location.hash = "#/playground";
|
||||
location.reload();
|
||||
});
|
||||
const skipBtn = h("button", { class: "btn ghost", onClick: async () => {
|
||||
try { await api.setupComplete(getLang(), false); } catch {}
|
||||
location.hash = "#/dashboard";
|
||||
} }, t("onboarding.skipSetup"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
h("h2", {}, "✅ " + t("onboarding.doneTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.doneSub")),
|
||||
ready.length
|
||||
? h("div", { class: "card" },
|
||||
h("div", { class: "card-title" }, t("onboarding.willRegister")),
|
||||
...ready.map((j) => h("div", { class: "muted small" }, "• cmf-" + (j.name || j.repo) + (j.size_bytes ? ` · ${fmtBytes(j.size_bytes)}` : "")))
|
||||
)
|
||||
: h("div", { class: "muted small" }, t("onboarding.nothingToRegister")),
|
||||
jobsHost,
|
||||
h("div", { class: "callout" }, t("onboarding.doneHint")),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
skipBtn,
|
||||
doneBtn
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
mount(root, h("div", { class: "onb" }, header, body));
|
||||
paintJobs(); // fill the freshly built hosts with whatever we already know
|
||||
if (jobs.some((j) => j.state === "running") && !jobsPoll) {
|
||||
jobsPoll = setInterval(refreshJobs, 1500);
|
||||
}
|
||||
}
|
||||
|
||||
loadFeatured().then(() => { render(); refreshJobs(); });
|
||||
render();
|
||||
// cleanup poll on detach
|
||||
const obs = new MutationObserver(() => { if (!root.isConnected && jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; obs.disconnect(); } });
|
||||
obs.observe(document.body, { childList: true, subtree: true });
|
||||
return root;
|
||||
}
|
||||
Vendored
+544
-39
@@ -1,17 +1,64 @@
|
||||
// Playground — send a prompt through the live pipeline, inspect routing decision.
|
||||
import { h, mount, money, ms } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api, testStream } from "../api.js";
|
||||
// Playground — three tabs over the live gateway:
|
||||
// chat → the routing pipeline (with streaming, token counts, tok/s, TTFT)
|
||||
// image → `cortiq imagine` over a local text-to-image .cmf (Lumina)
|
||||
// video → `cortiq animate` over a local text-to-video .cmf (MiniMax-H3)
|
||||
// Media results are decoded in the browser (PPM → canvas, MJPEG AVI → canvas
|
||||
// player synced to the wav) — the same no-ffmpeg stance as the engine itself.
|
||||
import { h, mount, toast, money, moneyEl, ms } from "../ui.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api, testStream, mediaFileUrl, mediaFileBytes } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
import { ppmToCanvas, fileToPpmB64, aviJpegFrames, mjpegPlayer } from "../media-utils.js";
|
||||
|
||||
function fmtBytes(n) {
|
||||
if (!n && n !== 0) return "—";
|
||||
if (n >= 1e9) return (n / 1e9).toFixed(2) + " GB";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + " MB";
|
||||
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
|
||||
return n + " B";
|
||||
}
|
||||
function kv(k, v) {
|
||||
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
|
||||
}
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
|
||||
export async function renderPlayground() {
|
||||
const meta = appState.meta || (await api.meta());
|
||||
const modelsData = await api.listModels();
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
let mediaModels = [];
|
||||
try {
|
||||
mediaModels = (await api.mediaModels()).models || [];
|
||||
} catch {
|
||||
/* media API unavailable — tabs will show the hint */
|
||||
}
|
||||
const imageModels = mediaModels.filter((m) => m.modality === "image");
|
||||
const videoModels = mediaModels.filter((m) => m.modality === "video");
|
||||
|
||||
let mode = "auto";
|
||||
const root = h("div");
|
||||
let tab = "chat";
|
||||
|
||||
// ── tabs ──
|
||||
const tabsSeg = h(
|
||||
"div",
|
||||
{ class: "seg" },
|
||||
h("button", { class: "active", "data-tab": "chat" }, "💬 " + t("pg.tab.chat")),
|
||||
h("button", { "data-tab": "image" }, "🖼 " + t("pg.tab.image")),
|
||||
h("button", { "data-tab": "video" }, "🎬 " + t("pg.tab.video"))
|
||||
);
|
||||
const panelHost = h("div", {});
|
||||
tabsSeg.querySelectorAll("button").forEach((b) =>
|
||||
b.addEventListener("click", () => {
|
||||
tab = b.dataset.tab;
|
||||
tabsSeg.querySelectorAll("button").forEach((x) => x.classList.toggle("active", x === b));
|
||||
mount(panelHost, panels[tab]);
|
||||
})
|
||||
);
|
||||
|
||||
// ════════════════════════════ chat panel ════════════════════════════
|
||||
let mode = "auto";
|
||||
const sysIn = h("textarea", { placeholder: t("pg.systemPlaceholder"), rows: 2 });
|
||||
const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 });
|
||||
const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 });
|
||||
@@ -44,16 +91,33 @@ export async function renderPlayground() {
|
||||
modeExtra = next;
|
||||
}
|
||||
|
||||
const out = h("div", { class: "answer" }, h("div", { class: "empty" }, t("pg.empty")));
|
||||
const chatLog = []; // the running conversation: {role, content}
|
||||
const out = h("div", { class: "answer chat-log" }, h("div", { class: "empty" }, t("pg.empty")));
|
||||
const decision = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
|
||||
|
||||
function bubble(role, text) {
|
||||
return h("div", { class: "chat-msg " + role }, h("div", { class: "chat-bubble" }, text));
|
||||
}
|
||||
function renderChat(liveText) {
|
||||
const items = chatLog.map((m) => bubble(m.role, m.content));
|
||||
if (liveText != null) items.push(bubble("assistant", liveText || "…"));
|
||||
mount(out, ...(items.length ? items : [h("div", { class: "empty" }, t("pg.empty"))]));
|
||||
out.scrollTop = out.scrollHeight;
|
||||
}
|
||||
const clearBtn = h("button", { class: "btn ghost sm" }, "🧹 " + t("pg.clearChat"));
|
||||
clearBtn.addEventListener("click", () => {
|
||||
chatLog.length = 0;
|
||||
renderChat();
|
||||
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
|
||||
});
|
||||
|
||||
const sendBtn = h("button", { class: "btn primary" }, t("pg.send"));
|
||||
sendBtn.addEventListener("click", async () => {
|
||||
const prompt = promptIn.value.trim();
|
||||
if (!prompt) return;
|
||||
const messages = [];
|
||||
if (sysIn.value.trim()) messages.push({ role: "system", content: sysIn.value.trim() });
|
||||
messages.push({ role: "user", content: prompt });
|
||||
messages.push(...chatLog, { role: "user", content: prompt });
|
||||
let model = "cortiq-auto";
|
||||
if (mode === "auto") {
|
||||
if (profileSel.value) model = "cortiq-auto:" + profileSel.value;
|
||||
@@ -62,7 +126,9 @@ export async function renderPlayground() {
|
||||
}
|
||||
sendBtn.disabled = true;
|
||||
sendBtn.textContent = t("pg.sending");
|
||||
mount(out, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.sending")));
|
||||
chatLog.push({ role: "user", content: prompt });
|
||||
promptIn.value = "";
|
||||
renderChat("");
|
||||
const body = {
|
||||
model,
|
||||
messages,
|
||||
@@ -73,12 +139,13 @@ export async function renderPlayground() {
|
||||
if (streamCb.checked) {
|
||||
const started = performance.now();
|
||||
let acc = "";
|
||||
out.textContent = "";
|
||||
const info = await testStream(body, (d) => {
|
||||
acc += d;
|
||||
out.textContent = acc;
|
||||
renderChat(acc);
|
||||
});
|
||||
out.textContent = acc || "—";
|
||||
chatLog.push({ role: "assistant", content: acc || "—" });
|
||||
renderChat();
|
||||
const latency = Math.round(performance.now() - started);
|
||||
renderDecision({
|
||||
cortiq: {
|
||||
task_label: info.task_label,
|
||||
@@ -88,37 +155,58 @@ export async function renderPlayground() {
|
||||
cost_usd: info.cost_usd,
|
||||
failover: false,
|
||||
},
|
||||
usage: {},
|
||||
latency_ms: Math.round(performance.now() - started),
|
||||
usage: info.usage || {},
|
||||
usage_estimated: info.usage_estimated,
|
||||
latency_ms: latency,
|
||||
ttft_ms: info.ttft_ms,
|
||||
});
|
||||
} else {
|
||||
const r = await api.test(body);
|
||||
if (r.ok) {
|
||||
mount(out, r.answer || h("span", { class: "muted" }, "—"));
|
||||
chatLog.push({ role: "assistant", content: r.answer || "—" });
|
||||
renderChat();
|
||||
renderDecision(r);
|
||||
} else {
|
||||
mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
|
||||
chatLog.pop(); // the user turn failed — let them retry it
|
||||
renderChat();
|
||||
promptIn.value = prompt;
|
||||
mount(out, out.firstChild || "", h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
|
||||
mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
mount(out, h("div", { class: "callout" }, String(e.message || e)));
|
||||
chatLog.pop();
|
||||
renderChat();
|
||||
promptIn.value = prompt;
|
||||
toast(String(e.message || e), "err");
|
||||
}
|
||||
sendBtn.disabled = false;
|
||||
sendBtn.textContent = t("pg.send");
|
||||
});
|
||||
// Cmd/Ctrl+Enter sends
|
||||
promptIn.addEventListener("keydown", (e) => {
|
||||
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") {
|
||||
e.preventDefault();
|
||||
sendBtn.click();
|
||||
}
|
||||
});
|
||||
|
||||
function kv(k, v) {
|
||||
return h("div", { class: "kv" }, h("span", { class: "k" }, k), h("span", { class: "v" }, v));
|
||||
}
|
||||
function renderDecision(r) {
|
||||
const c = r.cortiq || {};
|
||||
const score = c.complexity?.score ?? 0;
|
||||
const u = r.usage || {};
|
||||
const haveUsage = u.completion_tokens != null;
|
||||
// generation speed over the produce phase (after first token when known)
|
||||
let tokS = null;
|
||||
if (haveUsage && r.latency_ms) {
|
||||
const genMs = r.ttft_ms != null ? Math.max(1, r.latency_ms - r.ttft_ms) : r.latency_ms;
|
||||
tokS = (u.completion_tokens / (genMs / 1000)).toFixed(1);
|
||||
}
|
||||
mount(
|
||||
decision,
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))),
|
||||
kv(t("pg.task"), c.task_label || "—"),
|
||||
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, c.complexity.tier) : "—"),
|
||||
kv(t("pg.tier"), c.complexity?.tier ? h("span", { class: "badge tier-" + c.complexity.tier }, trTier(c.complexity.tier)) : "—"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "kv", style: "display:block" },
|
||||
@@ -127,17 +215,278 @@ export async function renderPlayground() {
|
||||
),
|
||||
kv(t("pg.selected"), h("span", { class: "mono" }, c.selected_model || "—")),
|
||||
kv(t("pg.source"), c.route_source || "—"),
|
||||
kv(t("pg.tokens"), `${r.usage?.prompt_tokens ?? 0}/${r.usage?.completion_tokens ?? 0}`),
|
||||
kv(t("pg.cost"), money(c.cost_usd)),
|
||||
kv(t("pg.tokens"), haveUsage ? `${r.usage_estimated ? "≈ " : ""}${u.prompt_tokens ?? 0} / ${u.completion_tokens ?? 0}` : "—"),
|
||||
kv(t("pg.tokSpeed"), tokS != null ? tokS + " tok/s" : "—"),
|
||||
kv(t("pg.ttft"), r.ttft_ms != null ? ms(r.ttft_ms) : "—"),
|
||||
kv(t("pg.cost"), moneyEl(c.cost_usd)),
|
||||
kv(t("pg.latency"), ms(r.latency_ms)),
|
||||
kv(t("pg.failover"), c.failover ? "yes" : "no")
|
||||
);
|
||||
}
|
||||
|
||||
mount(
|
||||
root,
|
||||
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
|
||||
const chatPanel = h(
|
||||
"div",
|
||||
{ class: "pg" },
|
||||
h(
|
||||
"div",
|
||||
{},
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
field(t("pg.system"), sysIn),
|
||||
field(t("pg.prompt"), promptIn),
|
||||
field(t("pg.mode"), modeSeg),
|
||||
modeExtra,
|
||||
h("div", { class: "row" }, field(t("pg.temp"), tempIn), field(t("pg.maxTokens"), maxTokIn)),
|
||||
h(
|
||||
"div",
|
||||
{ class: "flex", style: "justify-content:space-between; gap:8px" },
|
||||
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
|
||||
h("div", { class: "flex", style: "gap:8px" }, clearBtn, sendBtn)
|
||||
)
|
||||
),
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
|
||||
),
|
||||
decision
|
||||
);
|
||||
|
||||
// ═══════════════════════ shared media machinery ═══════════════════════
|
||||
let mediaPoll = null;
|
||||
function stopMediaPoll() {
|
||||
if (mediaPoll) clearInterval(mediaPoll);
|
||||
mediaPoll = null;
|
||||
}
|
||||
// one generation at a time server-side → one active job client-side
|
||||
function watchJob(id, ui, onDone) {
|
||||
stopMediaPoll();
|
||||
const started = performance.now();
|
||||
mediaPoll = setInterval(async () => {
|
||||
if (!root.isConnected) return stopMediaPoll();
|
||||
let j;
|
||||
try {
|
||||
j = await api.mediaJob(id);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const elapsed = (performance.now() - started) / 1000;
|
||||
const pct = j.progress != null ? Math.round(j.progress * 100) : null;
|
||||
// honest ETA once there is enough signal: total ≈ elapsed / progress
|
||||
let eta = "";
|
||||
if (j.state === "running" && j.progress > 0.03) {
|
||||
const left = elapsed / j.progress - elapsed;
|
||||
if (left > 1) eta = ` · ${t("pg.eta")} ~${left > 90 ? Math.round(left / 60) + " " + t("pg.min") : Math.round(left) + " s"}`;
|
||||
}
|
||||
let statusLine;
|
||||
if (j.state === "queued") {
|
||||
const pos = (j.queue_position ?? 0) + 1;
|
||||
statusLine = t("pg.queued", { n: pos });
|
||||
} else {
|
||||
statusLine =
|
||||
(j.phase || j.state) +
|
||||
(pct != null && j.state === "running" ? ` · ${pct}%` : "") +
|
||||
` · ${elapsed.toFixed(0)}s` + eta;
|
||||
}
|
||||
mount(
|
||||
ui.bar,
|
||||
h("div", { class: "pbar" },
|
||||
j.state === "running"
|
||||
? (pct != null
|
||||
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
|
||||
: h("div", { class: "pbar-fill run" }))
|
||||
: j.state === "queued"
|
||||
? h("div", { class: "pbar-fill run" })
|
||||
: h("div", { class: "pbar-fill " + (j.state === "done" ? "ok" : "err"), style: "width:100%" })
|
||||
),
|
||||
h("div", { class: "muted small" }, statusLine),
|
||||
(j.log || []).length && j.state === "running" ? h("pre", { class: "conv-log", style: "max-height:90px" }, j.log.slice(-5).join("\n")) : null
|
||||
);
|
||||
if (j.state !== "running" && j.state !== "queued") {
|
||||
stopMediaPoll();
|
||||
ui.goBtn.disabled = false;
|
||||
ui.cancelBtn.style.display = "none";
|
||||
if (j.state === "done") onDone(j);
|
||||
else if (j.state === "error") toast(t("pg.genFailed"), "err");
|
||||
}
|
||||
}, 1000);
|
||||
}
|
||||
function mediaControls(goBtn, onCancelId) {
|
||||
const cancelBtn = h("button", { class: "btn ghost", style: "display:none" }, "✕ " + t("import.cancel"));
|
||||
cancelBtn.addEventListener("click", async () => {
|
||||
try {
|
||||
await api.mediaCancel(onCancelId.id);
|
||||
} catch (e) {
|
||||
toast(e.message, "err");
|
||||
}
|
||||
});
|
||||
return cancelBtn;
|
||||
}
|
||||
function noModelsHint(kindLabel) {
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "callout" },
|
||||
t("pg.noMediaModels", { kind: kindLabel }),
|
||||
" ",
|
||||
h("a", { href: "#/import" }, t("settings.cmf.importLink") + " ↗"))
|
||||
);
|
||||
}
|
||||
function dice(seedIn) {
|
||||
const b = h("button", { class: "btn ghost sm", title: t("pg.randomSeed") }, "🎲");
|
||||
b.addEventListener("click", () => (seedIn.value = Math.floor(Math.random() * 1e9)));
|
||||
return b;
|
||||
}
|
||||
|
||||
// ── gallery: past generations of one kind, loaded lazily ──
|
||||
function buildGallery(kind, onOpen) {
|
||||
const host = h("div", {});
|
||||
async function refresh() {
|
||||
let jobs = [];
|
||||
try {
|
||||
jobs = ((await api.mediaJobs()).jobs || []).filter((j) => j.kind === kind && j.state === "done");
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!jobs.length) {
|
||||
mount(host);
|
||||
return;
|
||||
}
|
||||
const grid = h("div", { class: "gal-grid" });
|
||||
for (const j of jobs.slice(0, 12)) {
|
||||
const del = h("button", { class: "gal-del", title: t("import.delete") }, "×");
|
||||
del.addEventListener("click", async (e) => {
|
||||
e.stopPropagation();
|
||||
if (!window.confirm(t("pg.galDeleteConfirm"))) return;
|
||||
try {
|
||||
await api.mediaDelete(j.id);
|
||||
refresh();
|
||||
} catch (err) {
|
||||
toast(err.message, "err");
|
||||
}
|
||||
});
|
||||
const cap = h("div", { class: "gal-cap" }, `${j.width}×${j.height} · seed ${j.seed}`);
|
||||
const item = h("div", { class: "gal-item", title: j.prompt }, del, cap);
|
||||
if (kind === "image") {
|
||||
// lazy thumbnail: decode the PPM only when the item scrolls into view
|
||||
const thumb = h("canvas", { class: "gal-thumb" });
|
||||
item.prepend(thumb);
|
||||
const io = new IntersectionObserver(async (entries) => {
|
||||
if (!entries.some((x) => x.isIntersecting)) return;
|
||||
io.disconnect();
|
||||
try {
|
||||
const buf = await mediaFileBytes(j.id, "image");
|
||||
const full = ppmToCanvas(buf);
|
||||
thumb.width = 128;
|
||||
thumb.height = Math.round((128 * full.height) / full.width) || 128;
|
||||
thumb.getContext("2d").drawImage(full, 0, 0, thumb.width, thumb.height);
|
||||
} catch {
|
||||
/* file gone */
|
||||
}
|
||||
});
|
||||
io.observe(item);
|
||||
} else {
|
||||
item.prepend(h("div", { class: "gal-thumb gal-video" }, "🎬"));
|
||||
}
|
||||
item.addEventListener("click", () => onOpen(j));
|
||||
grid.appendChild(item);
|
||||
}
|
||||
mount(host, h("div", { class: "card-title", style: "margin-top:14px" }, t("pg.gallery")), grid);
|
||||
}
|
||||
refresh();
|
||||
return { host, refresh };
|
||||
}
|
||||
|
||||
// ════════════════════════════ image panel ════════════════════════════
|
||||
let imagePanel;
|
||||
if (!imageModels.length) {
|
||||
imagePanel = noModelsHint(t("pg.tab.image"));
|
||||
} else {
|
||||
const imModel = h("select", {}, ...imageModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
|
||||
const imPrompt = h("textarea", { placeholder: t("pg.imagePromptPlaceholder"), rows: 3 });
|
||||
const imW = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
|
||||
const imH = h("select", {}, ...[256, 384, 512, 640, 768, 1024].map((v) => h("option", { value: v, selected: v === 512 ? true : null }, v)));
|
||||
const imSteps = h("input", { type: "number", min: 1, max: 100, value: 30 });
|
||||
const imCfg = h("input", { type: "number", step: "0.5", value: 4 });
|
||||
const imSeed = h("input", { type: "number", value: 42 });
|
||||
const imBar = h("div", {});
|
||||
const imOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.imageEmpty")));
|
||||
const imMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
|
||||
const imGo = h("button", { class: "btn primary" }, "🖼 " + t("pg.generate"));
|
||||
const imCancelRef = { id: null };
|
||||
const imCancel = mediaControls(imGo, imCancelRef);
|
||||
|
||||
imGo.addEventListener("click", async () => {
|
||||
const prompt = imPrompt.value.trim();
|
||||
if (!prompt) return;
|
||||
imGo.disabled = true;
|
||||
imCancel.style.display = "";
|
||||
mount(imOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
|
||||
const params = {
|
||||
kind: "image",
|
||||
model: imModel.value,
|
||||
prompt,
|
||||
width: +imW.value,
|
||||
height: +imH.value,
|
||||
steps: +imSteps.value || 30,
|
||||
cfg: parseFloat(imCfg.value) || 4,
|
||||
seed: +imSeed.value || 42,
|
||||
};
|
||||
try {
|
||||
const r = await api.mediaGenerate(params);
|
||||
imCancelRef.id = r.job;
|
||||
const t0 = performance.now();
|
||||
watchJob(r.job, { bar: imBar, goBtn: imGo, cancelBtn: imCancel }, async (j) => {
|
||||
mount(imBar);
|
||||
const secs = (performance.now() - t0) / 1000;
|
||||
await showImageResult(j, secs);
|
||||
imGallery.refresh();
|
||||
});
|
||||
} catch (e) {
|
||||
imGo.disabled = false;
|
||||
imCancel.style.display = "none";
|
||||
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
});
|
||||
|
||||
async function showImageResult(j, secs) {
|
||||
try {
|
||||
const buf = await mediaFileBytes(j.id, "image");
|
||||
const canvas = ppmToCanvas(buf);
|
||||
canvas.className = "media-canvas";
|
||||
const dl = h("button", { class: "btn sm" }, "⬇ PNG");
|
||||
dl.addEventListener("click", () =>
|
||||
canvas.toBlob((b) => {
|
||||
const a = h("a", { href: URL.createObjectURL(b), download: `cortiq-${j.id}.png` });
|
||||
a.click();
|
||||
}, "image/png")
|
||||
);
|
||||
mount(imOut, canvas, h("div", { class: "actions" }, dl));
|
||||
const rows = [
|
||||
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
|
||||
kv(t("pg.resolution"), `${j.width}×${j.height}`),
|
||||
kv(t("pg.steps"), String(j.steps)),
|
||||
kv(t("pg.seed"), String(j.seed)),
|
||||
kv(t("pg.fileSize"), fmtBytes(j.out_bytes)),
|
||||
];
|
||||
if (secs != null) {
|
||||
rows.push(kv(t("pg.genTime"), secs.toFixed(1) + " s"));
|
||||
rows.push(kv(t("pg.timePerStep"), (secs / j.steps).toFixed(2) + " s"));
|
||||
} else if (j.finished && j.started) {
|
||||
const dur = j.finished - j.started;
|
||||
rows.push(kv(t("pg.genTime"), dur + " s"));
|
||||
rows.push(kv(t("pg.timePerStep"), (dur / j.steps).toFixed(2) + " s"));
|
||||
}
|
||||
mount(imMetrics, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), ...rows);
|
||||
} catch (e) {
|
||||
mount(imOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
}
|
||||
const imGallery = buildGallery("image", (j) => {
|
||||
imSeed.value = j.seed;
|
||||
imSteps.value = j.steps;
|
||||
showImageResult(j, null);
|
||||
});
|
||||
|
||||
imagePanel = h(
|
||||
"div",
|
||||
{ class: "pg" },
|
||||
h(
|
||||
@@ -146,22 +495,178 @@ export async function renderPlayground() {
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("label", { class: "field" }, h("span", {}, t("pg.system")), sysIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("pg.prompt")), promptIn),
|
||||
h("label", { class: "field" }, h("span", {}, t("pg.mode")), modeSeg),
|
||||
modeExtra,
|
||||
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("pg.temp")), tempIn), h("label", { class: "field" }, h("span", {}, t("pg.maxTokens")), maxTokIn)),
|
||||
h(
|
||||
"div",
|
||||
{ class: "flex", style: "justify-content:space-between" },
|
||||
h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
|
||||
sendBtn
|
||||
)
|
||||
field(t("pg.model"), imModel),
|
||||
field(t("pg.prompt"), imPrompt),
|
||||
h("div", { class: "row" }, field(t("pg.width"), imW), field(t("pg.height"), imH)),
|
||||
h("div", { class: "row" },
|
||||
field(t("pg.steps"), imSteps),
|
||||
field("CFG", imCfg, t("pg.cfgHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, imSeed, dice(imSeed)))),
|
||||
h("div", { class: "actions" }, imCancel, imGo),
|
||||
imBar
|
||||
),
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), imOut, imGallery.host)
|
||||
),
|
||||
decision
|
||||
)
|
||||
imMetrics
|
||||
);
|
||||
}
|
||||
|
||||
// ════════════════════════════ video panel ════════════════════════════
|
||||
let videoPanel;
|
||||
if (!videoModels.length) {
|
||||
videoPanel = noModelsHint(t("pg.tab.video"));
|
||||
} else {
|
||||
const vModel = h("select", {}, ...videoModels.map((m) => h("option", { value: m.name }, `${m.name} · ${fmtBytes(m.size_bytes)}`)));
|
||||
const vPrompt = h("textarea", { placeholder: t("pg.videoPromptPlaceholder"), rows: 3 });
|
||||
const vW = h("input", { type: "number", step: 32, value: 512 });
|
||||
const vH = h("input", { type: "number", step: 32, value: 288 });
|
||||
const vFrames = h("input", { type: "number", value: 39 });
|
||||
const vSteps = h("input", { type: "number", value: 4 });
|
||||
const vSeed = h("input", { type: "number", value: 42 });
|
||||
const vQuality = h("input", { type: "number", min: 10, max: 100, value: 92 });
|
||||
const vFirst = h("input", { type: "file", accept: "image/*" });
|
||||
const vLast = h("input", { type: "file", accept: "image/*" });
|
||||
const vBar = h("div", {});
|
||||
const vOut = h("div", { class: "media-out" }, h("div", { class: "empty" }, t("pg.videoEmpty")));
|
||||
const vMetrics = h("div", { class: "card routecard" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))), h("div", { class: "empty" }, "—"));
|
||||
const vGo = h("button", { class: "btn primary" }, "🎬 " + t("pg.generate"));
|
||||
const vCancelRef = { id: null };
|
||||
const vCancel = mediaControls(vGo, vCancelRef);
|
||||
|
||||
vGo.addEventListener("click", async () => {
|
||||
const prompt = vPrompt.value.trim();
|
||||
if (!prompt) return;
|
||||
vGo.disabled = true;
|
||||
vCancel.style.display = "";
|
||||
mount(vOut, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.generating")));
|
||||
const params = {
|
||||
kind: "video",
|
||||
model: vModel.value,
|
||||
prompt,
|
||||
width: +vW.value || 512,
|
||||
height: +vH.value || 288,
|
||||
frames: +vFrames.value || 39,
|
||||
steps: +vSteps.value || 4,
|
||||
seed: +vSeed.value || 42,
|
||||
quality: +vQuality.value || 92,
|
||||
};
|
||||
try {
|
||||
if (vFirst.files[0]) params.first_frame_b64 = await fileToPpmB64(vFirst.files[0], params.width, params.height);
|
||||
if (vLast.files[0]) params.last_frame_b64 = await fileToPpmB64(vLast.files[0], params.width, params.height);
|
||||
const r = await api.mediaGenerate(params);
|
||||
vCancelRef.id = r.job;
|
||||
const t0 = performance.now();
|
||||
watchJob(r.job, { bar: vBar, goBtn: vGo, cancelBtn: vCancel }, async (j) => {
|
||||
mount(vBar);
|
||||
vGallery.refresh();
|
||||
const secs = (performance.now() - t0) / 1000;
|
||||
try {
|
||||
await showVideoResult(j, vOut);
|
||||
const durS = (j.frames || 0) / 24;
|
||||
mount(
|
||||
vMetrics,
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("pg.genMetrics"))),
|
||||
kv(t("pg.model"), h("span", { class: "mono" }, j.model)),
|
||||
kv(t("pg.resolution"), `${j.width}×${j.height}`),
|
||||
kv(t("pg.frames"), `${j.frames} @ 24 fps`),
|
||||
kv(t("pg.duration"), durS.toFixed(1) + " s"),
|
||||
kv(t("pg.steps"), String(j.steps)),
|
||||
kv(t("pg.seed"), String(j.seed)),
|
||||
kv(t("pg.genTime"), secs.toFixed(1) + " s"),
|
||||
kv(t("pg.timePerFrame"), j.frames ? (secs / j.frames).toFixed(2) + " s" : "—"),
|
||||
kv(t("pg.fileSize"), fmtBytes(j.out_bytes))
|
||||
);
|
||||
} catch (e) {
|
||||
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
});
|
||||
} catch (e) {
|
||||
vGo.disabled = false;
|
||||
vCancel.style.display = "none";
|
||||
mount(vOut, h("div", { class: "callout" }, String(e.message || e)));
|
||||
}
|
||||
});
|
||||
|
||||
async function showVideoResult(j, host) {
|
||||
const downloads = h("div", { class: "actions" });
|
||||
for (const kind of j.outputs || []) {
|
||||
const label = { video: "⬇ AVI", audio: "⬇ WAV", mp4: "⬇ MP4" }[kind];
|
||||
if (!label) continue;
|
||||
const b = h("button", { class: "btn sm" }, label);
|
||||
b.addEventListener("click", async () => {
|
||||
const { url } = await mediaFileUrl(j.id, kind);
|
||||
const ext = kind === "video" ? "avi" : kind === "audio" ? "wav" : "mp4";
|
||||
h("a", { href: url, download: `cortiq-${j.id}.${ext}` }).click();
|
||||
});
|
||||
downloads.appendChild(b);
|
||||
}
|
||||
if ((j.outputs || []).includes("mp4")) {
|
||||
// ffmpeg was present server-side → native playback
|
||||
const { url } = await mediaFileUrl(j.id, "mp4");
|
||||
mount(host, h("video", { src: url, controls: true, class: "media-canvas" }), downloads);
|
||||
return;
|
||||
}
|
||||
// no ffmpeg anywhere: decode the MJPEG AVI ourselves, sync to the wav
|
||||
const buf = await mediaFileBytes(j.id, "video");
|
||||
const frames = aviJpegFrames(buf);
|
||||
let audio = null;
|
||||
if ((j.outputs || []).includes("audio")) {
|
||||
const { url } = await mediaFileUrl(j.id, "audio");
|
||||
audio = h("audio", { src: url });
|
||||
}
|
||||
const player = mjpegPlayer(frames, 24, audio);
|
||||
player.canvas.className = "media-canvas";
|
||||
await player.seek0();
|
||||
const playBtn = h("button", { class: "btn primary sm" }, "▶ " + t("pg.play"));
|
||||
playBtn.addEventListener("click", async () => {
|
||||
if (player.state.playing) {
|
||||
player.pause();
|
||||
playBtn.textContent = "▶ " + t("pg.play");
|
||||
} else {
|
||||
playBtn.textContent = "⏸";
|
||||
await player.play();
|
||||
}
|
||||
});
|
||||
mount(host, player.canvas, h("div", { class: "actions" }, playBtn, ...downloads.childNodes ? [downloads] : []));
|
||||
}
|
||||
|
||||
const vGallery = buildGallery("video", (j) => showVideoResult(j, vOut));
|
||||
|
||||
videoPanel = h(
|
||||
"div",
|
||||
{ class: "pg" },
|
||||
h(
|
||||
"div",
|
||||
{},
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
field(t("pg.model"), vModel),
|
||||
field(t("pg.prompt"), vPrompt),
|
||||
h("div", { class: "row" }, field(t("pg.width"), vW, t("pg.videoSizeHint")), field(t("pg.height"), vH)),
|
||||
h("div", { class: "row" }, field(t("pg.frames"), vFrames, t("pg.framesHint")), field(t("pg.steps"), vSteps, t("pg.videoStepsHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("pg.seed"), h("div", { class: "flex", style: "gap:6px" }, vSeed, dice(vSeed))),
|
||||
field(t("pg.quality"), vQuality)),
|
||||
h("div", { class: "row" }, field(t("pg.firstFrame"), vFirst, t("pg.keyframeHint")), field(t("pg.lastFrame"), vLast)),
|
||||
h("div", { class: "actions" }, vCancel, vGo),
|
||||
vBar
|
||||
),
|
||||
h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.result"))), vOut, vGallery.host)
|
||||
),
|
||||
vMetrics
|
||||
);
|
||||
}
|
||||
|
||||
const panels = { chat: chatPanel, image: imagePanel, video: videoPanel };
|
||||
|
||||
mount(
|
||||
root,
|
||||
h("div", { class: "page-head" }, h("h2", {}, t("pg.title")), h("p", {}, t("pg.subtitle"))),
|
||||
tabsSeg,
|
||||
panelHost
|
||||
);
|
||||
mount(panelHost, panels.chat);
|
||||
return root;
|
||||
}
|
||||
|
||||
Vendored
+262
-32
@@ -1,74 +1,293 @@
|
||||
// Routing — tier editor (ordered model chips), default model, and policy.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { t, trTier } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
|
||||
// Known labels of the hosted taxonomy — used for localized names/descriptions
|
||||
// and as the offline fallback; the live list comes from /routing/labels.
|
||||
const TASK_META = {
|
||||
chitchat: { icon: "💬" },
|
||||
code: { icon: "⌨" },
|
||||
"creative-writing": { icon: "✍" },
|
||||
extraction: { icon: "📑" },
|
||||
math: { icon: "∑" },
|
||||
qa: { icon: "❓" },
|
||||
summarization: { icon: "📋" },
|
||||
translation: { icon: "🌍" },
|
||||
};
|
||||
|
||||
export async function renderRouting() {
|
||||
const meta = appState.meta || (await api.meta());
|
||||
const [routing, modelsData] = await Promise.all([api.getRouting(), api.listModels()]);
|
||||
const ids = (modelsData.models || []).map((m) => m.id);
|
||||
|
||||
// Task labels: live from the router when reachable, built-in set otherwise;
|
||||
// labels that already have a rule are always shown (custom taxonomies).
|
||||
let taskLabels = Object.keys(TASK_META);
|
||||
let labelsSource = "builtin";
|
||||
try {
|
||||
const r = await api.routingLabels();
|
||||
if (r.labels && r.labels.length) {
|
||||
taskLabels = r.labels;
|
||||
labelsSource = "router";
|
||||
}
|
||||
} catch {
|
||||
/* offline — builtin list */
|
||||
}
|
||||
for (const k of Object.keys(routing.tasks || {})) {
|
||||
if (!taskLabels.includes(k)) taskLabels.push(k);
|
||||
}
|
||||
|
||||
// Share of recent traffic per task — shows which rules actually matter.
|
||||
let taskShare = {};
|
||||
try {
|
||||
const stat = await api.stats("7d", "label");
|
||||
const total = (stat.breakdown || []).reduce((a, b) => a + (b.requests || 0), 0);
|
||||
if (total > 0) {
|
||||
for (const b of stat.breakdown || []) taskShare[b.key] = (b.requests || 0) / total;
|
||||
}
|
||||
} catch {
|
||||
/* no stats yet */
|
||||
}
|
||||
|
||||
// local editable state — ensure standard tiers exist for a friendly editor
|
||||
const tiers = Object.assign({ low: [], medium: [], high: [] }, routing.tiers || {});
|
||||
const tasks = {};
|
||||
// A task rule is edited as ordered PAIRS (model, tier): "this model serves
|
||||
// this task at this complexity" (tier "any" = at every complexity).
|
||||
// GET returns {any:[], low:[], medium:[], high:[]}; legacy arrays = any.
|
||||
const taskPairs = {};
|
||||
for (const l of taskLabels) {
|
||||
const v = (routing.tasks || {})[l];
|
||||
const pairs = [];
|
||||
if (Array.isArray(v)) for (const m of v) pairs.push({ model: m, tier: "any" });
|
||||
else if (v && typeof v === "object") {
|
||||
for (const tier of ["low", "medium", "high", "any"])
|
||||
for (const m of v[tier] || []) pairs.push({ model: m, tier });
|
||||
}
|
||||
taskPairs[l] = pairs;
|
||||
}
|
||||
const st = {
|
||||
tiers,
|
||||
taskPairs,
|
||||
bands: routing.bands || null, // null = пороги роутера как есть
|
||||
default: routing.default || ids[0] || "",
|
||||
policy: routing.policy || { mode: "fixed_table", min_class: {} },
|
||||
};
|
||||
|
||||
const root = h("div");
|
||||
|
||||
function moveChip(tier, i, dir) {
|
||||
const arr = st.tiers[tier];
|
||||
function moveChip(map, key, i, dir) {
|
||||
const arr = map[key];
|
||||
const ni = i + dir;
|
||||
if (ni < 0 || ni >= arr.length) return;
|
||||
[arr[i], arr[ni]] = [arr[ni], arr[i]];
|
||||
render();
|
||||
}
|
||||
function removeChip(tier, i) {
|
||||
st.tiers[tier].splice(i, 1);
|
||||
function removeChip(map, key, i) {
|
||||
map[key].splice(i, 1);
|
||||
render();
|
||||
}
|
||||
function addChip(tier, id) {
|
||||
if (id && !st.tiers[tier].includes(id)) st.tiers[tier].push(id);
|
||||
function addChip(map, key, id) {
|
||||
if (id && !map[key].includes(id)) map[key].push(id);
|
||||
render();
|
||||
}
|
||||
|
||||
function tierRow(tier) {
|
||||
const arr = st.tiers[tier];
|
||||
function chipEditor(map, key, emptyLabel) {
|
||||
const arr = map[key];
|
||||
const avail = ids.filter((id) => !arr.includes(id));
|
||||
const addSel = h(
|
||||
"select",
|
||||
{
|
||||
style: "width:auto",
|
||||
onChange: (e) => {
|
||||
addChip(tier, e.target.value);
|
||||
},
|
||||
},
|
||||
{ style: "width:auto", onChange: (e) => addChip(map, key, e.target.value) },
|
||||
h("option", { value: "" }, t("routing.addModel")),
|
||||
...avail.map((id) => h("option", { value: id }, id))
|
||||
);
|
||||
return h(
|
||||
"div",
|
||||
{ class: "chips" },
|
||||
arr.length === 0 && emptyLabel ? h("span", { class: "muted small chips-empty" }, emptyLabel) : null,
|
||||
...arr.map((id, i) =>
|
||||
h(
|
||||
"div",
|
||||
{ class: "chip" },
|
||||
h("span", { class: "ord" }, i + 1),
|
||||
id,
|
||||
h("span", { class: "arrow", title: "←", onClick: () => moveChip(map, key, i, -1) }, "◄"),
|
||||
h("span", { class: "arrow", title: "→", onClick: () => moveChip(map, key, i, 1) }, "►"),
|
||||
h("span", { class: "x", onClick: () => removeChip(map, key, i) }, "×")
|
||||
)
|
||||
),
|
||||
avail.length ? addSel : null
|
||||
);
|
||||
}
|
||||
|
||||
// Linked boundary sliders: one track, two handles (low|medium boundaries),
|
||||
// three colored zones. Handles push each other, numbers update live —
|
||||
// nobody types score ranges by hand.
|
||||
function bandsCard() {
|
||||
const DEF = { low: 0.33, medium: 0.48 };
|
||||
const enabled = !!st.bands;
|
||||
const b = st.bands || { ...DEF };
|
||||
const MIN_GAP = 0.02;
|
||||
|
||||
const track = h("div", { class: "bands-track" });
|
||||
const zoneLow = h("div", { class: "bands-zone z-low" });
|
||||
const zoneMed = h("div", { class: "bands-zone z-med" });
|
||||
const zoneHigh = h("div", { class: "bands-zone z-high" });
|
||||
const hLow = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
|
||||
const hMed = h("div", { class: "bands-handle", tabindex: "0" }, h("span", { class: "bands-val" }));
|
||||
const legend = h("div", { class: "bands-legend" });
|
||||
track.append(zoneLow, zoneMed, zoneHigh, hLow, hMed);
|
||||
|
||||
function paint() {
|
||||
const lo = b.low, me = b.medium;
|
||||
zoneLow.style.width = lo * 100 + "%";
|
||||
zoneMed.style.left = lo * 100 + "%";
|
||||
zoneMed.style.width = (me - lo) * 100 + "%";
|
||||
zoneHigh.style.left = me * 100 + "%";
|
||||
zoneHigh.style.width = (1 - me) * 100 + "%";
|
||||
hLow.style.left = lo * 100 + "%";
|
||||
hMed.style.left = me * 100 + "%";
|
||||
hLow.querySelector(".bands-val").textContent = lo.toFixed(2);
|
||||
hMed.querySelector(".bands-val").textContent = me.toFixed(2);
|
||||
// подписи не наезжают, когда ручки рядом
|
||||
hMed.querySelector(".bands-val").classList.toggle("raised", me - lo < 0.09);
|
||||
mount(legend,
|
||||
h("span", { class: "badge tier-low" }, `${trTier("low")} ≤ ${lo.toFixed(2)}`),
|
||||
h("span", { class: "badge tier-medium" }, `${trTier("medium")} ≤ ${me.toFixed(2)}`),
|
||||
h("span", { class: "badge tier-high" }, `${trTier("high")} > ${me.toFixed(2)}`));
|
||||
track.classList.toggle("disabled", !st.bands);
|
||||
}
|
||||
|
||||
function drag(handle, which) {
|
||||
handle.addEventListener("pointerdown", (e) => {
|
||||
if (!st.bands) return;
|
||||
handle.setPointerCapture(e.pointerId);
|
||||
const rect = track.getBoundingClientRect();
|
||||
const move = (ev) => {
|
||||
let v = (ev.clientX - rect.left) / rect.width;
|
||||
v = Math.min(0.98, Math.max(0.02, Math.round(v * 100) / 100));
|
||||
if (which === "low") {
|
||||
b.low = v;
|
||||
// связанность: low толкает medium перед собой
|
||||
if (b.medium < b.low + MIN_GAP) b.medium = Math.min(0.99, b.low + MIN_GAP);
|
||||
} else {
|
||||
b.medium = v;
|
||||
// и medium толкает low назад
|
||||
if (b.low > b.medium - MIN_GAP) b.low = Math.max(0.01, b.medium - MIN_GAP);
|
||||
}
|
||||
st.bands = { low: b.low, medium: b.medium };
|
||||
paint();
|
||||
};
|
||||
const up = () => {
|
||||
handle.removeEventListener("pointermove", move);
|
||||
handle.removeEventListener("pointerup", up);
|
||||
};
|
||||
handle.addEventListener("pointermove", move);
|
||||
handle.addEventListener("pointerup", up);
|
||||
});
|
||||
}
|
||||
drag(hLow, "low");
|
||||
drag(hMed, "medium");
|
||||
|
||||
const toggle = h("input", { type: "checkbox", checked: enabled ? "" : null });
|
||||
toggle.checked = enabled;
|
||||
toggle.addEventListener("change", () => {
|
||||
st.bands = toggle.checked ? { low: b.low, medium: b.medium } : null;
|
||||
paint();
|
||||
});
|
||||
paint();
|
||||
return h(
|
||||
"div",
|
||||
{ class: "bands-wrap" },
|
||||
h("label", { class: "check", style: "margin-bottom:8px" }, toggle, t("routing.bands.enable")),
|
||||
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.bands.hint")),
|
||||
track,
|
||||
legend
|
||||
);
|
||||
}
|
||||
|
||||
function tierRow(tier) {
|
||||
return h(
|
||||
"div",
|
||||
{ class: "tierrow" },
|
||||
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, tier)),
|
||||
h("div", { class: "tname" }, h("span", { class: "badge tier-" + tier }, trTier(tier))),
|
||||
chipEditor(st.tiers, tier)
|
||||
);
|
||||
}
|
||||
|
||||
function taskRow(label) {
|
||||
const known = TASK_META[label];
|
||||
const name = known ? t("task." + label) : label;
|
||||
const share = taskShare[label];
|
||||
return h(
|
||||
"div",
|
||||
{ class: "tierrow taskrow" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "chips" },
|
||||
...arr.map((id, i) =>
|
||||
h(
|
||||
"div",
|
||||
{ class: "chip" },
|
||||
h("span", { class: "ord" }, i + 1),
|
||||
id,
|
||||
h("span", { class: "arrow", title: "←", onClick: () => moveChip(tier, i, -1) }, "◄"),
|
||||
h("span", { class: "arrow", title: "→", onClick: () => moveChip(tier, i, 1) }, "►"),
|
||||
h("span", { class: "x", onClick: () => removeChip(tier, i) }, "×")
|
||||
)
|
||||
),
|
||||
avail.length ? addSel : null
|
||||
)
|
||||
{ class: "tname taskname" },
|
||||
h("div", {}, (known ? known.icon + " " : "") + name),
|
||||
h("div", { class: "muted small mono" }, label),
|
||||
share != null ? h("div", { class: "muted small" }, t("routing.tasks.share", { p: Math.round(share * 100) })) : null
|
||||
),
|
||||
pairsEditor(label)
|
||||
);
|
||||
}
|
||||
|
||||
// Pairs editor: each row is [model ▾][complexity ▾][×]; "any" serves every
|
||||
// tier. Order matters — it is the fallback order inside the tier.
|
||||
function pairsEditor(label) {
|
||||
const pairs = st.taskPairs[label];
|
||||
const wrap = h("div", { class: "pairs grow" });
|
||||
function paint() {
|
||||
mount(
|
||||
wrap,
|
||||
...pairs.map((p, i) => {
|
||||
const modelSel = h(
|
||||
"select",
|
||||
{ class: "pair-model" },
|
||||
...ids.map((id) => h("option", { value: id }, id))
|
||||
);
|
||||
modelSel.value = p.model;
|
||||
modelSel.addEventListener("change", () => { p.model = modelSel.value; });
|
||||
const tierSel = h(
|
||||
"select",
|
||||
{ class: "pair-tier", title: t("routing.tasks.tierHint") },
|
||||
h("option", { value: "any" }, t("routing.tasks.tierAny")),
|
||||
...Object.keys(st.tiers).map((tr) => h("option", { value: tr }, trTier(tr)))
|
||||
);
|
||||
tierSel.value = p.tier;
|
||||
tierSel.addEventListener("change", () => { p.tier = tierSel.value; });
|
||||
const del = h("button", { class: "btn sm ghost danger", onClick: () => { pairs.splice(i, 1); paint(); } }, "×");
|
||||
return h("div", { class: "pair-row" }, modelSel, tierSel, del);
|
||||
}),
|
||||
pairs.length === 0 ? h("div", { class: "muted small", style: "align-self:center" }, t("routing.tasks.inherit")) : null,
|
||||
h("button", { class: "btn ghost sm", onClick: () => { pairs.push({ model: ids[0] || "", tier: "any" }); paint(); } }, "+ " + t("routing.tasks.addModel"))
|
||||
);
|
||||
}
|
||||
paint();
|
||||
return wrap;
|
||||
}
|
||||
|
||||
function tasksCard() {
|
||||
const addIn = h("input", { placeholder: t("routing.tasks.addPlaceholder"), style: "max-width:220px" });
|
||||
const addBtn = h("button", { class: "btn ghost sm" }, "+ " + t("routing.tasks.addCustom"));
|
||||
addBtn.addEventListener("click", () => {
|
||||
const v = addIn.value.trim().toLowerCase();
|
||||
if (!v || st.taskPairs[v]) return;
|
||||
st.taskPairs[v] = [];
|
||||
render();
|
||||
});
|
||||
return h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" },
|
||||
h("h3", {}, "🎯 " + t("routing.tasks.title")),
|
||||
h("span", { class: "card-head sub" },
|
||||
labelsSource === "router" ? t("routing.tasks.fromRouter") : t("routing.tasks.builtin"))),
|
||||
h("div", { class: "muted small", style: "margin-bottom:10px" }, t("routing.tasks.hint")),
|
||||
...Object.keys(st.taskPairs).map(taskRow),
|
||||
h("div", { class: "flex", style: "gap:8px; margin-top:10px" }, addIn, addBtn)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -116,7 +335,16 @@ export async function renderRouting() {
|
||||
saveBtn.addEventListener("click", async () => {
|
||||
saveBtn.disabled = true;
|
||||
try {
|
||||
await api.putRouting({ tiers: st.tiers, default: st.default, policy: st.policy });
|
||||
const tasks = {};
|
||||
for (const [k, pairs] of Object.entries(st.taskPairs)) {
|
||||
const rule = {};
|
||||
for (const p of pairs) {
|
||||
if (!p.model) continue;
|
||||
(rule[p.tier] = rule[p.tier] || []).push(p.model);
|
||||
}
|
||||
if (Object.keys(rule).length) tasks[k] = rule;
|
||||
}
|
||||
await api.putRouting({ tiers: st.tiers, tasks, bands: st.bands, default: st.default, policy: st.policy });
|
||||
toast(t("toast.saved"), "good");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
@@ -141,8 +369,10 @@ export async function renderRouting() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("routing.tiers")), h("span", { class: "card-head sub" }, t("routing.tierHint"))),
|
||||
bandsCard(),
|
||||
...Object.keys(st.tiers).map(tierRow)
|
||||
),
|
||||
tasksCard(),
|
||||
h(
|
||||
"div",
|
||||
{ class: "card" },
|
||||
|
||||
Vendored
+198
-17
@@ -1,7 +1,7 @@
|
||||
// Settings — router/route/breaker/log/telemetry/stats + export/import config.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { h, mount, toast, confirmDialog } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { api, getToken, postRaw } from "../api.js";
|
||||
import { appState, SITE_URL } from "../app.js";
|
||||
|
||||
// A small "?" badge with a hover/focus tooltip explaining a setting.
|
||||
@@ -194,8 +194,30 @@ export async function renderSettings() {
|
||||
const modelsData = await api.listModels();
|
||||
const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
|
||||
const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled, t("settings.cache.enabledHelp"));
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
|
||||
const cacheThresh = h("input", { type: "number", step: "0.01", min: "0.5", max: "0.999", value: Math.round(((s.cache?.threshold ?? 0.92) + Number.EPSILON) * 100) / 100 });
|
||||
const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
|
||||
// Live proof-of-life for the cache card: entries/hits/saved — and an
|
||||
// honest warning when the cache is on but has no embedding model to work
|
||||
// with (it silently does nothing in that state).
|
||||
let cacheStats = null;
|
||||
try {
|
||||
cacheStats = (await api.stats("24h", "model")).cache || null;
|
||||
} catch { /* stats off */ }
|
||||
function cacheStatus() {
|
||||
const wrap = h("div", { style: "margin-bottom:10px" });
|
||||
if (s.cache?.enabled && embedModels.length === 0 && !s.cache?.embed_model) {
|
||||
mount(wrap, h("div", { class: "callout warn" }, "⚠ " + t("settings.cache.noEmbed")));
|
||||
return wrap;
|
||||
}
|
||||
if (cacheStats && s.cache?.enabled) {
|
||||
mount(wrap, h("div", { class: "flex wrap", style: "gap:8px" },
|
||||
h("span", { class: "badge" }, t("settings.cache.entries") + ": " + cacheStats.entries),
|
||||
h("span", { class: "badge ok" }, t("settings.cache.hits") + ": " + cacheStats.hits),
|
||||
h("span", { class: "badge" }, "hit-rate " + Math.round((cacheStats.hit_rate || 0) * 100) + "%"),
|
||||
h("span", { class: "badge" }, t("settings.cache.saved") + " $" + (cacheStats.saved_usd || 0).toFixed(4))));
|
||||
}
|
||||
return wrap;
|
||||
}
|
||||
const cacheEmbedSel = h(
|
||||
"select",
|
||||
{},
|
||||
@@ -253,9 +275,103 @@ export async function renderSettings() {
|
||||
} catch (e) { mount(st, String(e.message || e)); }
|
||||
checkBtn.disabled = false;
|
||||
});
|
||||
const row = h("div", { class: "flex wrap", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), removeBtn, st);
|
||||
const entry = { node: row, read: () => ({ id: idIn.value.trim() || autoId(modelSel.value), model: modelSel.value.trim(), port: parseInt(portIn.value) || 8090, threads: parseInt(threadsIn.value) || 0, gpu: gpuIn.checked }) };
|
||||
|
||||
// ── O(1) Nyström Attention ──
|
||||
// NOT a "reasoning mode". This is a memory-efficient streaming attention
|
||||
// kernel (Nyström/landmark) that replaces standard KV-cache attention,
|
||||
// giving O(1) memory per decode step instead of O(n).
|
||||
const o1Sel = h("select", { style: "width:120px", title: "O(1) Nyström attention layer spec: which layers replace KV-cache with streaming Nyström attention" },
|
||||
h("option", { value: "" }, "O(1) авто"),
|
||||
h("option", { value: "all" }, "O(1) все слои"),
|
||||
h("option", { value: "deep" }, "O(1) глубокие"),
|
||||
h("option", { value: "off" }, "O(1) выкл."));
|
||||
if (sv.o1) o1Sel.value = sv.o1;
|
||||
const o1mIn = h("input", { type: "number", min: "4", placeholder: "32", value: sv.o1_m ?? "", style: "width:55px", title: "Landmark budget (≥4, default 32). More landmarks ≠ better: m=64 measured worse." });
|
||||
const o1wIn = h("input", { type: "number", min: "1", placeholder: "128", value: sv.o1_window ?? "", style: "width:60px", title: "Exact-window width — main quality lever (default 128)" });
|
||||
const o1sinkIn = h("input", { type: "number", min: "0", placeholder: "4", value: sv.o1_sink ?? "", style: "width:50px", title: "Permanent exact sink keys, StreamingLLM discipline (default 4)" });
|
||||
|
||||
// ── Generation parameters ──
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: sv.temperature ?? "", style: "width:60px" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: sv.top_p ?? "", style: "width:60px" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: sv.max_tokens ?? "", style: "width:75px" });
|
||||
// think_budget = 0 means reasoning disabled (enable_thinking=false);
|
||||
// null = model decides; N>0 = token budget for the <think> block.
|
||||
const noThinkCb = h("input", { type: "checkbox", checked: (sv.think_budget === 0) ? true : null, title: "Отключить режим размышлений (think_budget=0, enable_thinking=false для Qwen3/3.5)" });
|
||||
const thinkIn = h("input", { type: "number", min: "1", placeholder: "авто", value: (sv.think_budget && sv.think_budget > 0) ? sv.think_budget : "", style: "width:75px", title: "Бюджет токенов на размышления (think_budget). Пустое = модель решает сама." });
|
||||
if (sv.think_budget === 0) thinkIn.disabled = true;
|
||||
noThinkCb.addEventListener("change", () => {
|
||||
thinkIn.disabled = noThinkCb.checked;
|
||||
if (noThinkCb.checked) thinkIn.value = "";
|
||||
});
|
||||
const sysIn = h("input", { placeholder: "System prompt (optional)", value: sv.system_prompt || "", style: "flex:1;min-width:200px" });
|
||||
|
||||
// ── MTP (Multi-Token Prediction): speculative decoding, NOT reasoning ──
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: sv.skip_mtp ? true : null, title: "Disable MTP speculative decoding (env CMF_MTP=0)" });
|
||||
|
||||
// ── Settings drawer ──
|
||||
const drawerStyle = "display:none;width:100%;margin-top:6px;padding:10px 12px;background:var(--bg-subtle);border-radius:6px;border:1px solid var(--border)";
|
||||
const groupStyle = "display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:4px 0";
|
||||
const labelStyle = "font-size:11px;color:var(--text-muted);font-weight:600;text-transform:uppercase;letter-spacing:0.5px;width:100%;margin-bottom:2px";
|
||||
|
||||
const drawer = h("div", { style: drawerStyle },
|
||||
// Group 1: O(1) Nyström Attention
|
||||
h("div", { style: labelStyle }, "O(1) Nyström Attention"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Слои:"), o1Sel,
|
||||
h("span", { class: "small muted" }, "m:"), o1mIn,
|
||||
h("span", { class: "small muted" }, "Window:"), o1wIn,
|
||||
h("span", { class: "small muted" }, "Sink:"), o1sinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 2: Generation
|
||||
h("div", { style: labelStyle }, "Генерация"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Temp:"), tempIn,
|
||||
h("span", { class: "small muted" }, "Top-P:"), topPIn,
|
||||
h("span", { class: "small muted" }, "Max токенов:"), maxTokIn
|
||||
),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "System prompt:"), sysIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 3: Reasoning (thinking)
|
||||
h("div", { style: labelStyle }, "Режим размышлений"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "Для моделей Qwen3/3.5: enable_thinking=false — модель отвечает напрямую без блока <think>" }, noThinkCb, "Отключить размышления"),
|
||||
h("span", { class: "small muted" }, "Think budget:"), thinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 4: Runtime
|
||||
h("div", { style: labelStyle }, "Runtime"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "MTP — Multi-Token Prediction (спекулятивное декодирование). Отключение замедляет генерацию, но стабилизирует output." }, skipMtpCb, "Отключить MTP")
|
||||
)
|
||||
);
|
||||
|
||||
const gearBtn = h("button", { class: "btn small", type: "button", title: "⚙ Настройки модели" }, "⚙");
|
||||
gearBtn.addEventListener("click", () => {
|
||||
drawer.style.display = drawer.style.display === "none" ? "block" : "none";
|
||||
});
|
||||
|
||||
const row = h("div", { class: "flex wrap cmf-server-row", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), gearBtn, removeBtn, st, drawer);
|
||||
const entry = { node: row, read: () => ({
|
||||
id: idIn.value.trim() || autoId(modelSel.value),
|
||||
model: modelSel.value.trim(),
|
||||
port: parseInt(portIn.value) || 8090,
|
||||
threads: parseInt(threadsIn.value) || 0,
|
||||
gpu: gpuIn.checked,
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: noThinkCb.checked ? 0 : (thinkIn.value !== "" ? parseInt(thinkIn.value) : null),
|
||||
o1: o1Sel.value || null,
|
||||
o1_m: o1mIn.value !== "" ? parseInt(o1mIn.value) : null,
|
||||
o1_window: o1wIn.value !== "" ? parseInt(o1wIn.value) : null,
|
||||
o1_sink: o1sinkIn.value !== "" ? parseInt(o1sinkIn.value) : null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysIn.value.trim() || null,
|
||||
}) };
|
||||
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
|
||||
serverRows.push(entry);
|
||||
serversWrap.appendChild(row);
|
||||
@@ -386,22 +502,33 @@ export async function renderSettings() {
|
||||
});
|
||||
|
||||
async function exportConfig() {
|
||||
const cfg = await api.getConfig();
|
||||
const blob = new Blob([JSON.stringify(cfg, null, 2)], { type: "application/json" });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = h("a", { href: url, download: "gateway-config.json" });
|
||||
// the real gateway.toml from disk — comments and all
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch("/admin/api/config/export", { headers });
|
||||
if (!r.ok) {
|
||||
toast(await r.text(), "bad");
|
||||
return;
|
||||
}
|
||||
const url = URL.createObjectURL(await r.blob());
|
||||
const a = h("a", { href: url, download: "gateway.toml" });
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
URL.revokeObjectURL(url);
|
||||
}
|
||||
const fileIn = h("input", { type: "file", accept: ".json,application/json", style: "display:none" });
|
||||
const fileIn = h("input", { type: "file", accept: ".toml,.json,application/json", style: "display:none" });
|
||||
fileIn.addEventListener("change", async () => {
|
||||
const f = fileIn.files[0];
|
||||
if (!f) return;
|
||||
try {
|
||||
const cfg = JSON.parse(await f.text());
|
||||
await api.putConfig(cfg);
|
||||
const text = await f.text();
|
||||
if (f.name.endsWith(".json")) {
|
||||
await api.putConfig(JSON.parse(text)); // legacy JSON export
|
||||
} else {
|
||||
await postRaw("/config/import", text); // gateway.toml — validated + hot-reloaded
|
||||
}
|
||||
toast(t("toast.imported"), "good");
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
@@ -409,6 +536,54 @@ export async function renderSettings() {
|
||||
fileIn.value = "";
|
||||
});
|
||||
|
||||
const restartBtn = h("button", { class: "btn danger" }, "⟳ " + t("settings.restart"));
|
||||
restartBtn.addEventListener("click", async () => {
|
||||
if (!(await confirmDialog(t("settings.restartConfirm")))) return;
|
||||
restartBtn.disabled = true;
|
||||
try {
|
||||
await api.restart();
|
||||
toast(t("settings.restarting"), "good");
|
||||
// the gateway comes back on the same port — reload once it answers
|
||||
const poll = setInterval(async () => {
|
||||
try {
|
||||
await api.health();
|
||||
clearInterval(poll);
|
||||
location.reload();
|
||||
} catch {
|
||||
/* still restarting */
|
||||
}
|
||||
}, 1000);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
restartBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
const stopBtn = h("button", { class: "btn danger" }, "\u23fb " + t("settings.shutdown"));
|
||||
stopBtn.addEventListener("click", async () => {
|
||||
if (!(await confirmDialog(t("settings.shutdownConfirm")))) return;
|
||||
stopBtn.disabled = true;
|
||||
restartBtn.disabled = true;
|
||||
try {
|
||||
await api.shutdown();
|
||||
// the server is gone in ~300ms — replace the app with a static farewell
|
||||
document.body.innerHTML = "";
|
||||
document.body.appendChild(
|
||||
h(
|
||||
"div",
|
||||
{ class: "shutdown-screen" },
|
||||
h("div", { class: "shutdown-icon" }, "\u23fb"),
|
||||
h("h2", {}, t("settings.shutdownDone")),
|
||||
h("p", { class: "hint" }, t("settings.shutdownHint"))
|
||||
)
|
||||
);
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
stopBtn.disabled = false;
|
||||
restartBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
mount(
|
||||
root,
|
||||
h(
|
||||
@@ -442,9 +617,11 @@ export async function renderSettings() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.route"))),
|
||||
field(t("settings.route.strategy"), strategySel),
|
||||
h("div", { class: "row" }, field(t("settings.route.maxChars"), maxChars), field(t("settings.route.cacheTtl"), cacheTtl)),
|
||||
field(t("settings.route.profile"), profileSel)
|
||||
field(t("settings.route.strategy"), strategySel, null, t("settings.route.strategyHelp")),
|
||||
h("div", { class: "row" },
|
||||
field(t("settings.route.maxChars"), maxChars, null, t("settings.route.maxCharsHelp")),
|
||||
field(t("settings.route.cacheTtl"), cacheTtl, null, t("settings.route.cacheTtlHelp"))),
|
||||
field(t("settings.route.profile"), profileSel, null, t("settings.route.profileHelp"))
|
||||
),
|
||||
h(
|
||||
"div",
|
||||
@@ -480,6 +657,7 @@ export async function renderSettings() {
|
||||
"div",
|
||||
{ class: "card" },
|
||||
h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
|
||||
cacheStatus(),
|
||||
cacheEnabled.node,
|
||||
h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
|
||||
field(t("settings.cache.embed"), cacheEmbedSel),
|
||||
@@ -527,7 +705,10 @@ export async function renderSettings() {
|
||||
{ class: "flex wrap" },
|
||||
h("button", { class: "btn", onClick: exportConfig }, t("settings.export")),
|
||||
h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
|
||||
fileIn
|
||||
fileIn,
|
||||
h("div", { class: "grow" }),
|
||||
restartBtn,
|
||||
stopBtn
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user