Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
32ce2f7ed7 | ||
|
|
9ec8861464 | ||
|
|
ad438a5dcb | ||
|
|
45def5c6b6 | ||
|
|
5d8455b4a0 | ||
|
|
b255c60b0b | ||
|
|
a75cacd0ec | ||
|
|
850a54acdf | ||
|
|
22110ac524 | ||
|
|
b3b37d27f4 | ||
|
|
a7c8ed72c7 | ||
|
|
f2e6047bff | ||
|
|
0b28aa3d59 | ||
|
|
1414ad1b52 | ||
|
|
94c8551206 | ||
|
|
6ecc709fd2 | ||
|
|
1661c19550 | ||
|
|
66ee32b233 | ||
|
|
f6ee045989 | ||
|
|
59e2eb3215 | ||
|
|
5b743a4691 | ||
|
|
c59ed9c949 | ||
|
|
b55e72789f | ||
|
|
0d1283c3fa | ||
|
|
0105a43ef5 | ||
|
|
b8d48c31f7 | ||
|
|
196ff52f0f | ||
|
|
e25814a54c | ||
|
|
0c20eb34f0 | ||
|
|
f8c1bb0914 | ||
|
|
0f299b77e6 | ||
|
|
7c49510c46 | ||
|
|
31da84cc45 | ||
|
|
749a47ce9b | ||
|
|
d2b1d50eb8 | ||
|
|
d48349a144 | ||
|
|
f05d26bfbb | ||
|
|
39f1fcb600 | ||
|
|
42377a75d4 | ||
|
|
6cf2608fc8 | ||
|
|
e6d1a630c9 | ||
|
|
8dd77ca7ff | ||
|
|
630149d141 | ||
|
|
85fc508f04 | ||
|
|
010171e05c | ||
|
|
39c56145ed | ||
|
|
17c8186383 | ||
|
|
e773e18efd | ||
|
|
6466006de6 | ||
|
|
6a25d8ac08 | ||
|
|
e486617e51 | ||
|
|
729a8bfbba | ||
|
|
5759de153b | ||
|
|
bf8534f13d | ||
|
|
70d8cde258 | ||
|
|
3d71b03788 | ||
|
|
492953d4c2 | ||
|
|
3b0231b32d | ||
|
|
fd2f2a165b | ||
|
|
cbc385f6c7 |
@@ -1,4 +1,6 @@
|
||||
target
|
||||
models
|
||||
*.cmf
|
||||
.git
|
||||
.github
|
||||
docs
|
||||
|
||||
@@ -21,9 +21,20 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
# Skip when this version is already on crates.io — a version can never be
|
||||
# re-uploaded, so publishing one that exists (released by hand, or a re-run
|
||||
# of this job) is a guaranteed failure rather than a real problem.
|
||||
- name: Publish to crates.io
|
||||
if: ${{ env.CARGO_REGISTRY_TOKEN != '' }}
|
||||
run: cargo publish
|
||||
run: |
|
||||
name=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].name')
|
||||
version=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].version')
|
||||
if curl -sf -H 'User-Agent: cortiq-gateway-release' \
|
||||
"https://crates.io/api/v1/crates/$name/$version" >/dev/null; then
|
||||
echo "$name $version is already published — skipping"
|
||||
else
|
||||
cargo publish
|
||||
fi
|
||||
|
||||
# --- prebuilt binaries + GitHub Release -----------------------------------
|
||||
release:
|
||||
|
||||
+113
@@ -3,8 +3,121 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.37] - 2026-08-06
|
||||
|
||||
### Added
|
||||
- **Zero-argument startup**: `cortiq-gateway` now runs with no flags. The config is created on
|
||||
first launch (repo `config/gateway.toml` when run from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml`, or `$CORTIQ_GATEWAY_HOME`), the browser opens by itself,
|
||||
and a busy port is swapped for the next free one. `--no-browser` / `--no-port-search` opt out;
|
||||
`--config` still takes an explicit path.
|
||||
- **No admin token required by default**: the console opens without auth. Set `--admin-token`,
|
||||
`[admin].token` or `[admin].token_env` to protect it.
|
||||
- **First-run wizard** (`/admin#/onboarding`): pick a language, download a ready-made `.cmf`
|
||||
model, and finish — Finish connects every completed download as a local model, starts its
|
||||
`cortiq serve` and points `routing.default` at it. Nothing to edit by hand.
|
||||
- **Ready-made models are read live** from the `infosave` HuggingFace account in both the wizard
|
||||
and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a
|
||||
delete button. Newly published models appear without a gateway update.
|
||||
- **Windows-friendly data dir**: `%APPDATA%` / `%LOCALAPPDATA%` / `%USERPROFILE%` are used before
|
||||
falling back, so running the exe from `Downloads` never writes into `Program Files`.
|
||||
- **Full localization**: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr).
|
||||
Import, Settings → Local models, the model form and the whole first-run wizard were English-only
|
||||
in de/fr/es/zh/tr before this release.
|
||||
|
||||
### Fixed
|
||||
- **Download progress jumped straight to 100% and then hung**: with `-L`, curl prints a complete
|
||||
`0→100%` progress bar for the redirect hop before the transfer starts, and that bar was being
|
||||
parsed as the download's progress. Progress is now derived from bytes on disk against the size
|
||||
reported by the HuggingFace tree (or a `HEAD` request), so the bar and the reported
|
||||
`done / total` bytes track the real transfer.
|
||||
- **Re-downloading over an existing file reported 100% instantly**: the previous copy is now
|
||||
removed before the transfer starts, so the size poller cannot read stale bytes.
|
||||
- **Downloaded models were not recognised**: the admin UI derived the output file name
|
||||
independently of the gateway, so a finished download did not mark its card as downloaded and the
|
||||
delete button never appeared. The gateway now returns the exact output stem (`local_name`) and
|
||||
the UI sends it back verbatim for download, progress and delete.
|
||||
- **Registering a local model produced an unusable pool entry**: with `manage_server = false` the
|
||||
model was added to the static pool pointing at `cortiq_server_url`, where nothing listens —
|
||||
every call failed with `upstream unavailable`. Registration now always creates a managed server
|
||||
the gateway runs itself, and repairs a stale static entry for the same file.
|
||||
- **Cancel did nothing during a sharded download**: cancellation was only checked between part
|
||||
files, so a multi-GB part had to finish first. The running transfer is now aborted immediately
|
||||
and the partial output removed.
|
||||
- **Deleting a model left the config broken**: removing a `.cmf` now also unregisters the model
|
||||
and stops its managed server instead of leaving a pool entry pointing at a missing file.
|
||||
- **Two variants of one repo collapsed into a single job row**; jobs are now keyed by repo + variant.
|
||||
- Curl's stderr is captured into the job log, so a failed download explains itself instead of
|
||||
showing a bare exit code.
|
||||
|
||||
## [0.2.36] - 2026-07-25
|
||||
|
||||
### Fixed
|
||||
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
|
||||
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
|
||||
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
|
||||
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.
|
||||
|
||||
## [0.2.35] - 2026-07-23
|
||||
|
||||
### Fixed
|
||||
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
|
||||
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.
|
||||
|
||||
## [0.2.34] - 2026-07-23
|
||||
|
||||
### Added
|
||||
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
|
||||
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.
|
||||
|
||||
### Fixed
|
||||
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
|
||||
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.
|
||||
|
||||
## [0.2.31] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
|
||||
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.
|
||||
|
||||
### Fixed
|
||||
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
|
||||
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.
|
||||
|
||||
## [0.2.30] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
|
||||
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.
|
||||
|
||||
|
||||
### Added
|
||||
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
|
||||
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
### Added
|
||||
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
|
||||
gateway + a local Ollama model in one command. GHCR badge in the READMEs.
|
||||
|
||||
Generated
+31
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.20"
|
||||
version = "0.2.37"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
@@ -297,6 +297,7 @@ dependencies = [
|
||||
"futures",
|
||||
"getrandom 0.2.17",
|
||||
"mime_guess",
|
||||
"open",
|
||||
"reqwest",
|
||||
"rust-embed",
|
||||
"serde",
|
||||
@@ -743,6 +744,25 @@ version = "2.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
|
||||
|
||||
[[package]]
|
||||
name = "is-docker"
|
||||
version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "928bae27f42bc99b60d9ac7334e3a21d10ad8f1835a4e12ec3ec0464765ed1b3"
|
||||
dependencies = [
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is-wsl"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "173609498df190136aa7dea1a91db051746d339e18476eed5ca40521f02d7aa5"
|
||||
dependencies = [
|
||||
"is-docker",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is_terminal_polyfill"
|
||||
version = "1.70.2"
|
||||
@@ -874,6 +894,16 @@ version = "1.70.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
||||
|
||||
[[package]]
|
||||
name = "open"
|
||||
version = "5.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f9cfef937e9c486488c7e3d949ae31c0f1d06bdacd75b99c086cb35356e30408"
|
||||
dependencies = [
|
||||
"is-wsl",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot"
|
||||
version = "0.12.5"
|
||||
|
||||
+15
-5
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.20"
|
||||
version = "0.2.37"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
@@ -10,12 +10,16 @@ homepage = "https://api.allaigate.com"
|
||||
readme = "README.md"
|
||||
keywords = ["llm", "gateway", "proxy", "openai", "routing"]
|
||||
categories = ["web-programming::http-server", "command-line-utilities"]
|
||||
# Files shipped in the published crate. The embedded admin SPA (web/) MUST be
|
||||
# included because rust-embed compiles it in; docs/screenshots are excluded to
|
||||
# keep the crate lean.
|
||||
# Files shipped in the published crate. The embedded admin SPA MUST be included
|
||||
# because rust-embed compiles it in; docs/screenshots are excluded to keep the
|
||||
# crate lean. The web/ subfolders are listed one by one rather than as web/**/*
|
||||
# so stray scratch files dropped next to the SPA never end up published.
|
||||
include = [
|
||||
"src/**/*",
|
||||
"web/**/*",
|
||||
"web/index.html",
|
||||
"web/favicon.svg",
|
||||
"web/css/**/*",
|
||||
"web/js/**/*",
|
||||
"config/gateway.example.toml",
|
||||
"README.md",
|
||||
"CHANGELOG.md",
|
||||
@@ -54,6 +58,8 @@ tokio-stream = "0.1"
|
||||
|
||||
# CLI / config
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
# open browser on first run (optional)
|
||||
open = { version = "5", optional = true }
|
||||
|
||||
# errors / logging / metrics
|
||||
thiserror = "1"
|
||||
@@ -61,6 +67,10 @@ anyhow = "1"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
|
||||
[features]
|
||||
default = ["open-browser"]
|
||||
open-browser = ["open"]
|
||||
|
||||
[profile.release]
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
|
||||
+13
-5
@@ -1,5 +1,7 @@
|
||||
# ---- build stage ------------------------------------------------------------
|
||||
FROM rust:1-slim AS builder
|
||||
# pin builder and runtime to the same Debian release — a floating `rust:1-slim`
|
||||
# drifts to a newer glibc than the runtime image and the binary refuses to start
|
||||
FROM rust:1-slim-trixie AS builder
|
||||
WORKDIR /app
|
||||
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
@@ -8,18 +10,24 @@ COPY web ./web
|
||||
RUN cargo build --release
|
||||
|
||||
# ---- runtime stage ----------------------------------------------------------
|
||||
FROM debian:bookworm-slim
|
||||
FROM debian:trixie-slim
|
||||
# curl is required at runtime — model imports download ready .cmf files with it
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& useradd -r -u 10001 cortiq \
|
||||
&& mkdir -p /app/config
|
||||
&& mkdir -p /app/config /app/data/models
|
||||
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
|
||||
# ship the example as the default config so `docker run` works out of the box
|
||||
COPY config/gateway.example.toml /app/config/gateway.toml
|
||||
RUN chown -R 10001:10001 /app
|
||||
WORKDIR /app
|
||||
USER cortiq
|
||||
# all gateway state (downloaded models, stats, secrets) lives under /app/data —
|
||||
# mount a volume there to keep it across container recreates
|
||||
ENV CORTIQ_GATEWAY_HOME=/app/data
|
||||
# runs as root so writes to bind-mounted host folders (Docker Desktop on
|
||||
# Windows/macOS) don't fail with permission errors (curl exit code 23);
|
||||
# add `--user 10001` to docker run to harden when not using host bind mounts
|
||||
EXPOSE 9000
|
||||
ENTRYPOINT ["cortiq-gateway"]
|
||||
CMD ["--config", "/app/config/gateway.toml"]
|
||||
|
||||
@@ -36,6 +36,7 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Zero-config start.** `cortiq-gateway` with no flags: the config is created for you, a busy port is swapped for a free one, the browser opens on a setup wizard that downloads a ready-made local model and wires it into routing — no TOML, no token, no separate model server.
|
||||
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
|
||||
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
|
||||
@@ -54,11 +55,13 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
```bash
|
||||
# Docker (GitHub Container Registry)
|
||||
docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
-v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
-v "$PWD/config:/app/config" -v cortiq-data:/app/data \
|
||||
ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
# → admin console at http://localhost:9000/admin
|
||||
|
||||
# from crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # no flags, no config file, no token — see Quick start
|
||||
|
||||
# or build from source
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -67,7 +70,10 @@ cargo build --release # ./target/release/cortiq-gateway
|
||||
```
|
||||
|
||||
The image ships a default config and runs out of the box; mount `/app/config` (with a
|
||||
`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
|
||||
`gateway.toml`) to use your own. Downloaded models and stats live in `/app/data` — mount
|
||||
a volume there so they survive container recreates (a host folder works too, e.g.
|
||||
`-v "$PWD/models:/app/data/models"`). Tags: `:latest`, `:X.Y.Z`, `:edge`
|
||||
(linux/amd64 + arm64).
|
||||
|
||||
**Docker Compose** — gateway + a local Ollama model in one command:
|
||||
|
||||
@@ -98,32 +104,54 @@ The gateway ships with an embedded web console at **`/admin`**:
|
||||
|  |
|
||||
| test the live pipeline and see the routing decision |
|
||||
|
||||
The console is **open by default** — no token needed. To protect it, set any one of
|
||||
`--admin-token <TOKEN>`, `[admin].token`, or `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <YOUR_TOKEN>
|
||||
cortiq-gateway --admin-token <YOUR_TOKEN>
|
||||
# open http://localhost:9000/admin?token=<YOUR_TOKEN>
|
||||
```
|
||||
|
||||
If `--admin-token` / `[admin].token_env` are not set, a token is generated at startup
|
||||
and printed to the log. All `/admin/api/*` endpoints require a Bearer admin token;
|
||||
secret values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
Once a token is set, every `/admin/api/*` endpoint requires it as a Bearer token. Secret
|
||||
values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick start
|
||||
|
||||
```bash
|
||||
# 1. start a router (the hosted allaigate router, or a local cortiq-router)
|
||||
# it listens e.g. on http://localhost:8080 (or https://api.allaigate.com)
|
||||
|
||||
# 2. describe your model pool
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. run the gateway
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
# Gateway listens on 0.0.0.0:9000 and serves the admin console at /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
That is the whole command. On first run the gateway:
|
||||
|
||||
1. **creates its config** — `config/gateway.toml` when started from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml` (override with `$CORTIQ_GATEWAY_HOME` or `--config`);
|
||||
2. **picks a free port** if 9000 is taken, and writes the choice back to the config;
|
||||
3. **opens your browser** on the setup wizard at `/admin#/onboarding`.
|
||||
|
||||
The wizard takes you from nothing to a working local model:
|
||||
|
||||
| Step | What happens |
|
||||
|---|---|
|
||||
| **Language** | 7 languages; changeable later from the header |
|
||||
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
|
||||
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
|
||||
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, and `routing.default` points at it |
|
||||
|
||||
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
|
||||
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
|
||||
both still work, and changes hot-reload without a restart.
|
||||
|
||||
```bash
|
||||
# explicit config / port / no browser — all optional
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
For smart routing across a pool, point `[router]` at the hosted allaigate router or a local
|
||||
`cortiq-router` (see [docs/ROUTER.md](docs/ROUTER.md)). Without a router the gateway degrades
|
||||
gracefully to the default model, so a local-only setup works out of the box.
|
||||
|
||||
Now any OpenAI client works through the gateway:
|
||||
|
||||
```python
|
||||
|
||||
+38
-14
@@ -36,6 +36,7 @@
|
||||
|
||||
## ✨ Ключевое
|
||||
|
||||
- **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели.
|
||||
- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
|
||||
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
|
||||
@@ -57,6 +58,7 @@ docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
|
||||
# с crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # без флагов, без конфига, без токена — см. Быстрый старт
|
||||
|
||||
# или сборка из исходников
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -94,32 +96,54 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
||||
|  |  |
|
||||
| **Playground** — тест живого пайплайна с решением роутинга | **Dashboard** — трафик, стоимость, латентность |
|
||||
|
||||
Панель **по умолчанию открыта** — токен не нужен. Чтобы закрыть её, задайте любое из
|
||||
`--admin-token <ТОКЕН>`, `[admin].token` или `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <ВАШ_ТОКЕН>
|
||||
cortiq-gateway --admin-token <ВАШ_ТОКЕН>
|
||||
# откройте http://localhost:9000/admin?token=<ВАШ_ТОКЕН>
|
||||
```
|
||||
|
||||
Если `--admin-token` / `[admin].token_env` не заданы — токен генерируется при старте и
|
||||
печатается в лог. Все эндпоинты `/admin/api/*` требуют Bearer admin-токен; значения
|
||||
секретов API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
Когда токен задан, все эндпоинты `/admin/api/*` требуют его как Bearer. Значения секретов
|
||||
API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Быстрый старт
|
||||
|
||||
```bash
|
||||
# 1. поднимите роутер (хостовый allaigate или локальный cortiq-router)
|
||||
# он слушает, например, http://localhost:8080 (или https://api.allaigate.com)
|
||||
|
||||
# 2. опишите свой пул моделей
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. запустите шлюз
|
||||
cargo run --release -- --config config/gateway.toml
|
||||
# Шлюз слушает 0.0.0.0:9000 и отдаёт панель на /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
Это вся команда. При первом запуске шлюз:
|
||||
|
||||
1. **создаёт конфиг** — `config/gateway.toml`, если запущен из репозитория, иначе
|
||||
`~/.cortiq-gateway/gateway.toml` (переопределяется через `$CORTIQ_GATEWAY_HOME` или `--config`);
|
||||
2. **подбирает свободный порт**, если 9000 занят, и записывает выбор в конфиг;
|
||||
3. **открывает браузер** на мастере настройки `/admin#/onboarding`.
|
||||
|
||||
Мастер доводит от нуля до работающей локальной модели:
|
||||
|
||||
| Шаг | Что происходит |
|
||||
|---|---|
|
||||
| **Язык** | 7 языков; потом меняется в шапке |
|
||||
| **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки |
|
||||
| **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` |
|
||||
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё |
|
||||
|
||||
Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели.
|
||||
Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг
|
||||
руками — оба пути работают, изменения применяются на лету без перезапуска.
|
||||
|
||||
```bash
|
||||
# явный конфиг / порт / без браузера — всё опционально
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
Для умного роутинга по пулу направьте `[router]` на хостовый allaigate или локальный
|
||||
`cortiq-router` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)). Без роутера шлюз мягко
|
||||
деградирует до модели по умолчанию, поэтому чисто локальная установка работает сразу.
|
||||
|
||||
Теперь любой OpenAI-клиент работает через шлюз:
|
||||
|
||||
```python
|
||||
|
||||
@@ -17,6 +17,10 @@ services:
|
||||
CORTIQ_ROUTER_KEY: ${CORTIQ_ROUTER_KEY:-}
|
||||
volumes:
|
||||
- ./config/gateway.compose.toml:/app/config/gateway.toml:ro
|
||||
# downloaded .cmf models + stats persist here across recreates; swap the
|
||||
# named volume for a host folder (e.g. ./models:/app/data/models) if you
|
||||
# want the files visible next to the compose file
|
||||
- cortiq-data:/app/data
|
||||
depends_on:
|
||||
- ollama
|
||||
restart: unless-stopped
|
||||
@@ -30,4 +34,5 @@ services:
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
cortiq-data:
|
||||
ollama:
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ timeout_ms = 15000 # allow for oracle escalation on hard
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <path> only if you keep the config elsewhere
|
||||
```
|
||||
|
||||
## API contract
|
||||
|
||||
+1
-1
@@ -49,7 +49,7 @@ timeout_ms = 15000 # запас под oracle-эскал
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <путь> нужен, только если конфиг лежит в другом месте
|
||||
```
|
||||
|
||||
## Контракт API
|
||||
|
||||
+715
-147
@@ -20,7 +20,7 @@ use axum::{
|
||||
http::{header, StatusCode},
|
||||
middleware::{self, Next},
|
||||
response::{IntoResponse, Response},
|
||||
routing::{get, post},
|
||||
routing::{delete, get, post},
|
||||
Json, Router,
|
||||
};
|
||||
use serde::Deserialize;
|
||||
@@ -139,10 +139,11 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
// ───────────────────────────── router ───────────────────────────────────────
|
||||
|
||||
/// Admin API routes (token-protected). Mounted into the main router.
|
||||
/// Admin API routes. If `admin_token` is empty, the panel is open (no auth);
|
||||
/// otherwise every `/admin/api/*` request needs `Authorization: Bearer <token>`.
|
||||
/// The token is captured in a middleware closure so it does not depend on State.
|
||||
pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
Router::new()
|
||||
let router = Router::new()
|
||||
.route("/admin/api/health", get(health))
|
||||
.route("/admin/api/router/probe", post(router_probe))
|
||||
.route("/admin/api/meta", get(meta))
|
||||
@@ -183,10 +184,18 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
.route("/admin/api/requests", get(get_requests))
|
||||
.route("/admin/api/test", post(run_test))
|
||||
.route("/admin/api/test/stream", post(run_test_stream))
|
||||
.route_layer(middleware::from_fn(move |req: Request, next: Next| {
|
||||
.route("/admin/api/setup/status", get(setup_status))
|
||||
.route("/admin/api/setup/complete", post(setup_complete))
|
||||
.route("/admin/api/setup/featured", get(setup_featured))
|
||||
.route("/admin/api/setup/file", delete(delete_setup_file));
|
||||
if admin_token.is_empty() {
|
||||
router
|
||||
} else {
|
||||
router.route_layer(middleware::from_fn(move |req: Request, next: Next| {
|
||||
let expected = admin_token.clone();
|
||||
async move { auth(expected, req, next).await }
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract Bearer admin token from the `Authorization` header.
|
||||
@@ -467,16 +476,111 @@ async fn provider_models(
|
||||
}
|
||||
// OpenAI + Anthropic both return { "data": [ { "id": ... } ] }.
|
||||
let v: Value = serde_json::from_str(&body).unwrap_or(json!({}));
|
||||
let mut models: Vec<String> = v["data"]
|
||||
let mut models: Vec<Value> = v["data"]
|
||||
.as_array()
|
||||
.map(|arr| {
|
||||
arr.iter()
|
||||
.filter_map(|m| m["id"].as_str().map(|s| s.to_string()))
|
||||
.filter_map(|m| {
|
||||
let id = m["id"].as_str()?.to_string();
|
||||
let mut obj = json!({ "id": id });
|
||||
let mut caps = vec![];
|
||||
if let Some(pricing) = m.get("pricing").filter(|v| !v.is_null()) {
|
||||
if let Some(prompt) = pricing
|
||||
.get("prompt")
|
||||
.and_then(|s| s.as_str())
|
||||
.and_then(|s| s.parse::<f64>().ok())
|
||||
{
|
||||
obj["price_in"] = json!(prompt * 1_000_000.0);
|
||||
}
|
||||
if let Some(comp) = pricing
|
||||
.get("completion")
|
||||
.and_then(|s| s.as_str())
|
||||
.and_then(|s| s.parse::<f64>().ok())
|
||||
{
|
||||
obj["price_out"] = json!(comp * 1_000_000.0);
|
||||
}
|
||||
if pricing
|
||||
.get("input_cache_read")
|
||||
.filter(|v| !v.is_null())
|
||||
.is_some()
|
||||
|| pricing
|
||||
.get("input_cache_write")
|
||||
.filter(|v| !v.is_null())
|
||||
.is_some()
|
||||
{
|
||||
caps.push("caching");
|
||||
}
|
||||
}
|
||||
if let Some(params) = m.get("supported_parameters").and_then(|p| p.as_array()) {
|
||||
let has_tools = params.iter().any(|p| {
|
||||
p.as_str() == Some("tools") || p.as_str() == Some("tool_choice")
|
||||
});
|
||||
if has_tools {
|
||||
caps.push("tools");
|
||||
}
|
||||
let has_json = params.iter().any(|p| p.as_str() == Some("response_format"));
|
||||
if has_json {
|
||||
caps.push("json");
|
||||
}
|
||||
let has_reasoning = params.iter().any(|p| {
|
||||
p.as_str() == Some("reasoning")
|
||||
|| p.as_str() == Some("include_reasoning")
|
||||
});
|
||||
if has_reasoning || m.get("reasoning").filter(|v| !v.is_null()).is_some() {
|
||||
caps.push("reasoning");
|
||||
}
|
||||
}
|
||||
if let Some(arch) = m.get("architecture").filter(|v| !v.is_null()) {
|
||||
if let Some(inputs) =
|
||||
arch.get("input_modalities").and_then(|i| i.as_array())
|
||||
{
|
||||
if inputs.iter().any(|i| i.as_str() == Some("image")) {
|
||||
caps.push("vision");
|
||||
}
|
||||
if inputs.iter().any(|i| i.as_str() == Some("audio")) {
|
||||
caps.push("audio");
|
||||
}
|
||||
}
|
||||
}
|
||||
if !caps.is_empty() {
|
||||
obj["caps"] = json!(caps);
|
||||
}
|
||||
Some(obj)
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
models.sort();
|
||||
ok(json!({ "ok": true, "count": models.len(), "models": models }))
|
||||
models.sort_by(|a, b| a["id"].as_str().cmp(&b["id"].as_str()));
|
||||
|
||||
let mut resp = json!({ "ok": true, "count": models.len(), "models": models });
|
||||
|
||||
if b.provider == "openrouter" {
|
||||
if let Some(k) = key {
|
||||
if let Ok(cred_resp) = client
|
||||
.get("https://openrouter.ai/api/v1/credits")
|
||||
.header("authorization", format!("Bearer {k}"))
|
||||
.send()
|
||||
.await
|
||||
{
|
||||
if let Ok(cred_body) = cred_resp.text().await {
|
||||
let cred_v: Value = serde_json::from_str(&cred_body).unwrap_or(json!({}));
|
||||
if let Some(data) = cred_v.get("data") {
|
||||
let total = data
|
||||
.get("total_credits")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
let usage = data
|
||||
.get("total_usage")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
resp["balance"] = json!(total - usage);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ok(resp)
|
||||
}
|
||||
|
||||
async fn meta() -> ApiResult {
|
||||
@@ -606,10 +710,7 @@ fn cmf_file_for_model(
|
||||
cmf: &crate::config::CmfCfg,
|
||||
m: &crate::config::ModelCfg,
|
||||
) -> Option<std::path::PathBuf> {
|
||||
// Any loopback URL counts as local — managed servers may sit on any port.
|
||||
let local = m.base_url == cmf.cortiq_server_url
|
||||
|| m.base_url.contains("//127.0.0.1:")
|
||||
|| m.base_url.contains("//localhost:")
|
||||
|| m.base_url
|
||||
.contains(&format!("{}:{}", cmf.local_host, cmf.local_port));
|
||||
let stem = m.model.trim();
|
||||
@@ -649,24 +750,12 @@ fn cmf_path_referenced(cfg: &crate::config::Config, path: &std::path::Path) -> b
|
||||
.any(|m| cmf_file_for_model(&cfg.cmf, m).as_deref() == Some(path))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct DeleteModelQuery {
|
||||
/// Also remove the local `.cmf` from disk. Defaults to true (the historical
|
||||
/// behaviour); the UI asks the user and passes it explicitly.
|
||||
delete_file: Option<bool>,
|
||||
}
|
||||
|
||||
async fn delete_model(
|
||||
State(state): State<SharedState>,
|
||||
Path(id): Path<String>,
|
||||
Query(q): Query<DeleteModelQuery>,
|
||||
) -> ApiResult {
|
||||
async fn delete_model(State(state): State<SharedState>, Path(id): Path<String>) -> ApiResult {
|
||||
let mut cfg = current_cfg(&state);
|
||||
let delete_file = q.delete_file.unwrap_or(true);
|
||||
|
||||
// Managed local models live in [cmf], not [[models]]. Deleting one stops its
|
||||
// server, removes it from the managed config, and (when asked) deletes its
|
||||
// file — unless another model still points at the same file.
|
||||
// server, removes it from the managed config, and deletes its file — unless
|
||||
// another model still points at the same file.
|
||||
let managed = cfg.cmf.effective_servers().into_iter().find(|s| s.id == id);
|
||||
if let Some(server) = managed {
|
||||
if !cfg.models.iter().any(|x| x.id == id) {
|
||||
@@ -681,10 +770,10 @@ async fn delete_model(
|
||||
let shared = cmf_path_referenced(&cfg, &path);
|
||||
state.cmf.stop_one(&id).await;
|
||||
state.reload(cfg)?;
|
||||
let file_removed = if delete_file && !shared {
|
||||
remove_cmf_path(&path)
|
||||
} else {
|
||||
let file_removed = if shared {
|
||||
false
|
||||
} else {
|
||||
remove_cmf_path(&path)
|
||||
};
|
||||
return ok(json!({ "ok": true, "file_removed": file_removed }));
|
||||
}
|
||||
@@ -700,8 +789,8 @@ async fn delete_model(
|
||||
let file = victim
|
||||
.as_ref()
|
||||
.and_then(|m| cmf_file_for_model(&cfg.cmf, m));
|
||||
// Only remove the file if asked to AND no remaining model still references it.
|
||||
let file = file.filter(|p| delete_file && !cmf_path_referenced(&cfg, p));
|
||||
// Only remove the file if no remaining model still references it.
|
||||
let file = file.filter(|p| !cmf_path_referenced(&cfg, p));
|
||||
state.reload(cfg)?;
|
||||
let file_removed = file.map(|p| remove_cmf_path(&p)).unwrap_or(false);
|
||||
ok(json!({ "ok": true, "file_removed": file_removed }))
|
||||
@@ -908,101 +997,22 @@ async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBo
|
||||
needs_restart = true;
|
||||
cfg.cache = v;
|
||||
}
|
||||
let mut cmf_apply: Option<(Vec<crate::config::CmfServer>, CmfCfg)> = None;
|
||||
if let Some(v) = b.cmf {
|
||||
let old_servers = cfg.cmf.effective_servers();
|
||||
cfg.cmf = v;
|
||||
// A managed row supersedes a same-id STATIC pool entry that points at a
|
||||
// local server (the shape Import→register creates): drop the static one
|
||||
// instead of failing validation with "duplicate managed model id".
|
||||
let managed_ids: std::collections::HashSet<String> = cfg
|
||||
.cmf
|
||||
.effective_servers()
|
||||
.into_iter()
|
||||
.map(|s| s.id)
|
||||
.collect();
|
||||
cfg.models.retain(|m| {
|
||||
!(managed_ids.contains(&m.id)
|
||||
&& (m.base_url.contains("//127.0.0.1:") || m.base_url.contains("//localhost:")))
|
||||
});
|
||||
// Replacing or renaming a managed model must not brick the save: routing
|
||||
// may still reference the old id. Map a vanished id to its replacement
|
||||
// (the new server on the same port), strip ids that are truly gone, and
|
||||
// repair the default so validate() passes.
|
||||
heal_routing_after_cmf_change(&mut cfg, &old_servers);
|
||||
// Server-list changes apply LIVE after a successful reload: removed /
|
||||
// redefined servers are stopped (RAM freed), new ones are spawned.
|
||||
if cfg.cmf.effective_servers() != old_servers {
|
||||
cmf_apply = Some((old_servers, cfg.cmf.clone()));
|
||||
// manage_server / local_model / threads / gpu apply on restart (the server
|
||||
// is spawned once at startup with those env vars); local_only / router
|
||||
// routing apply immediately.
|
||||
if v.manage_server != cfg.cmf.manage_server
|
||||
|| v.local_model != cfg.cmf.local_model
|
||||
|| v.threads != cfg.cmf.threads
|
||||
|| v.gpu != cfg.cmf.gpu
|
||||
|| v.servers != cfg.cmf.servers
|
||||
{
|
||||
needs_restart = true;
|
||||
}
|
||||
cfg.cmf = v;
|
||||
}
|
||||
state.reload(cfg)?;
|
||||
let applied_live = cmf_apply.is_some();
|
||||
if let Some((old, new_cfg)) = cmf_apply {
|
||||
tokio::spawn(crate::cmf_runtime::apply_server_changes(
|
||||
state.cmf.clone(),
|
||||
old,
|
||||
new_cfg,
|
||||
));
|
||||
}
|
||||
ok(json!({ "ok": true, "needs_restart": needs_restart, "applied_live": applied_live }))
|
||||
}
|
||||
|
||||
/// After the managed-servers list changed, rewrite [routing] so it only
|
||||
/// references ids that still exist. A removed server whose port is now held by
|
||||
/// a NEW server is treated as a replacement (old id → new id); other vanished
|
||||
/// ids are stripped. An emptied default falls back to the first managed server,
|
||||
/// then the first pool model.
|
||||
fn heal_routing_after_cmf_change(cfg: &mut Config, old_servers: &[crate::config::CmfServer]) {
|
||||
let servers = cfg.cmf.effective_servers();
|
||||
let known: std::collections::HashSet<String> = cfg
|
||||
.models
|
||||
.iter()
|
||||
.map(|m| m.id.clone())
|
||||
.chain(servers.iter().map(|s| s.id.clone()))
|
||||
.collect();
|
||||
let new_by_port: HashMap<u16, String> =
|
||||
servers.iter().map(|s| (s.port, s.id.clone())).collect();
|
||||
let mut rename: HashMap<String, String> = HashMap::new();
|
||||
for old in old_servers {
|
||||
if !known.contains(&old.id) {
|
||||
if let Some(new_id) = new_by_port.get(&old.port) {
|
||||
rename.insert(old.id.clone(), new_id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
let fix = |id: &str| -> Option<String> {
|
||||
if known.contains(id) {
|
||||
Some(id.to_string())
|
||||
} else {
|
||||
rename.get(id).cloned()
|
||||
}
|
||||
};
|
||||
if !cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = fix(&cfg.routing.default).unwrap_or_default();
|
||||
}
|
||||
if cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = servers
|
||||
.first()
|
||||
.map(|s| s.id.clone())
|
||||
.or_else(|| cfg.models.first().map(|m| m.id.clone()))
|
||||
.unwrap_or_default();
|
||||
}
|
||||
for tt in cfg.routing.tiers.values_mut() {
|
||||
let list = match tt {
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
let mut healed: Vec<String> = Vec::new();
|
||||
for id in &list {
|
||||
if let Some(new_id) = fix(id) {
|
||||
if !healed.contains(&new_id) {
|
||||
healed.push(new_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
*tt = TierTargets::List(healed);
|
||||
}
|
||||
ok(json!({ "ok": true, "needs_restart": needs_restart }))
|
||||
}
|
||||
|
||||
// ───────────────────────────── cmf runtime ─────────────────────────────────
|
||||
@@ -1025,7 +1035,6 @@ async fn cmf_status(State(state): State<SharedState>) -> ApiResult {
|
||||
"port": s.port,
|
||||
"threads": s.threads,
|
||||
"gpu": s.gpu,
|
||||
"o1": s.o1,
|
||||
"running": live.map(|l| l.running).unwrap_or(false),
|
||||
"healthy": live.map(|l| l.healthy).unwrap_or(false),
|
||||
"last_error": live.and_then(|l| l.last_error.clone()),
|
||||
@@ -1123,9 +1132,6 @@ async fn cmf_port_check(State(state): State<SharedState>, Query(q): Query<PortQu
|
||||
ok(json!({
|
||||
"port": port,
|
||||
"available": available,
|
||||
// true = held by this gateway's own managed server (a replaced/edited
|
||||
// row frees it on restart) — the UI shows this as info, not an error
|
||||
"managed": running_here,
|
||||
"detail": detail,
|
||||
"suggested": suggested,
|
||||
}))
|
||||
@@ -1347,6 +1353,507 @@ async fn hf_search(State(state): State<SharedState>, Query(q): Query<HfQuery>) -
|
||||
ok(json!({ "models": models }))
|
||||
}
|
||||
|
||||
// ── first-run onboarding ──
|
||||
|
||||
fn setup_marker(state: &SharedState) -> std::path::PathBuf {
|
||||
std::path::Path::new(&state.config_path)
|
||||
.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.join(".setup_done")
|
||||
}
|
||||
|
||||
async fn setup_status(State(state): State<SharedState>) -> ApiResult {
|
||||
let live = state.live();
|
||||
let marker = setup_marker(&state);
|
||||
let done = marker.exists();
|
||||
let has_models = !live.cfg.models.is_empty() || !live.cfg.cmf.effective_servers().is_empty();
|
||||
let has_imports = !state.imports.list().is_empty();
|
||||
ok(json!({
|
||||
"setup_done": done,
|
||||
"has_models": has_models,
|
||||
"has_imports": has_imports,
|
||||
"needs_setup": !done && !has_models,
|
||||
"listen": live.cfg.listen,
|
||||
"version": env!("CARGO_PKG_VERSION"),
|
||||
}))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SetupCompleteBody {
|
||||
#[serde(default)]
|
||||
lang: Option<String>,
|
||||
/// Connect finished downloads as local models (default: yes).
|
||||
#[serde(default)]
|
||||
register: Option<bool>,
|
||||
}
|
||||
|
||||
/// Finish the wizard. Everything that finished downloading is connected as a
|
||||
/// local model here — the wizard promises exactly that, and leaving it to a
|
||||
/// separate button is how a first run ends with an empty model pool.
|
||||
async fn setup_complete(
|
||||
State(state): State<SharedState>,
|
||||
Json(b): Json<SetupCompleteBody>,
|
||||
) -> ApiResult {
|
||||
let marker = setup_marker(&state);
|
||||
if let Some(dir) = marker.parent() {
|
||||
let _ = std::fs::create_dir_all(dir);
|
||||
}
|
||||
let lang = b.lang.unwrap_or_else(|| "en".into());
|
||||
let _ = std::fs::write(
|
||||
&marker,
|
||||
format!(
|
||||
"done lang={lang} ts={}",
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
),
|
||||
);
|
||||
|
||||
let mut registered: Vec<String> = Vec::new();
|
||||
if b.register.unwrap_or(true) {
|
||||
let finished: Vec<String> = state
|
||||
.imports
|
||||
.list()
|
||||
.into_iter()
|
||||
.filter(|j| j.state == "done")
|
||||
.map(|j| j.output)
|
||||
.collect();
|
||||
if !finished.is_empty() {
|
||||
let mut cfg = current_cfg(&state);
|
||||
for output in &finished {
|
||||
if let Some(id) = register_cmf_output(&mut cfg, output) {
|
||||
registered.push(id);
|
||||
}
|
||||
}
|
||||
if cfg.routing.default.is_empty() {
|
||||
if let Some(first) = registered.first() {
|
||||
cfg.routing.default = first.clone();
|
||||
}
|
||||
}
|
||||
if !registered.is_empty() {
|
||||
state.reload(cfg)?;
|
||||
}
|
||||
}
|
||||
}
|
||||
let runtime = crate::cmf_runtime::installed_version(&state.live().cfg.cmf.cortiq_bin);
|
||||
ok(json!({
|
||||
"ok": true,
|
||||
"registered": registered,
|
||||
"runtime_installed": runtime.is_some(),
|
||||
"runtime_version": runtime,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Curated ready-to-run .cmf models from infosave (HF author) — proxied server-side.
|
||||
async fn setup_featured(State(state): State<SharedState>) -> ApiResult {
|
||||
let live = state.live();
|
||||
let token = if live.cfg.cmf.hf_token_env.is_empty() {
|
||||
None
|
||||
} else {
|
||||
state.secrets.resolve(&live.cfg.cmf.hf_token_env)
|
||||
};
|
||||
// Try HF author search; on failure return curated fallback
|
||||
let hf_result = crate::import::hf_search_author("infosave", 30, token.as_deref()).await;
|
||||
let models = match hf_result {
|
||||
Ok(v) => v,
|
||||
Err(_) => serde_json::json!([]),
|
||||
};
|
||||
// curated fallback with real sizes (usedStorage from HF)
|
||||
let fallback = json!([
|
||||
{"id": "infosave/Bonsai-1.7Bcmf", "label": "Bonsai 1.7B", "badge": "1.7B · Q1", "desc": "Small, fast local model — fits 4GB RAM", "size": "334 MB", "size_bytes": 334185278},
|
||||
{"id": "infosave/Bonsai-8B_2bit_cmf", "label": "Bonsai 8B 2-bit", "badge": "8B · Q1T", "desc": "Balanced quality/size — 2-bit ternary", "size": "2.3 GB", "size_bytes": 2322861074_u64},
|
||||
{"id": "infosave/Bonsai-27Bcmf", "label": "Bonsai 27B", "badge": "27B · Q1", "desc": "High quality — needs 16GB RAM", "size": "5.1 GB", "size_bytes": 5103462190_u64},
|
||||
{"id": "infosave/Qwen3.6-27Bcmf", "label": "Qwen 3.6 27B CMF", "badge": "27B · Q4 · MoE", "desc": "MoE model, strong reasoning", "size": "14.3 GB", "size_bytes": 14264207150_u64},
|
||||
{"id": "infosave/Nanbeige4.2-3Bcmf", "label": "Nanbeige 4.2 3B", "badge": "3B · looped", "desc": "Looped transformer, compact", "size": "2.4 GB", "size_bytes": 2364287523_u64},
|
||||
{"id": "infosave/KAT-Coder-V2.5-CMF", "label": "KAT Coder 2.5", "badge": "code · MoE", "desc": "Code specialist", "size": "12.7 GB", "size_bytes": 12653975221_u64},
|
||||
]);
|
||||
let arr = if models.as_array().map(|a| a.is_empty()).unwrap_or(true) {
|
||||
fallback
|
||||
} else {
|
||||
// enrich HF data with label/badge/size for UI — fetch usedStorage concurrently
|
||||
let mut ids: Vec<String> = Vec::new();
|
||||
if let Some(arr) = models.as_array() {
|
||||
for m in arr {
|
||||
if let Some(id) = m["id"].as_str() {
|
||||
if id.to_ascii_lowercase().contains("cmf") {
|
||||
ids.push(id.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if ids.is_empty() {
|
||||
return ok(json!({ "models": fallback }));
|
||||
}
|
||||
// fetch per-variant sizes in parallel (tree API, 10s each)
|
||||
let token_owned = token.map(|s| s.to_string());
|
||||
let variant_lists: Vec<(String, Vec<(String, u64)>)> =
|
||||
futures::future::join_all(ids.iter().map(|id| {
|
||||
let tid = token_owned.clone();
|
||||
let iid = id.clone();
|
||||
async move {
|
||||
let vars = crate::import::hf_repo_variants(&iid, tid.as_deref()).await;
|
||||
(iid, vars)
|
||||
}
|
||||
}))
|
||||
.await;
|
||||
let mut variant_map: std::collections::HashMap<String, Vec<(String, u64)>> =
|
||||
std::collections::HashMap::new();
|
||||
let mut fallback_map: std::collections::HashMap<String, (String, u64)> =
|
||||
std::collections::HashMap::new();
|
||||
let mut fallback_badge: std::collections::HashMap<String, String> =
|
||||
std::collections::HashMap::new();
|
||||
// also keep fallback sizes/badges for when tree is unreachable
|
||||
for v in fallback.as_array().unwrap_or(&vec![]) {
|
||||
if let (Some(id), Some(sz)) = (v["id"].as_str(), v["size_bytes"].as_u64()) {
|
||||
if let Some(s) = v["size"].as_str() {
|
||||
fallback_map.insert(id.to_string(), (s.to_string(), sz));
|
||||
}
|
||||
}
|
||||
if let (Some(id), Some(b)) = (v["id"].as_str(), v["badge"].as_str()) {
|
||||
fallback_badge.insert(id.to_string(), b.to_string());
|
||||
}
|
||||
}
|
||||
for (id, vars) in variant_lists {
|
||||
if !vars.is_empty() {
|
||||
variant_map.insert(id, vars);
|
||||
}
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
if let Some(arr) = models.as_array() {
|
||||
for m in arr {
|
||||
let id = m["id"].as_str().unwrap_or("").to_string();
|
||||
if !id.to_ascii_lowercase().contains("cmf") {
|
||||
continue;
|
||||
}
|
||||
let likes = m["likes"].as_u64().unwrap_or(0);
|
||||
let vars = variant_map.get(&id);
|
||||
// extract size token from id
|
||||
let lower = id.to_ascii_lowercase();
|
||||
let size_token = ["1.7b", "48b", "35b", "27b", "12b", "8b", "7b", "3b", "4b"]
|
||||
.iter()
|
||||
.find(|c| lower.contains(*c))
|
||||
.map(|c| match *c {
|
||||
"1.7b" => "1.7B",
|
||||
"48b" => "48B",
|
||||
"35b" => "35B",
|
||||
"27b" => "27B",
|
||||
"12b" => "12B",
|
||||
"8b" => "8B",
|
||||
"7b" => "7B",
|
||||
"3b" => "3B",
|
||||
"4b" => "4B",
|
||||
_ => "CMF",
|
||||
})
|
||||
.unwrap_or("CMF");
|
||||
// quant from variant name if available (more accurate for Bonsai q1), else id
|
||||
let quant_src = vars
|
||||
.and_then(|vs| vs.first().map(|(n, _)| n.as_str()))
|
||||
.unwrap_or(&id)
|
||||
.to_ascii_lowercase();
|
||||
let quant = if quant_src.contains("q1t")
|
||||
|| quant_src.contains("2bit")
|
||||
|| lower.contains("q1t")
|
||||
|| lower.contains("2bit")
|
||||
{
|
||||
"Q1T"
|
||||
} else if quant_src.contains("q2tp") || lower.contains("q2tp") {
|
||||
"Q2TP"
|
||||
} else if quant_src.contains("q4tp") || lower.contains("q4tp") {
|
||||
"Q4TP"
|
||||
} else if quant_src.contains("q1") || lower.contains("q1") {
|
||||
"Q1"
|
||||
} else if quant_src.contains("q4") || lower.contains("q4") {
|
||||
"Q4"
|
||||
} else if quant_src.contains("q8") || lower.contains("q8") {
|
||||
"Q8"
|
||||
} else {
|
||||
""
|
||||
};
|
||||
let mut badge = if quant.is_empty() || size_token == "CMF" {
|
||||
if size_token == "CMF" {
|
||||
"CMF · ready".to_string()
|
||||
} else {
|
||||
size_token.to_string()
|
||||
}
|
||||
} else {
|
||||
format!("{size_token} · {quant}")
|
||||
};
|
||||
// fallback has more accurate quant (e.g. Bonsai 1.7B Q1) when tree fetch failed
|
||||
if let Some(fb) = fallback_badge.get(&id) {
|
||||
if badge == size_token && fb.contains('·') {
|
||||
badge = fb.clone();
|
||||
}
|
||||
}
|
||||
let (size, size_bytes, variants_json) = if let Some(vs) = vars {
|
||||
if vs.len() == 1 {
|
||||
let (name, sz) = &vs[0];
|
||||
let s = crate::import::format_bytes(*sz);
|
||||
(
|
||||
Some(s.clone()),
|
||||
Some(*sz),
|
||||
json!([{"name": name, "size": s, "size_bytes": sz}]),
|
||||
)
|
||||
} else {
|
||||
// dedupe same-size variants (e.g. q2tp v2/v3 both 112GB → keep latest v3)
|
||||
let mut seen_sz = std::collections::HashSet::new();
|
||||
let mut uniq_vs: Vec<(String, u64)> = Vec::new();
|
||||
// iterate reversed to keep latest (v3) over v2 when same size
|
||||
for (n, sz) in vs.iter().rev() {
|
||||
let s = crate::import::format_bytes(*sz);
|
||||
if seen_sz.insert(s) {
|
||||
uniq_vs.push((n.clone(), *sz));
|
||||
}
|
||||
}
|
||||
uniq_vs.reverse();
|
||||
let parts: Vec<String> = uniq_vs
|
||||
.iter()
|
||||
.map(|(_, sz)| crate::import::format_bytes(*sz))
|
||||
.collect();
|
||||
let max_sz = uniq_vs.iter().map(|(_, sz)| *sz).max();
|
||||
let vars_json: Vec<Value> = uniq_vs.iter().map(|(n, sz)| json!({"name": n, "size": crate::import::format_bytes(*sz), "size_bytes": sz})).collect();
|
||||
(Some(parts.join(" / ")), max_sz, json!(vars_json))
|
||||
}
|
||||
} else if let Some((s, sz)) = fallback_map.get(&id) {
|
||||
(
|
||||
Some(s.clone()),
|
||||
Some(*sz),
|
||||
json!([{"name": id.split('/').next_back().unwrap_or(&id), "size": s, "size_bytes": sz}]),
|
||||
)
|
||||
} else {
|
||||
(None::<String>, None::<u64>, Value::Null)
|
||||
};
|
||||
// check if already downloaded locally (case-insensitive, handles sharded dirs)
|
||||
let models_dir = live.cfg.cmf.models_dir.clone();
|
||||
let base = crate::import::sanitize_for_id(id.split('/').next_back().unwrap_or(&id));
|
||||
let mut downloaded = false;
|
||||
let mut downloaded_variants: Vec<Value> = Vec::new();
|
||||
if let Some(vs) = variants_json.as_array() {
|
||||
for v in vs {
|
||||
let vname = v["name"].as_str().unwrap_or("");
|
||||
let vbase = if vs.len() == 1 {
|
||||
base.clone()
|
||||
} else {
|
||||
format!("{}-{}", base, crate::import::sanitize_for_id(vname))
|
||||
};
|
||||
let expected = format!("{vbase}.cmf").to_lowercase();
|
||||
// case-insensitive search via read_dir (handles /tmp symlink and case)
|
||||
let mut exists = false;
|
||||
let mut local_sz: u64 = 0;
|
||||
if let Ok(rd) = std::fs::read_dir(&models_dir) {
|
||||
for entry in rd.filter_map(|e| e.ok()) {
|
||||
let name = entry.file_name().to_string_lossy().to_lowercase();
|
||||
if name == expected {
|
||||
exists = true;
|
||||
local_sz = entry.metadata().map(|m| m.len()).unwrap_or(0);
|
||||
if local_sz == 0 && entry.path().is_dir() {
|
||||
local_sz = std::fs::read_dir(entry.path())
|
||||
.ok()
|
||||
.map(|r| {
|
||||
r.filter_map(|e| e.ok())
|
||||
.filter_map(|e| e.metadata().ok())
|
||||
.map(|m| m.len())
|
||||
.sum()
|
||||
})
|
||||
.unwrap_or(0);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// fallback direct path check (for non-listed or still being written)
|
||||
if !exists {
|
||||
let p = std::path::Path::new(&models_dir).join(format!("{vbase}.cmf"));
|
||||
if p.exists() || p.is_dir() {
|
||||
exists = true;
|
||||
local_sz =
|
||||
std::fs::metadata(&p).map(|m| m.len()).unwrap_or_else(|_| {
|
||||
std::fs::read_dir(&p)
|
||||
.ok()
|
||||
.map(|r| {
|
||||
r.filter_map(|e| e.ok())
|
||||
.filter_map(|e| e.metadata().ok())
|
||||
.map(|m| m.len())
|
||||
.sum()
|
||||
})
|
||||
.unwrap_or(0)
|
||||
});
|
||||
}
|
||||
}
|
||||
let remote_sz = v["size_bytes"].as_u64().unwrap_or(0);
|
||||
let is_complete =
|
||||
exists && local_sz != 0 && remote_sz != 0 && local_sz == remote_sz;
|
||||
// downloaded = only when complete (not partial)
|
||||
downloaded = downloaded || is_complete;
|
||||
let mut vv = v.clone();
|
||||
vv["downloaded"] = json!(is_complete);
|
||||
vv["local_size"] = json!(local_sz);
|
||||
// the exact output stem the import will produce, and the id of
|
||||
// the job that produces it — the UI passes both back verbatim so
|
||||
// download / progress / delete can never disagree on the file.
|
||||
vv["local_name"] = json!(vbase);
|
||||
vv["job"] = json!(crate::import::gen_id(&format!("{id}::{vname}")));
|
||||
if exists && !is_complete && local_sz != 0 {
|
||||
// partial or outdated — show as update available if not currently downloading
|
||||
// we don't have job state here, so just mark update if sizes differ
|
||||
vv["update_available"] = json!(true);
|
||||
} else if is_complete && remote_sz != 0 && local_sz != remote_sz {
|
||||
vv["update_available"] = json!(true);
|
||||
}
|
||||
downloaded_variants.push(vv);
|
||||
}
|
||||
}
|
||||
let final_variants = if downloaded_variants.is_empty() {
|
||||
variants_json
|
||||
} else {
|
||||
json!(downloaded_variants)
|
||||
};
|
||||
let any_update = downloaded_variants
|
||||
.iter()
|
||||
.any(|v| v["update_available"].as_bool().unwrap_or(false));
|
||||
out.push(json!({
|
||||
"id": id,
|
||||
"label": id.split('/').next_back().unwrap_or(&id),
|
||||
"local_name": base,
|
||||
"badge": badge,
|
||||
"desc": m["pipeline_tag"].as_str().unwrap_or("text-generation"),
|
||||
"likes": likes,
|
||||
"size": size,
|
||||
"size_bytes": size_bytes,
|
||||
"variants": final_variants,
|
||||
"downloaded": downloaded,
|
||||
"update_available": any_update,
|
||||
"raw": m,
|
||||
}));
|
||||
}
|
||||
}
|
||||
if out.is_empty() {
|
||||
fallback
|
||||
} else {
|
||||
json!(out)
|
||||
}
|
||||
};
|
||||
ok(json!({ "models": arr }))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SetupFileQuery {
|
||||
repo: String,
|
||||
#[serde(default)]
|
||||
variant: Option<String>,
|
||||
/// Exact output stem (`local_name` from `setup/featured`). Preferred over the
|
||||
/// repo/variant guess, which cannot know how many variants a repo has.
|
||||
#[serde(default)]
|
||||
name: Option<String>,
|
||||
}
|
||||
|
||||
/// Drop a downloaded `.cmf` from the config: any managed server or pool model
|
||||
/// backed by that stem. Returns the ids that were unregistered.
|
||||
fn unregister_cmf_stem(state: &SharedState, stem: &str) -> Result<Vec<String>, ApiError> {
|
||||
let mut cfg = current_cfg(state);
|
||||
let stem_of = |path: &str| {
|
||||
std::path::Path::new(path)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_lowercase())
|
||||
};
|
||||
let mut ids: Vec<String> = cfg
|
||||
.cmf
|
||||
.servers
|
||||
.iter()
|
||||
.filter(|s| stem_of(&s.model).as_deref() == Some(stem))
|
||||
.map(|s| s.id.clone())
|
||||
.collect();
|
||||
for m in &cfg.models {
|
||||
if cmf_file_for_model(&cfg.cmf, m)
|
||||
.and_then(|p| stem_of(&p.to_string_lossy()))
|
||||
.as_deref()
|
||||
== Some(stem)
|
||||
{
|
||||
ids.push(m.id.clone());
|
||||
}
|
||||
}
|
||||
if ids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
cfg.cmf
|
||||
.servers
|
||||
.retain(|s| stem_of(&s.model).as_deref() != Some(stem));
|
||||
if stem_of(&cfg.cmf.local_model).as_deref() == Some(stem) {
|
||||
cfg.cmf.local_model = String::new();
|
||||
cfg.cmf.manage_server = false;
|
||||
}
|
||||
cfg.models.retain(|m| !ids.contains(&m.id));
|
||||
for id in &ids {
|
||||
strip_id_from_routing(&mut cfg.routing, id);
|
||||
}
|
||||
state.reload(cfg)?;
|
||||
Ok(ids)
|
||||
}
|
||||
|
||||
async fn delete_setup_file(
|
||||
State(state): State<SharedState>,
|
||||
Query(q): Query<SetupFileQuery>,
|
||||
) -> ApiResult {
|
||||
let repo = q.repo.trim();
|
||||
if repo.is_empty() {
|
||||
return Err(ApiError::bad("repo is required"));
|
||||
}
|
||||
let live = state.live();
|
||||
let models_dir = live.cfg.cmf.models_dir.clone();
|
||||
let base = crate::import::sanitize_for_id(repo.split('/').next_back().unwrap_or(repo));
|
||||
let variant = q
|
||||
.variant
|
||||
.as_deref()
|
||||
.map(|s| s.trim())
|
||||
.filter(|s| !s.is_empty());
|
||||
// Exact stem wins; sanitizing it also keeps the delete inside models_dir.
|
||||
let vbase = match q.name.as_deref().map(str::trim).filter(|s| !s.is_empty()) {
|
||||
Some(n) => crate::import::sanitize_for_id(n),
|
||||
// for single-file the file is {base}.cmf, for sharded {base}-{variant}.cmf
|
||||
None => match variant {
|
||||
// only sharded variants need suffix; single file ignores
|
||||
Some(v) if v.starts_with("parts-") => {
|
||||
format!("{}-{}", base, crate::import::sanitize_for_id(v))
|
||||
}
|
||||
_ => base.clone(),
|
||||
},
|
||||
};
|
||||
if vbase.is_empty() {
|
||||
return Err(ApiError::bad("cannot derive a file name to delete"));
|
||||
}
|
||||
// Stop serving it before the file goes away, so a managed server doesn't keep
|
||||
// a deleted .cmf open (and the pool doesn't advertise a model that is gone).
|
||||
let unregistered = unregister_cmf_stem(&state, &vbase)?;
|
||||
for id in &unregistered {
|
||||
state.cmf.stop_one(id).await;
|
||||
}
|
||||
let p = std::path::Path::new(&models_dir).join(format!("{vbase}.cmf"));
|
||||
let mut removed = false;
|
||||
if p.exists() || p.is_dir() {
|
||||
removed = remove_cmf_path(&p);
|
||||
} else {
|
||||
// try case-insensitive search (macOS /tmp symlink, etc.)
|
||||
if let Ok(rd) = std::fs::read_dir(&models_dir) {
|
||||
let expected = format!("{vbase}.cmf").to_lowercase();
|
||||
for entry in rd.filter_map(|e| e.ok()) {
|
||||
if entry.file_name().to_string_lossy().to_lowercase() == expected {
|
||||
removed = remove_cmf_path(&entry.path());
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// also remove any job with same repo+variant id (so list doesn't keep stale "done")
|
||||
let jid = crate::import::gen_id(&format!("{}::{}", repo, variant.unwrap_or("")));
|
||||
let _ = state.imports.delete(&jid);
|
||||
// also try to delete by repo alone (old non-variant ids)
|
||||
if variant.is_none() {
|
||||
let jid2 = crate::import::gen_id(repo);
|
||||
let _ = state.imports.delete(&jid2);
|
||||
}
|
||||
ok(json!({ "ok": true, "removed": removed, "unregistered": unregistered }))
|
||||
}
|
||||
|
||||
async fn list_imports(State(state): State<SharedState>) -> ApiResult {
|
||||
ok(json!({ "jobs": state.imports.list() }))
|
||||
}
|
||||
@@ -1380,11 +1887,80 @@ async fn cancel_import(State(state): State<SharedState>, Path(job): Path<String>
|
||||
|
||||
/// Delete a finished conversion and its converted `.cmf` file(s) from disk.
|
||||
async fn delete_import(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
|
||||
if state.imports.get(&job).is_none() {
|
||||
let Some(j) = state.imports.get(&job) else {
|
||||
return Err(ApiError::not_found(format!("job '{job}' not found")));
|
||||
};
|
||||
// If it was already registered, drop it from the config first — otherwise the
|
||||
// pool keeps serving a model whose .cmf is about to disappear.
|
||||
let stem = std::path::Path::new(&j.output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_lowercase())
|
||||
.unwrap_or_default();
|
||||
let unregistered = if stem.is_empty() {
|
||||
Vec::new()
|
||||
} else {
|
||||
unregister_cmf_stem(&state, &stem)?
|
||||
};
|
||||
for id in &unregistered {
|
||||
state.cmf.stop_one(id).await;
|
||||
}
|
||||
let removed = state.imports.delete(&job).map_err(ApiError::bad)?;
|
||||
ok(json!({ "ok": removed }))
|
||||
ok(json!({ "ok": removed, "unregistered": unregistered }))
|
||||
}
|
||||
|
||||
/// Add a finished `.cmf` to the config as a *managed* local model: the gateway
|
||||
/// spawns and supervises its own `cortiq serve` for it. Registering it into the
|
||||
/// static pool instead would point the model at `cortiq_server_url` with nothing
|
||||
/// listening there, which is an upstream error the moment anyone calls it.
|
||||
///
|
||||
/// A stale static-pool entry for the same file is migrated rather than rejected,
|
||||
/// so pressing Register again repairs a model left dead by an earlier version.
|
||||
/// `None` = already registered and served.
|
||||
fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<String> {
|
||||
let base = std::path::Path::new(output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "cmf-local".into());
|
||||
let id = format!("cmf-{base}");
|
||||
let served = cfg.cmf.manage_server
|
||||
&& cfg
|
||||
.cmf
|
||||
.servers
|
||||
.iter()
|
||||
.any(|s| s.id == id || s.model == output);
|
||||
if served {
|
||||
return None;
|
||||
}
|
||||
// Same id sitting in the static pool → it is the dead form of this model.
|
||||
// Routing keeps referencing the id, so nothing needs re-pointing.
|
||||
cfg.models.retain(|m| m.id != id);
|
||||
cfg.cmf.servers.retain(|s| s.id != id && s.model != output);
|
||||
cfg.cmf.manage_server = true;
|
||||
let host = if cfg.cmf.local_host.trim().is_empty() {
|
||||
"127.0.0.1"
|
||||
} else {
|
||||
&cfg.cmf.local_host
|
||||
};
|
||||
let base_port = cfg.cmf.servers.iter().map(|s| s.port).max().unwrap_or(8089);
|
||||
let port = next_free_port(host, base_port).unwrap_or_else(|| base_port + 1);
|
||||
cfg.cmf.servers.push(crate::config::CmfServer {
|
||||
id: id.clone(),
|
||||
model: output.to_string(),
|
||||
port,
|
||||
threads: cfg.cmf.threads,
|
||||
gpu: cfg.cmf.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
});
|
||||
Some(id)
|
||||
}
|
||||
|
||||
/// Register a finished `.cmf` as a local model (OpenAI provider → cortiq-server).
|
||||
@@ -1396,27 +1972,19 @@ async fn register_import(State(state): State<SharedState>, Path(job): Path<Strin
|
||||
if j.state != "done" {
|
||||
return Err(ApiError::bad("conversion not finished"));
|
||||
}
|
||||
let base = std::path::Path::new(&j.output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "cmf-local".into());
|
||||
let id = format!("cmf-{base}");
|
||||
let mut cfg = current_cfg(&state);
|
||||
if cfg.models.iter().any(|m| m.id == id) {
|
||||
return Err(ApiError::bad(format!("model '{id}' already registered")));
|
||||
let id = register_cmf_output(&mut cfg, &j.output).ok_or_else(|| {
|
||||
ApiError::bad(format!(
|
||||
"model 'cmf-{}' is already registered and running",
|
||||
std::path::Path::new(&j.output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_default()
|
||||
))
|
||||
})?;
|
||||
if cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = id.clone();
|
||||
}
|
||||
cfg.models.push(crate::config::ModelCfg {
|
||||
id: id.clone(),
|
||||
provider: "openai".into(),
|
||||
base_url: cfg.cmf.cortiq_server_url.clone(),
|
||||
model: base,
|
||||
cost_tier: "local".into(),
|
||||
price_in: 0.0,
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
});
|
||||
state.reload(cfg)?;
|
||||
ok(json!({ "ok": true, "model_id": id }))
|
||||
}
|
||||
|
||||
+21
-87
@@ -173,31 +173,6 @@ impl CmfRuntime {
|
||||
}
|
||||
}
|
||||
|
||||
/// Ensure a status slot exists for `server` (used when servers are added at
|
||||
/// runtime); refreshes model/port if the slot already exists.
|
||||
fn add_server_slot(&self, server: &CmfServer, host: &str) {
|
||||
let mut s = self.status.lock().unwrap();
|
||||
s.enabled = true;
|
||||
if let Some(sv) = s.servers.iter_mut().find(|x| x.id == server.id) {
|
||||
sv.model = server.model.clone();
|
||||
sv.port = server.port;
|
||||
sv.base_url = format!("http://{host}:{}/v1", server.port);
|
||||
sv.running = false;
|
||||
sv.healthy = false;
|
||||
sv.last_error = None;
|
||||
} else {
|
||||
s.servers.push(CmfServerStatus {
|
||||
id: server.id.clone(),
|
||||
model: server.model.clone(),
|
||||
port: server.port,
|
||||
base_url: format!("http://{host}:{}/v1", server.port),
|
||||
running: false,
|
||||
healthy: false,
|
||||
last_error: None,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Stop and forget a single managed server (used when a model is deleted).
|
||||
pub async fn stop_one(&self, id: &str) {
|
||||
let removed = {
|
||||
@@ -315,15 +290,30 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
if server.threads > 1 {
|
||||
cmd.env("CMF_THREADS", server.threads.to_string());
|
||||
}
|
||||
// Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
|
||||
// on unified memory, so mostly a prefill win; opt-in.
|
||||
if server.gpu {
|
||||
cmd.env("CMF_GPU", "1");
|
||||
}
|
||||
// Serve-time O(1) attention override: "off" forces exact attention (undoes a
|
||||
// too-aggressive --o1 conversion hint without re-converting the file).
|
||||
if !server.o1.trim().is_empty() {
|
||||
cmd.env("CMF_O1", server.o1.trim());
|
||||
// O(1) Nyström attention: layers spec + sub-parameters.
|
||||
if let Some(o1) = &server.o1 {
|
||||
let s = o1.trim();
|
||||
if !s.is_empty() {
|
||||
// Pass even "off" so it explicitly overrides a file converter hint.
|
||||
cmd.arg("--o1").arg(s);
|
||||
}
|
||||
}
|
||||
if let Some(m) = server.o1_m {
|
||||
cmd.arg("--o1-m").arg(m.to_string());
|
||||
}
|
||||
if let Some(w) = server.o1_window {
|
||||
cmd.arg("--o1-window").arg(w.to_string());
|
||||
}
|
||||
if let Some(sink) = server.o1_sink {
|
||||
cmd.arg("--o1-sink").arg(sink.to_string());
|
||||
}
|
||||
// MTP (Multi-Token Prediction) speculative decoding: the engine reads
|
||||
// CMF_MTP env; "0" disables. There is no CLI flag for this.
|
||||
if server.skip_mtp {
|
||||
cmd.env("CMF_MTP", "0");
|
||||
}
|
||||
cmd.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
@@ -352,62 +342,6 @@ async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
/// Apply a changed managed-servers list LIVE (no gateway restart): stop servers
|
||||
/// that were removed or redefined — freeing their memory immediately — and
|
||||
/// spawn servers that are new or redefined.
|
||||
pub async fn apply_server_changes(rt: Arc<CmfRuntime>, old: Vec<CmfServer>, cfg: CmfCfg) {
|
||||
let new = cfg.effective_servers();
|
||||
// 1. Stop what's gone or changed (kills the child → RAM released).
|
||||
for o in &old {
|
||||
let unchanged = new.iter().any(|n| n == o);
|
||||
if !unchanged {
|
||||
rt.stop_one(&o.id).await;
|
||||
rt.log(format!("stopped '{}' (removed or changed)", o.id));
|
||||
}
|
||||
}
|
||||
if new.is_empty() {
|
||||
return;
|
||||
}
|
||||
// 2. Spawn what's new or changed.
|
||||
if installed_version(&cfg.cortiq_bin).is_none() {
|
||||
rt.fail("cortiq binary not found — cannot start local model servers");
|
||||
return;
|
||||
}
|
||||
for n in &new {
|
||||
let unchanged = old.iter().any(|o| o == n);
|
||||
if unchanged {
|
||||
continue; // already running with the same definition
|
||||
}
|
||||
rt.add_server_slot(n, &cfg.local_host);
|
||||
if !std::path::Path::new(&n.model).exists() {
|
||||
rt.fail_server(&n.id, format!("model file not found: {}", n.model));
|
||||
continue;
|
||||
}
|
||||
rt.log(format!(
|
||||
"starting: cortiq serve {} --host {} --port {}",
|
||||
n.model, cfg.local_host, n.port
|
||||
));
|
||||
match spawn_serve(n, &cfg) {
|
||||
Ok(child) => rt.add_child(n.id.clone(), child),
|
||||
Err(e) => {
|
||||
rt.fail_server(&n.id, format!("failed to spawn: {e}"));
|
||||
continue;
|
||||
}
|
||||
}
|
||||
let (rtc, host, id, port) = (rt.clone(), cfg.local_host.clone(), n.id.clone(), n.port);
|
||||
tokio::spawn(async move {
|
||||
if wait_healthy(&host, port, 240).await {
|
||||
rtc.set_healthy(&id, true);
|
||||
rtc.log(format!(
|
||||
"local CMF server ready — '{id}' at http://{host}:{port}/v1"
|
||||
));
|
||||
} else {
|
||||
rtc.fail_server(&id, "did not become healthy in time");
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn each
|
||||
/// configured local model → wait healthy. Safe to call in a background task; it
|
||||
/// never panics.
|
||||
|
||||
+160
-19
@@ -9,6 +9,9 @@ use std::path::PathBuf;
|
||||
/// `$CORTIQ_GATEWAY_HOME`, then `$XDG_DATA_HOME/cortiq-gateway`, then
|
||||
/// `~/.cortiq-gateway` — so defaults work regardless of the process CWD
|
||||
/// (important once the gateway is installed rather than run from its repo).
|
||||
/// On Windows also checks `%APPDATA%`, `%LOCALAPPDATA%` and `%USERPROFILE%`
|
||||
/// so double-clicking the exe from `Downloads` does not try to write into
|
||||
/// `Program Files`.
|
||||
pub fn data_dir() -> PathBuf {
|
||||
for (var, sub) in [
|
||||
("CORTIQ_GATEWAY_HOME", ""),
|
||||
@@ -25,6 +28,20 @@ pub fn data_dir() -> PathBuf {
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(windows)]
|
||||
{
|
||||
for (var, sub) in [
|
||||
("APPDATA", "cortiq-gateway"),
|
||||
("LOCALAPPDATA", "cortiq-gateway"),
|
||||
("USERPROFILE", ".cortiq-gateway"),
|
||||
] {
|
||||
if let Ok(v) = std::env::var(var) {
|
||||
if !v.is_empty() {
|
||||
return PathBuf::from(v).join(sub);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
PathBuf::from("data")
|
||||
}
|
||||
|
||||
@@ -83,12 +100,36 @@ pub struct CmfServer {
|
||||
/// Offload matvecs to the GPU (CMF_GPU=1).
|
||||
#[serde(default)]
|
||||
pub gpu: bool,
|
||||
/// Serve-time O(1) attention override (exported as CMF_O1). Empty = follow
|
||||
/// the hint recorded in the .cmf; "off" = force exact attention (fixes a
|
||||
/// too-aggressive conversion without re-converting); "all"/"deepN"/"i,j,k"
|
||||
/// = force-enable on those layers.
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) Nyström attention layer spec (all | deepN | i,j,k | off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// O(1) landmark budget (default 32; ≥4).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_m: Option<u32>,
|
||||
/// O(1) exact-window width (default 128).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_window: Option<u32>,
|
||||
/// O(1) permanent exact sink keys (default 4, StreamingLLM discipline).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_sink: Option<u32>,
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub o1: String,
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -178,7 +219,16 @@ impl CmfCfg {
|
||||
port: self.local_port,
|
||||
threads: self.threads,
|
||||
gpu: self.gpu,
|
||||
o1: String::new(),
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
@@ -313,6 +363,27 @@ pub struct ModelCfg {
|
||||
pub api_key_env: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub caps: Vec<String>,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) constant-memory streaming attention hint (all, deep, custom, off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// Skip Multi-Token Prediction (MTP) heads.
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
fn default_kind() -> String {
|
||||
"chat".into()
|
||||
@@ -468,10 +539,12 @@ pub struct AdminCfg {
|
||||
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
|
||||
#[serde(default = "default_admin_enabled")]
|
||||
pub enabled: bool,
|
||||
/// Name of the env variable holding the admin token. If unset (or empty),
|
||||
/// a token is generated at startup and printed to the log.
|
||||
/// Name of the env variable holding the admin token.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token_env: Option<String>,
|
||||
/// Persistent admin token configured directly in toml.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token: Option<String>,
|
||||
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
|
||||
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
@@ -482,6 +555,7 @@ impl Default for AdminCfg {
|
||||
Self {
|
||||
enabled: default_admin_enabled(),
|
||||
token_env: None,
|
||||
token: None,
|
||||
listen: None,
|
||||
}
|
||||
}
|
||||
@@ -645,6 +719,67 @@ fn default_semcache_max() -> usize {
|
||||
1000
|
||||
}
|
||||
|
||||
/// Sensible defaults for a fresh install (empty pool — user fills via wizard/UI).
|
||||
impl Default for Config {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
listen: "0.0.0.0:9000".to_string(),
|
||||
router: RouterCfg {
|
||||
enabled: true,
|
||||
url: "http://localhost:8080".to_string(),
|
||||
timeout_ms: default_router_timeout(),
|
||||
verify_tls: true,
|
||||
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
|
||||
taxonomy_id: Some("general.task-type".to_string()),
|
||||
},
|
||||
route: RouteCfg::default(),
|
||||
models: Vec::new(),
|
||||
routing: RoutingCfg {
|
||||
tiers: HashMap::new(),
|
||||
default: String::new(),
|
||||
policy: RoutingPolicy::default(),
|
||||
},
|
||||
breaker: BreakerCfg::default(),
|
||||
protocols: ProtocolsCfg {
|
||||
openai_chat: true,
|
||||
openai_completions: true,
|
||||
openai_embeddings: true,
|
||||
openai_models: true,
|
||||
anthropic_messages: false,
|
||||
mcp: false,
|
||||
native_passthrough: true,
|
||||
},
|
||||
cortiq: CortiqCfg::default(),
|
||||
api_keys: Vec::new(),
|
||||
idempotency: IdempotencyCfg::default(),
|
||||
log: LogCfg::default(),
|
||||
telemetry: TelemetryCfg::default(),
|
||||
admin: AdminCfg::default(),
|
||||
stats: StatsCfg::default(),
|
||||
cache: CacheCfg::default(),
|
||||
shadow: ShadowCfg::default(),
|
||||
cmf: CmfCfg::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Where the config lives when `--config` is not given.
|
||||
/// Repo checkout (`config/gateway.toml` exists or `config/` dir exists) → keep it there;
|
||||
/// installed binary (no repo) → `$data_dir/gateway.toml`.
|
||||
pub fn default_config_path() -> String {
|
||||
let repo_path = "config/gateway.toml";
|
||||
if std::path::Path::new(repo_path).exists() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
if std::path::Path::new("Cargo.toml").exists() && std::path::Path::new("config").exists() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
data_dir()
|
||||
.join("gateway.toml")
|
||||
.to_string_lossy()
|
||||
.into_owned()
|
||||
}
|
||||
|
||||
impl Config {
|
||||
/// Load and validate the config from a file.
|
||||
pub fn load(path: &str) -> anyhow::Result<Self> {
|
||||
@@ -655,6 +790,22 @@ impl Config {
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Load if exists, otherwise create a fresh default config at `path` and return it.
|
||||
/// Never requires the user to create or copy a file by hand.
|
||||
pub fn load_or_create(path: &str) -> anyhow::Result<Self> {
|
||||
if std::path::Path::new(path).exists() {
|
||||
return Self::load(path);
|
||||
}
|
||||
let cfg = Self::default();
|
||||
// ensure parent dir exists
|
||||
if let Some(dir) = std::path::Path::new(path).parent() {
|
||||
let _ = std::fs::create_dir_all(dir);
|
||||
}
|
||||
cfg.save(path)?;
|
||||
eprintln!("created default config at {path} — open /admin to finish setup");
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Integrity validation: non-empty model pool, valid routing targets.
|
||||
pub fn validate(&self) -> anyhow::Result<()> {
|
||||
// A managed local CMF model counts as an available model even though it
|
||||
@@ -682,19 +833,9 @@ impl Config {
|
||||
self.models.iter().map(|m| m.id.as_str()).collect();
|
||||
// managed local servers are known ids too; also enforce unique ports/ids
|
||||
let mut ports = std::collections::HashSet::new();
|
||||
let mut managed_seen = std::collections::HashSet::new();
|
||||
for s in &servers {
|
||||
if !managed_seen.insert(s.id.as_str()) {
|
||||
anyhow::bail!(
|
||||
"config: two managed local models share the id '{}' — give one of the rows a different id",
|
||||
s.id
|
||||
);
|
||||
}
|
||||
if !ids.insert(s.id.as_str()) {
|
||||
anyhow::bail!(
|
||||
"config: managed local model '{}' has the same id as a pool model — rename the row or delete the pool entry",
|
||||
s.id
|
||||
);
|
||||
anyhow::bail!("config: duplicate managed model id '{}'", s.id);
|
||||
}
|
||||
if !ports.insert(s.port) {
|
||||
anyhow::bail!("config: two managed models share port {}", s.port);
|
||||
|
||||
+752
-17
@@ -30,6 +30,47 @@ fn cleanup_output(path: &str) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Bytes an output occupies: a file's length, or the recursive sum for a sharded
|
||||
/// `.cmf` directory. 0 when nothing has been written yet.
|
||||
pub fn path_size(p: &std::path::Path) -> u64 {
|
||||
match std::fs::metadata(p) {
|
||||
Ok(m) if m.is_file() => m.len(),
|
||||
Ok(m) if m.is_dir() => std::fs::read_dir(p)
|
||||
.map(|rd| {
|
||||
rd.filter_map(|e| e.ok())
|
||||
.map(|e| path_size(&e.path()))
|
||||
.sum()
|
||||
})
|
||||
.unwrap_or(0),
|
||||
_ => 0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Aborts the wrapped task when it goes out of scope — used for the progress
|
||||
/// poller so every early return in a download path stops it.
|
||||
struct AbortOnDrop(tokio::task::JoinHandle<()>);
|
||||
impl Drop for AbortOnDrop {
|
||||
fn drop(&mut self) {
|
||||
self.0.abort();
|
||||
}
|
||||
}
|
||||
|
||||
/// Drive progress from bytes on disk. curl's own meter is unreliable here (its
|
||||
/// bar is redrawn with `\r` into a pipe, and a sharded download runs one curl
|
||||
/// per part), but the output size never lies.
|
||||
fn spawn_size_poller(store: Arc<JobStore>, id: String, path: String) -> AbortOnDrop {
|
||||
AbortOnDrop(tokio::spawn(async move {
|
||||
let p = std::path::PathBuf::from(&path);
|
||||
loop {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(600)).await;
|
||||
let done = path_size(&p);
|
||||
if done > 0 {
|
||||
store.set_bytes(&id, done);
|
||||
}
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
fn now() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
@@ -37,16 +78,12 @@ fn now() -> u64 {
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Job id from a hash of (repo, nanos) — unique without a uuid dependency.
|
||||
fn gen_id(repo: &str) -> String {
|
||||
/// Job id from repo (deterministic) — same repo+variant reuses same id, so
|
||||
/// the job list doesn't fill with duplicates and "already downloaded" can be detected.
|
||||
pub fn gen_id(repo: &str) -> String {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos();
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
repo.hash(&mut h);
|
||||
nanos.hash(&mut h);
|
||||
format!("{:016x}", h.finish())
|
||||
}
|
||||
|
||||
@@ -58,6 +95,10 @@ pub struct ImportParams {
|
||||
pub quant: String,
|
||||
#[serde(default)]
|
||||
pub name: String, // output basename; empty → derived from repo
|
||||
/// For sharded CMF repos (e.g. DeepSeek with `parts-q2tp-v2`/`parts-q4tp`), which
|
||||
/// variant directory to download. Empty = auto (first .cmf or first variant).
|
||||
#[serde(default)]
|
||||
pub variant: Option<String>,
|
||||
// ── advanced ──
|
||||
#[serde(default)]
|
||||
pub linear_core: Option<String>, // gated_delta_net | vmf_phase
|
||||
@@ -97,6 +138,11 @@ pub struct Job {
|
||||
pub repo: String,
|
||||
pub quant: String,
|
||||
pub output: String,
|
||||
/// Output basename without `.cmf` — the id the UI uses to match a job to the
|
||||
/// featured card that started it, and to the file on disk.
|
||||
pub name: String,
|
||||
/// Sharded-variant directory / `.cmf` filename this job downloads, if any.
|
||||
pub variant: Option<String>,
|
||||
pub state: String, // running | done | error | cancelled
|
||||
pub log: Vec<String>,
|
||||
pub started: u64,
|
||||
@@ -104,6 +150,10 @@ pub struct Job {
|
||||
pub size_bytes: Option<u64>,
|
||||
pub progress: Option<f32>, // 0..1, parsed from converter @PROGRESS
|
||||
pub phase: Option<String>, // current phase label
|
||||
/// Expected download size (from the HF tree) and bytes already on disk —
|
||||
/// what makes the percentage in the UI real rather than a guess.
|
||||
pub total_bytes: Option<u64>,
|
||||
pub done_bytes: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
@@ -154,12 +204,60 @@ impl JobStore {
|
||||
}
|
||||
fn set_progress(&self, id: &str, frac: f32, phase: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.progress = Some(frac);
|
||||
let f = frac.clamp(0.0, 1.0);
|
||||
let cur = j.progress.unwrap_or(0.0);
|
||||
// monotonic only — never go backwards (curl reprints, re-download),
|
||||
// but an unset bar takes whatever the first reading is
|
||||
if j.progress.is_none() || f > cur + 0.001 {
|
||||
j.progress = Some(f);
|
||||
}
|
||||
if !phase.is_empty() {
|
||||
j.phase = Some(phase);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// Expected total download size, once the HF tree is known.
|
||||
fn set_total(&self, id: &str, total: u64) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
if total > 0 {
|
||||
j.total_bytes = Some(total);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// Report bytes on disk; derives the percentage whenever the total is known.
|
||||
fn set_bytes(&self, id: &str, done: u64) {
|
||||
let total = {
|
||||
let mut g = self.jobs.lock().unwrap();
|
||||
let Some(j) = g.get_mut(id) else { return };
|
||||
if j.state != "running" {
|
||||
return;
|
||||
}
|
||||
if done > j.done_bytes.unwrap_or(0) {
|
||||
j.done_bytes = Some(done);
|
||||
}
|
||||
j.total_bytes
|
||||
};
|
||||
if let Some(t) = total.filter(|t| *t > 0) {
|
||||
self.set_progress(id, done as f32 / t as f32, "downloading".into());
|
||||
}
|
||||
}
|
||||
fn set_phase(&self, id: &str, phase: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.phase = Some(phase);
|
||||
// keep progress as is; if it was 0.0, make it indeterminate (None) so bar animates
|
||||
if j.progress == Some(0.0) {
|
||||
j.progress = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
fn update_quant(&self, id: &str, quant: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.quant = quant.clone();
|
||||
if !j.log.is_empty() && j.log[0].starts_with("→ downloading ready .cmf model") {
|
||||
j.log[0] = format!("→ downloading ready .cmf model {} ({})", j.repo, quant);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn set_state(&self, id: &str, state: &str) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.state = state.into();
|
||||
@@ -172,11 +270,14 @@ impl JobStore {
|
||||
return; // a concurrent cancel already settled this job
|
||||
}
|
||||
j.finished = Some(now());
|
||||
j.size_bytes = std::fs::metadata(&j.output).ok().map(|m| m.len());
|
||||
let done = ok && j.size_bytes.unwrap_or(0) > 0;
|
||||
let on_disk = path_size(std::path::Path::new(&j.output));
|
||||
j.size_bytes = if on_disk > 0 { Some(on_disk) } else { None };
|
||||
let done = ok && on_disk > 0;
|
||||
j.state = if done { "done" } else { "error" }.into();
|
||||
if done {
|
||||
j.progress = Some(1.0);
|
||||
j.done_bytes = Some(on_disk);
|
||||
j.total_bytes = Some(j.total_bytes.unwrap_or(on_disk).max(on_disk));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -197,7 +298,19 @@ impl JobStore {
|
||||
Ok(true)
|
||||
}
|
||||
pub fn list(&self) -> Vec<Job> {
|
||||
let mut v: Vec<Job> = self.jobs.lock().unwrap().values().cloned().collect();
|
||||
let mut map: HashMap<String, Job> = HashMap::new();
|
||||
for j in self.jobs.lock().unwrap().values().cloned() {
|
||||
// dedupe by repo+variant — keep the latest (so re-download doesn't create
|
||||
// duplicate rows) while two variants of one repo stay separate rows
|
||||
let key = format!("{}::{}", j.repo, j.variant.clone().unwrap_or_default());
|
||||
match map.get(&key) {
|
||||
Some(prev) if prev.started >= j.started => {}
|
||||
_ => {
|
||||
map.insert(key, j);
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut v: Vec<Job> = map.into_values().collect();
|
||||
v.sort_by_key(|x| std::cmp::Reverse(x.started));
|
||||
v.truncate(20);
|
||||
v
|
||||
@@ -234,6 +347,153 @@ pub async fn hf_search(
|
||||
resp.json().await.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
pub async fn hf_search_author(
|
||||
author: &str,
|
||||
limit: usize,
|
||||
token: Option<&str>,
|
||||
) -> Result<serde_json::Value, String> {
|
||||
let url = format!(
|
||||
"https://huggingface.co/api/models?author={}&limit={}&sort=trendingScore&direction=-1",
|
||||
url_escape(author),
|
||||
limit
|
||||
);
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(15))
|
||||
.build()
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = req.send().await.map_err(|e| e.to_string())?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(format!("HuggingFace API {}", resp.status()));
|
||||
}
|
||||
resp.json().await.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
pub async fn hf_model_size(id: &str, token: Option<&str>) -> Option<u64> {
|
||||
let url = format!("https://huggingface.co/api/models/{}", url_escape(id));
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(8))
|
||||
.build()
|
||||
.ok()?;
|
||||
let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = req.send().await.ok()?;
|
||||
if !resp.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
let v: serde_json::Value = resp.json().await.ok()?;
|
||||
v.get("usedStorage").and_then(|x| x.as_u64())
|
||||
}
|
||||
|
||||
/// Per-variant sizes for a repo (for CMF repos with sharded directories like
|
||||
/// `parts-q2tp-v2`, `parts-q4tp`). Returns `[(variant_name, bytes)]`. For a
|
||||
/// single-file repo returns one entry. Empty = tree unreachable.
|
||||
pub async fn hf_repo_variants(id: &str, token: Option<&str>) -> Vec<(String, u64)> {
|
||||
let url = format!(
|
||||
"https://huggingface.co/api/models/{}/tree/main?recursive=true",
|
||||
url_escape(id)
|
||||
);
|
||||
let client = match reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
{
|
||||
Ok(c) => c,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = match req.send().await {
|
||||
Ok(r) if r.status().is_success() => r,
|
||||
_ => return Vec::new(),
|
||||
};
|
||||
let arr: serde_json::Value = match resp.json().await {
|
||||
Ok(v) => v,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
let list = match arr.as_array() {
|
||||
Some(a) => a,
|
||||
None => return Vec::new(),
|
||||
};
|
||||
use std::collections::HashMap;
|
||||
let mut dir_sizes: HashMap<String, u64> = HashMap::new();
|
||||
let mut single_cmf: Vec<(String, u64)> = Vec::new();
|
||||
for item in list {
|
||||
if item.get("type").and_then(|t| t.as_str()) != Some("file") {
|
||||
continue;
|
||||
}
|
||||
let path = match item.get("path").and_then(|p| p.as_str()) {
|
||||
Some(p) => p,
|
||||
None => continue,
|
||||
};
|
||||
let sz = item
|
||||
.get("size")
|
||||
.and_then(|v| v.as_u64())
|
||||
.or_else(|| {
|
||||
item.get("lfs")
|
||||
.and_then(|l| l.get("size"))
|
||||
.and_then(|v| v.as_u64())
|
||||
})
|
||||
.unwrap_or(0);
|
||||
if sz == 0 {
|
||||
continue;
|
||||
}
|
||||
if path.to_ascii_lowercase().ends_with(".cmf") {
|
||||
single_cmf.push((path.to_string(), sz));
|
||||
} else if path.contains('/') && path.starts_with("parts-") {
|
||||
let top = path.split('/').next().unwrap_or(path).to_string();
|
||||
*dir_sizes.entry(top).or_default() += sz;
|
||||
}
|
||||
}
|
||||
if !single_cmf.is_empty() {
|
||||
return single_cmf;
|
||||
}
|
||||
let mut out: Vec<(String, u64)> = dir_sizes.into_iter().collect();
|
||||
out.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
out
|
||||
}
|
||||
|
||||
/// `Content-Length` of a download URL (redirects followed) — the fallback total
|
||||
/// when the repo tree didn't tell us how big the file is.
|
||||
async fn head_content_length(url: &str, token: Option<&str>) -> Option<u64> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.ok()?;
|
||||
let mut req = client.head(url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = req.send().await.ok()?;
|
||||
if !resp.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
resp.headers()
|
||||
.get(reqwest::header::CONTENT_LENGTH)?
|
||||
.to_str()
|
||||
.ok()?
|
||||
.parse()
|
||||
.ok()
|
||||
}
|
||||
|
||||
pub fn format_bytes(n: u64) -> String {
|
||||
if n >= 1_000_000_000 {
|
||||
format!("{:.1} GB", n as f64 / 1_000_000_000.0)
|
||||
} else if n >= 1_000_000 {
|
||||
format!("{} MB", n / 1_000_000)
|
||||
} else if n >= 1_000 {
|
||||
format!("{} KB", n / 1_000)
|
||||
} else {
|
||||
format!("{n} B")
|
||||
}
|
||||
}
|
||||
|
||||
fn url_escape(s: &str) -> String {
|
||||
s.chars()
|
||||
.map(|c| match c {
|
||||
@@ -255,19 +515,389 @@ fn sanitize(s: &str) -> String {
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
pub fn sanitize_for_id(s: &str) -> String {
|
||||
sanitize(&s.to_lowercase())
|
||||
}
|
||||
|
||||
/// Kick off a conversion. Returns the job id immediately; progress streams
|
||||
/// Map the UI quant label to the native `cortiq` quant flag.
|
||||
fn map_quant(q: &str) -> &'static str {
|
||||
match q.to_ascii_uppercase().as_str() {
|
||||
s if s.starts_with("Q4T") || s == "Q4_TILED" => "q4t",
|
||||
s if s.starts_with("Q4") => "q4",
|
||||
"Q8_2F" => "q8_2f",
|
||||
"Q8_2F" | "Q82F" => "q8_2f",
|
||||
"F16" | "FP16" => "f16",
|
||||
"VBIT" => "vbit",
|
||||
"Q1" => "q1",
|
||||
"Q1P" | "Q1_PTQ" => "q1p",
|
||||
"Q1S" | "Q1_MASK" => "q1s",
|
||||
"Q1T" | "Q1_TERNARY" => "q1t",
|
||||
_ => "q8",
|
||||
}
|
||||
}
|
||||
|
||||
fn is_cmf_repo(repo: &str) -> bool {
|
||||
let lower = repo.to_ascii_lowercase();
|
||||
lower.contains("cmf") || lower.ends_with(".cmf")
|
||||
}
|
||||
|
||||
async fn download_cmf_repo(
|
||||
repo: String,
|
||||
output_abs: String,
|
||||
hf_token: Option<String>,
|
||||
store: Arc<JobStore>,
|
||||
id: String,
|
||||
cancel: Arc<Notify>,
|
||||
variant: Option<String>,
|
||||
) {
|
||||
// don't show 0% "listing files" — it looks stuck; show indeterminate "preparing"
|
||||
store.set_phase(&id, "preparing".into());
|
||||
|
||||
let tree_url = format!("https://huggingface.co/api/models/{repo}/tree/main?recursive=true");
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.unwrap_or_else(|_| reqwest::Client::new());
|
||||
let mut req = client.get(&tree_url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = &hf_token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
|
||||
let mut cmf_files: Vec<(String, u64)> = Vec::new();
|
||||
let mut sharded_dirs: std::collections::HashMap<String, Vec<String>> =
|
||||
std::collections::HashMap::new();
|
||||
let mut dir_sizes: std::collections::HashMap<String, u64> = std::collections::HashMap::new();
|
||||
if let Ok(resp) = req.send().await {
|
||||
if resp.status().is_success() {
|
||||
if let Ok(json) = resp.json::<serde_json::Value>().await {
|
||||
if let Some(arr) = json.as_array() {
|
||||
for item in arr {
|
||||
if item.get("type").and_then(|t| t.as_str()) != Some("file") {
|
||||
continue;
|
||||
}
|
||||
if let Some(path) = item.get("path").and_then(|p| p.as_str()) {
|
||||
let sz = item
|
||||
.get("size")
|
||||
.and_then(|v| v.as_u64())
|
||||
.or_else(|| {
|
||||
item.get("lfs")
|
||||
.and_then(|l| l.get("size"))
|
||||
.and_then(|v| v.as_u64())
|
||||
})
|
||||
.unwrap_or(0);
|
||||
if path.to_ascii_lowercase().ends_with(".cmf") {
|
||||
cmf_files.push((path.to_string(), sz));
|
||||
} else if path.contains('/') && path.starts_with("parts-") {
|
||||
let top = path.split('/').next().unwrap_or(path).to_string();
|
||||
sharded_dirs
|
||||
.entry(top.clone())
|
||||
.or_default()
|
||||
.push(path.to_string());
|
||||
*dir_sizes.entry(top).or_default() += sz;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Decide what to download: single .cmf file vs sharded variant directory
|
||||
let is_sharded = !sharded_dirs.is_empty();
|
||||
let chosen_variant = variant
|
||||
.as_deref()
|
||||
.map(|v| v.trim())
|
||||
.filter(|v| !v.is_empty());
|
||||
|
||||
if is_sharded {
|
||||
// pick variant: explicit > first alphabetically
|
||||
let target_dir = if let Some(v) = chosen_variant {
|
||||
if sharded_dirs.contains_key(v) {
|
||||
v.to_string()
|
||||
} else {
|
||||
sharded_dirs.keys().next().cloned().unwrap_or_default()
|
||||
}
|
||||
} else {
|
||||
// prefer smallest? use first sorted
|
||||
let mut keys: Vec<String> = sharded_dirs.keys().cloned().collect();
|
||||
keys.sort();
|
||||
keys.into_iter().next().unwrap_or_default()
|
||||
};
|
||||
let files = sharded_dirs.get(&target_dir).cloned().unwrap_or_default();
|
||||
if files.is_empty() {
|
||||
store.push_line(&id, format!("✗ variant '{target_dir}' not found"));
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
let total_sz = dir_sizes.get(&target_dir).copied().unwrap_or(0);
|
||||
let q = if target_dir.contains("q2tp") {
|
||||
"Q2TP"
|
||||
} else if target_dir.contains("q4tp") {
|
||||
"Q4TP"
|
||||
} else {
|
||||
"READY_CMF"
|
||||
};
|
||||
store.update_quant(&id, q.to_string());
|
||||
store.push_line(
|
||||
&id,
|
||||
format!(
|
||||
"→ downloading {} ({} files, {}) from HuggingFace...",
|
||||
target_dir,
|
||||
files.len(),
|
||||
format_bytes(total_sz)
|
||||
),
|
||||
);
|
||||
store.set_total(&id, total_sz);
|
||||
store.set_progress(&id, 0.001, "downloading".into());
|
||||
|
||||
// output is a directory: ensure {base}.cmf is a directory
|
||||
let out_dir = std::path::Path::new(&output_abs);
|
||||
// Drop a previous copy first (re-download / update). There is no resume,
|
||||
// so it would be overwritten anyway — and leaving it there would make the
|
||||
// size poller read the old bytes and report 100% before we transfer any.
|
||||
cleanup_output(&output_abs);
|
||||
// if output ends with .cmf and variant is sharded, keep it as dir (e.g. model.cmf/parts-*)
|
||||
// create base dir
|
||||
let _ = std::fs::create_dir_all(out_dir);
|
||||
// live percentage from bytes landing on disk (curl runs quiet, per part)
|
||||
let _poller = spawn_size_poller(store.clone(), id.clone(), output_abs.clone());
|
||||
// download each part file
|
||||
for (idx, rel) in files.iter().enumerate() {
|
||||
let filename = rel.rsplit('/').next().unwrap_or(rel);
|
||||
let dest = out_dir.join(filename);
|
||||
let url = format!("https://huggingface.co/{repo}/resolve/main/{rel}");
|
||||
store.push_line(&id, format!(" [{}/{}] {filename}", idx + 1, files.len()));
|
||||
let mut cmd = tokio::process::Command::new("curl");
|
||||
cmd.arg("-f")
|
||||
.arg("-L")
|
||||
.arg("-s")
|
||||
.arg("-S")
|
||||
.arg("-o")
|
||||
.arg(&dest)
|
||||
.arg(&url);
|
||||
if let Some(t) = &hf_token {
|
||||
cmd.arg("-H").arg(format!("Authorization: Bearer {t}"));
|
||||
}
|
||||
cmd.stdout(Stdio::null());
|
||||
cmd.stderr(Stdio::piped());
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed {filename}: {e}"));
|
||||
cleanup_output(&output_abs);
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let t_err = child.stderr.take().map(|s| {
|
||||
let st = store.clone();
|
||||
let jid = id.clone();
|
||||
tokio::spawn(async move {
|
||||
let mut lines = BufReader::new(s).lines();
|
||||
while let Ok(Some(l)) = lines.next_line().await {
|
||||
let l = l.trim().to_string();
|
||||
if !l.is_empty() {
|
||||
st.push_line(&jid, l);
|
||||
}
|
||||
}
|
||||
})
|
||||
});
|
||||
// Race the part against a cancel — a single part can be many GB, so
|
||||
// waiting for it to finish first would make Cancel look dead.
|
||||
let status = tokio::select! {
|
||||
r = child.wait() => {
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
r.map_err(|e| e.to_string())
|
||||
}
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, "✗ cancelled by user — partial output removed".into());
|
||||
store.set_state(&id, "cancelled");
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
match status {
|
||||
Ok(s) if s.success() => {
|
||||
// exact byte count after each part (the poller covers the gaps)
|
||||
let on_disk = path_size(out_dir);
|
||||
if total_sz > 0 {
|
||||
store.set_bytes(&id, on_disk);
|
||||
} else {
|
||||
let frac = (idx + 1) as f32 / files.len() as f32;
|
||||
store.set_progress(&id, frac, "downloading".into());
|
||||
}
|
||||
}
|
||||
Ok(s) => {
|
||||
store.push_line(&id, format!("✗ failed {filename}: exit {:?}", s.code()));
|
||||
cleanup_output(&output_abs);
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed {filename}: {e}"));
|
||||
cleanup_output(&output_abs);
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
store.push_line(&id, "✓ done".into());
|
||||
store.finish(&id, true);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
|
||||
// single-file path (original logic)
|
||||
let (cmf_path, cmf_size) = if !cmf_files.is_empty() {
|
||||
// if variant specified and matches a file prefix, prefer it
|
||||
if let Some(v) = chosen_variant {
|
||||
cmf_files
|
||||
.iter()
|
||||
.find(|(p, _)| p.contains(v))
|
||||
.cloned()
|
||||
.unwrap_or_else(|| cmf_files[0].clone())
|
||||
} else {
|
||||
cmf_files[0].clone()
|
||||
}
|
||||
} else {
|
||||
let repo_name = repo.rsplit('/').next().unwrap_or(&repo);
|
||||
(format!("{repo_name}.cmf"), 0)
|
||||
};
|
||||
|
||||
let path_lower = cmf_path.to_ascii_lowercase();
|
||||
let detected_quant =
|
||||
if path_lower.contains("-q1t.") || repo.to_ascii_lowercase().contains("2bit") {
|
||||
"Q1T".to_string()
|
||||
} else if path_lower.contains("-q1.") || repo.to_ascii_lowercase().contains("cmf") {
|
||||
"Q1".to_string()
|
||||
} else if path_lower.contains("-q4.") {
|
||||
"Q4".to_string()
|
||||
} else if path_lower.contains("-q8.") {
|
||||
"Q8".to_string()
|
||||
} else {
|
||||
"READY_CMF".to_string()
|
||||
};
|
||||
store.update_quant(&id, detected_quant);
|
||||
|
||||
let download_url = format!("https://huggingface.co/{repo}/resolve/main/{cmf_path}");
|
||||
// The tree listing is the usual source of the size; ask the CDN directly when
|
||||
// it wasn't (private repo, tree fetch failed) so the bar still gets a total.
|
||||
let total = if cmf_size > 0 {
|
||||
cmf_size
|
||||
} else {
|
||||
head_content_length(&download_url, hf_token.as_deref())
|
||||
.await
|
||||
.unwrap_or(0)
|
||||
};
|
||||
store.push_line(&id, {
|
||||
let sz = if total > 0 {
|
||||
format!(" ({})", format_bytes(total))
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
format!("→ downloading {cmf_path}{sz} from HuggingFace...")
|
||||
});
|
||||
store.set_total(&id, total);
|
||||
store.set_progress(&id, 0.001, "downloading".into());
|
||||
// Drop a previous copy first (re-download / update). There is no resume, so
|
||||
// it would be overwritten anyway — and leaving it there would make the size
|
||||
// poller read the old bytes and report 100% before we transfer any.
|
||||
cleanup_output(&output_abs);
|
||||
// Progress comes from bytes on disk only. curl's own meter is worse than
|
||||
// useless here: with -L it prints a complete 0→100% bar for the redirect hop
|
||||
// before the real transfer starts, which used to pin the job at 100%.
|
||||
let _poller = spawn_size_poller(store.clone(), id.clone(), output_abs.clone());
|
||||
|
||||
let mut cmd = tokio::process::Command::new("curl");
|
||||
cmd.arg("-f")
|
||||
.arg("-L")
|
||||
.arg("-s")
|
||||
.arg("-S")
|
||||
.arg("-o")
|
||||
.arg(&output_abs);
|
||||
|
||||
if let Some(t) = &hf_token {
|
||||
cmd.arg("-H").arg(format!("Authorization: Bearer {t}"));
|
||||
}
|
||||
|
||||
cmd.arg(&download_url);
|
||||
cmd.stderr(std::process::Stdio::piped());
|
||||
cmd.stdout(std::process::Stdio::null());
|
||||
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed to start curl download: {e}"));
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// -sS leaves stderr for real errors only — drain it into the log so a failure
|
||||
// says what went wrong instead of just an exit code.
|
||||
let stderr = child.stderr.take();
|
||||
let mut t_err = None;
|
||||
if let Some(err_stream) = stderr {
|
||||
let store_clone = store.clone();
|
||||
let id_clone = id.clone();
|
||||
t_err = Some(tokio::spawn(async move {
|
||||
let mut lines = BufReader::new(err_stream).lines();
|
||||
while let Ok(Some(l)) = lines.next_line().await {
|
||||
let l = l.trim().to_string();
|
||||
if !l.is_empty() {
|
||||
store_clone.push_line(&id_clone, l);
|
||||
}
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
tokio::select! {
|
||||
status_res = child.wait() => {
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
match status_res {
|
||||
Ok(status) if status.success() => {
|
||||
store.push_line(&id, "✓ done".into());
|
||||
store.finish(&id, true);
|
||||
}
|
||||
Ok(status) => {
|
||||
cleanup_output(&output_abs);
|
||||
// curl 23 = write error: almost always a non-writable models dir
|
||||
let hint = match status.code() {
|
||||
Some(23) => " (write error — is the models dir writable?)",
|
||||
Some(22) => " (HTTP error — repo/file not found or auth required)",
|
||||
_ => "",
|
||||
};
|
||||
store.push_line(&id, format!("✗ download failed: exit code {:?}{hint}", status.code()));
|
||||
store.finish(&id, false);
|
||||
}
|
||||
Err(e) => {
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, format!("✗ download process error: {e}"));
|
||||
store.finish(&id, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, "✗ cancelled by user — partial output removed".into());
|
||||
store.set_state(&id, "cancelled");
|
||||
}
|
||||
}
|
||||
store.drop_cancel(&id);
|
||||
}
|
||||
|
||||
/// into the job log. Fails fast if the converter script is missing.
|
||||
pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Result<String, String> {
|
||||
if p.repo.trim().is_empty() {
|
||||
@@ -311,12 +941,32 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
}
|
||||
}
|
||||
}
|
||||
let _ = std::fs::create_dir_all(&cfg.models_dir);
|
||||
let base = if p.name.trim().is_empty() {
|
||||
std::fs::create_dir_all(&cfg.models_dir).map_err(|e| {
|
||||
format!(
|
||||
"cannot create models dir {}: {e} — check that the gateway has write \
|
||||
access (in Docker, mount a writable volume at the models path)",
|
||||
cfg.models_dir
|
||||
)
|
||||
})?;
|
||||
let mut base = if p.name.trim().is_empty() {
|
||||
sanitize(p.repo.rsplit('/').next().unwrap_or(&p.repo))
|
||||
} else {
|
||||
sanitize(&p.name)
|
||||
};
|
||||
// for sharded variants make output distinct (e.g. DeepSeek parts-q2tp vs q4tp); single-file CMF ignores variant
|
||||
if let Some(v) = p
|
||||
.variant
|
||||
.as_deref()
|
||||
.map(|s| s.trim())
|
||||
.filter(|s| !s.is_empty())
|
||||
{
|
||||
if v.starts_with("parts-") {
|
||||
let vs = sanitize(v);
|
||||
if !base.ends_with(&vs) {
|
||||
base = format!("{base}-{vs}");
|
||||
}
|
||||
}
|
||||
}
|
||||
let output = std::path::Path::new(&cfg.models_dir)
|
||||
.join(format!("{base}.cmf"))
|
||||
.to_string_lossy()
|
||||
@@ -325,19 +975,66 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
.map(|d| d.join(format!("{base}.cmf")).to_string_lossy().to_string())
|
||||
.unwrap_or_else(|_| output.clone());
|
||||
|
||||
let id = gen_id(&p.repo);
|
||||
let id = gen_id(&format!(
|
||||
"{}::{}",
|
||||
p.repo,
|
||||
p.variant.as_deref().unwrap_or("")
|
||||
));
|
||||
// if same repo/variant is already running, don't reset progress — just return it
|
||||
if let Some(existing) = store.get(&id) {
|
||||
if existing.state == "running" {
|
||||
return Ok(id);
|
||||
}
|
||||
}
|
||||
let is_cmf = is_cmf_repo(&p.repo);
|
||||
let quant_display = if is_cmf {
|
||||
let r = p.repo.to_lowercase();
|
||||
if r.contains("2bit") || r.contains("q1t") {
|
||||
"Q1T".to_string()
|
||||
} else if r.contains("1.7bcmf")
|
||||
|| r.contains("27bcmf")
|
||||
|| r.contains("cmf")
|
||||
|| p.quant.is_empty()
|
||||
|| p.quant == "auto"
|
||||
|| p.quant == "Q8_2F"
|
||||
{
|
||||
"Q1".to_string()
|
||||
} else {
|
||||
p.quant.clone()
|
||||
}
|
||||
} else if p.quant.is_empty() || p.quant == "auto" {
|
||||
"q8".to_string()
|
||||
} else {
|
||||
p.quant.clone()
|
||||
};
|
||||
store.insert(Job {
|
||||
id: id.clone(),
|
||||
repo: p.repo.clone(),
|
||||
quant: p.quant.clone(),
|
||||
quant: quant_display.clone(),
|
||||
output: output_abs.clone(),
|
||||
name: base.clone(),
|
||||
variant: p
|
||||
.variant
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(str::to_string),
|
||||
state: "running".into(),
|
||||
log: vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)],
|
||||
log: if is_cmf {
|
||||
vec![format!(
|
||||
"→ downloading ready .cmf model {} ({})",
|
||||
p.repo, quant_display
|
||||
)]
|
||||
} else {
|
||||
vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)]
|
||||
},
|
||||
started: now(),
|
||||
finished: None,
|
||||
size_bytes: None,
|
||||
progress: Some(0.0),
|
||||
phase: Some("starting".into()),
|
||||
total_bytes: None,
|
||||
done_bytes: None,
|
||||
});
|
||||
let cancel = Arc::new(Notify::new());
|
||||
store.set_cancel(&id, cancel.clone());
|
||||
@@ -348,6 +1045,15 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
std::env::var(&cfg.hf_token_env).ok()
|
||||
};
|
||||
|
||||
if is_cmf && !use_python {
|
||||
let ret_id = id.clone();
|
||||
let variant = p.variant.clone();
|
||||
tokio::spawn(async move {
|
||||
download_cmf_repo(p.repo, output_abs, hf_token, store, id, cancel, variant).await;
|
||||
});
|
||||
return Ok(ret_id);
|
||||
}
|
||||
|
||||
// Program + args: `cortiq convert` by default, or the Python converter for
|
||||
// advanced options. Both stream `@PROGRESS <frac>` markers into the log.
|
||||
let (program, args, workdir): (String, Vec<String>, std::path::PathBuf) = if use_python {
|
||||
@@ -530,3 +1236,32 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
|
||||
Ok(ret_id)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_map_quant_variants() {
|
||||
assert_eq!(map_quant("Q8_2F"), "q8_2f");
|
||||
assert_eq!(map_quant("q82f"), "q8_2f");
|
||||
assert_eq!(map_quant("Q8_ROW"), "q8");
|
||||
assert_eq!(map_quant("Q4_BLOCK"), "q4");
|
||||
assert_eq!(map_quant("Q4_TILED"), "q4t");
|
||||
assert_eq!(map_quant("q4t"), "q4t");
|
||||
assert_eq!(map_quant("vbit"), "vbit");
|
||||
assert_eq!(map_quant("q1"), "q1");
|
||||
assert_eq!(map_quant("Q1P"), "q1p");
|
||||
assert_eq!(map_quant("q1s"), "q1s");
|
||||
assert_eq!(map_quant("Q1T"), "q1t");
|
||||
assert_eq!(map_quant("F16"), "f16");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_cmf_repo_check() {
|
||||
assert!(is_cmf_repo("infosave/Bonsai-8B_2bit_cmf"));
|
||||
assert!(is_cmf_repo("infosave/Bonsai-1.7Bcmf"));
|
||||
assert!(is_cmf_repo("user/model.cmf"));
|
||||
assert!(!is_cmf_repo("Qwen/Qwen2.5-0.5B-Instruct"));
|
||||
}
|
||||
}
|
||||
|
||||
+197
-18
@@ -38,37 +38,171 @@ use state::SharedState;
|
||||
about = "Universal LLM gateway with intelligent routing"
|
||||
)]
|
||||
struct Args {
|
||||
/// Path to the TOML config file.
|
||||
#[arg(long, default_value = "config/gateway.toml")]
|
||||
config: String,
|
||||
/// Path to the TOML config file. If omitted, uses `config/gateway.toml` when running
|
||||
/// inside the repo, otherwise `~/.cortiq-gateway/gateway.toml` (or `$CORTIQ_GATEWAY_HOME`).
|
||||
/// Missing file is created automatically — no manual copying needed.
|
||||
#[arg(long)]
|
||||
config: Option<String>,
|
||||
/// Admin token for the web panel (overrides `[admin].token_env`).
|
||||
#[arg(long)]
|
||||
admin_token: Option<String>,
|
||||
/// Do not automatically open the browser on first run.
|
||||
#[arg(long)]
|
||||
no_browser: bool,
|
||||
/// Do not auto-select a free port if the configured one is busy.
|
||||
#[arg(long)]
|
||||
no_port_search: bool,
|
||||
}
|
||||
|
||||
/// Resolve the admin token: env from `[admin].token_env` → `--admin-token` →
|
||||
/// generate a random one. Returns `(token, generated)`.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
|
||||
///
|
||||
/// By default no token is required (open mode). User sets it explicitly via
|
||||
/// `--admin-token`, `[admin].token` or `[admin].token_env` if they want auth.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> String {
|
||||
if let Some(env) = &cfg.admin.token_env {
|
||||
if let Ok(v) = std::env::var(env) {
|
||||
if !v.is_empty() {
|
||||
return (v, false);
|
||||
return v;
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(t) = &args.admin_token {
|
||||
if !t.is_empty() {
|
||||
return (t.clone(), false);
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
(admin::random_token(24), true)
|
||||
if let Some(t) = &cfg.admin.token {
|
||||
if !t.is_empty() {
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
String::new()
|
||||
}
|
||||
|
||||
fn is_docker() -> bool {
|
||||
std::path::Path::new("/.dockerenv").exists()
|
||||
|| std::fs::read_to_string("/proc/1/cgroup")
|
||||
.map(|s| s.contains("docker") || s.contains("kubepods"))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn should_open_browser(args: &Args) -> bool {
|
||||
if args.no_browser {
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CORTIQ_NO_BROWSER")
|
||||
.map(|v| !v.is_empty() && v != "0")
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CI").is_ok() || std::env::var("NO_BROWSER").is_ok() {
|
||||
return false;
|
||||
}
|
||||
if is_docker() {
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn parse_listen(listen: &str) -> (String, u16) {
|
||||
if let Some((host, port_s)) = listen.rsplit_once(':') {
|
||||
if let Ok(p) = port_s.parse::<u16>() {
|
||||
return (host.to_string(), p);
|
||||
}
|
||||
}
|
||||
(listen.to_string(), 9000)
|
||||
}
|
||||
|
||||
fn is_port_free(host: &str, port: u16) -> bool {
|
||||
let addr = format!("{host}:{port}");
|
||||
// bind test + connect test (docker port forwarding can make bind succeed while port is still forwarded)
|
||||
if std::net::TcpListener::bind(&addr).is_err() {
|
||||
return false;
|
||||
}
|
||||
use std::net::ToSocketAddrs;
|
||||
let probe_host = if host == "0.0.0.0" { "127.0.0.1" } else { host };
|
||||
if let Ok(mut addrs) = (probe_host, port).to_socket_addrs() {
|
||||
if let Some(a) = addrs.next() {
|
||||
if std::net::TcpStream::connect_timeout(&a, std::time::Duration::from_millis(200))
|
||||
.is_ok()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn resolve_listen(cfg: &mut Config, no_search: bool) -> String {
|
||||
if no_search {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
let (host, port) = parse_listen(&cfg.listen);
|
||||
let clean_host = if host.is_empty() {
|
||||
"0.0.0.0".to_string()
|
||||
} else {
|
||||
host
|
||||
};
|
||||
if is_port_free(&clean_host, port) {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
for p in (port + 1)..=port.saturating_add(50) {
|
||||
if is_port_free(&clean_host, p) {
|
||||
let new_listen = format!("{clean_host}:{p}");
|
||||
eprintln!("port {port} busy — switching to free port {p} ({new_listen})");
|
||||
cfg.listen = new_listen.clone();
|
||||
return new_listen;
|
||||
}
|
||||
}
|
||||
if !is_port_free(&clean_host, port) {
|
||||
eprintln!(
|
||||
"warning: port {port} busy and no free port found in next 50, will try to bind anyway"
|
||||
);
|
||||
}
|
||||
cfg.listen.clone()
|
||||
}
|
||||
|
||||
fn setup_marker_path(config_path: &str) -> std::path::PathBuf {
|
||||
// marker lives next to config file: gateway.toml -> .setup_done
|
||||
std::path::Path::new(config_path)
|
||||
.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.join(".setup_done")
|
||||
}
|
||||
|
||||
fn needs_first_run_wizard(cfg: &Config, config_path: &str) -> bool {
|
||||
let marker = setup_marker_path(config_path);
|
||||
if marker.exists() {
|
||||
return false;
|
||||
}
|
||||
// also consider "needs setup" if model pool is empty and no managed cmf
|
||||
if !cfg.models.is_empty() || !cfg.cmf.effective_servers().is_empty() {
|
||||
// still show wizard once if marker missing, but not if user already has models
|
||||
// we treat missing marker + existing models as "already configured" to avoid nagging
|
||||
// persist marker lazily
|
||||
let _ = std::fs::write(&marker, "done");
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let args = Args::parse();
|
||||
let config_path = args
|
||||
.config
|
||||
.clone()
|
||||
.unwrap_or_else(config::default_config_path);
|
||||
|
||||
let cfg = Config::load(&args.config)?;
|
||||
let mut cfg = Config::load_or_create(&config_path)?;
|
||||
// auto-pick a free port if the configured one is busy (unless disabled)
|
||||
let original_listen = cfg.listen.clone();
|
||||
let listen = resolve_listen(&mut cfg, args.no_port_search);
|
||||
// persist the resolved port if it changed
|
||||
if listen != original_listen {
|
||||
let _ = cfg.save(&config_path);
|
||||
}
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
@@ -76,11 +210,11 @@ async fn main() -> anyhow::Result<()> {
|
||||
)
|
||||
.init();
|
||||
|
||||
let listen = cfg.listen.clone();
|
||||
let admin_enabled = cfg.admin.enabled;
|
||||
let (admin_token, generated) = resolve_admin_token(&cfg, &args);
|
||||
let admin_token = resolve_admin_token(&cfg, &args);
|
||||
let first_run = needs_first_run_wizard(&cfg, &config_path);
|
||||
|
||||
let state = SharedState::build(cfg, args.config.clone())?;
|
||||
let state = SharedState::build(cfg, config_path.clone())?;
|
||||
|
||||
// Close the loop with the CMF format (github.com/infosave2007/cmf): install
|
||||
// / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
|
||||
@@ -94,6 +228,7 @@ async fn main() -> anyhow::Result<()> {
|
||||
|
||||
let mut app = Router::new()
|
||||
.route("/healthz", get(|| async { "ok" }))
|
||||
.route("/health", get(|| async { "ok" }))
|
||||
.route("/readyz", get(|| async { "ready" }))
|
||||
.route("/metrics", get(admin::metrics))
|
||||
.merge(protocols::build_router());
|
||||
@@ -104,16 +239,60 @@ async fn main() -> anyhow::Result<()> {
|
||||
.fallback(admin::assets::fallback);
|
||||
}
|
||||
|
||||
let app = app.with_state(state);
|
||||
let cors = tower_http::cors::CorsLayer::new()
|
||||
.allow_origin(tower_http::cors::Any)
|
||||
.allow_methods(tower_http::cors::Any)
|
||||
.allow_headers(tower_http::cors::Any);
|
||||
|
||||
let app = app.with_state(state).layer(cors);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(&listen).await?;
|
||||
tracing::info!("cortiq-gateway listening on {listen}");
|
||||
if admin_enabled {
|
||||
tracing::info!("admin console: http://{listen}/admin");
|
||||
if generated {
|
||||
tracing::warn!(
|
||||
"admin token (auto-generated, set [admin].token_env to persist): {admin_token}"
|
||||
);
|
||||
let host_for_url = if listen.starts_with("0.0.0.0:") {
|
||||
listen.replacen("0.0.0.0", "127.0.0.1", 1)
|
||||
} else {
|
||||
listen.clone()
|
||||
};
|
||||
// On first run open the onboarding wizard, otherwise the normal console.
|
||||
let admin_path = if first_run {
|
||||
"/admin#/onboarding"
|
||||
} else {
|
||||
"/admin"
|
||||
};
|
||||
let url = if admin_token.is_empty() {
|
||||
format!("http://{host_for_url}{admin_path}")
|
||||
} else {
|
||||
format!("http://{host_for_url}{admin_path}?token={admin_token}")
|
||||
};
|
||||
if admin_token.is_empty() {
|
||||
tracing::info!("admin console (open, no token): http://{host_for_url}/admin");
|
||||
} else {
|
||||
tracing::info!("admin console: http://{host_for_url}/admin");
|
||||
}
|
||||
if first_run {
|
||||
tracing::info!("first run — opening onboarding wizard at {url}");
|
||||
}
|
||||
if admin_enabled && should_open_browser(&args) {
|
||||
let browser_url = url.clone();
|
||||
tokio::spawn(async move {
|
||||
// give server a moment to be ready
|
||||
tokio::time::sleep(std::time::Duration::from_millis(600)).await;
|
||||
#[cfg(feature = "open-browser")]
|
||||
{
|
||||
if let Err(e) = open::that(&browser_url) {
|
||||
tracing::warn!("failed to open browser: {e}");
|
||||
} else {
|
||||
tracing::info!("opened browser at {browser_url}");
|
||||
}
|
||||
}
|
||||
#[cfg(not(feature = "open-browser"))]
|
||||
{
|
||||
tracing::info!("open browser: {browser_url} (feature disabled)");
|
||||
}
|
||||
});
|
||||
} else if first_run {
|
||||
tracing::info!("open this URL to finish setup: {url}");
|
||||
}
|
||||
}
|
||||
axum::serve(listener, app).await?;
|
||||
|
||||
@@ -29,6 +29,7 @@ pub struct GenParams {
|
||||
pub temperature: Option<f32>,
|
||||
pub max_tokens: Option<u32>,
|
||||
pub top_p: Option<f32>,
|
||||
pub think_budget: Option<u32>,
|
||||
pub stop: Vec<String>,
|
||||
/// Fields we do not interpret but must proxy through to the provider.
|
||||
pub passthrough: serde_json::Map<String, serde_json::Value>,
|
||||
|
||||
+117
-2
@@ -341,7 +341,64 @@ impl Pipeline {
|
||||
route_source
|
||||
);
|
||||
|
||||
match provider.chat(req.clone()).await {
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat(req_call).await {
|
||||
Ok(mut resp) => {
|
||||
let cost =
|
||||
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
|
||||
@@ -415,7 +472,65 @@ impl Pipeline {
|
||||
model_id,
|
||||
route_source
|
||||
);
|
||||
match provider.chat_stream(req.clone()).await {
|
||||
// Merge per-model/server config (same as run_inner).
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat_stream(req_call).await {
|
||||
Ok(stream) => {
|
||||
let info = RouteInfo {
|
||||
task_label: decision.task_label.clone(),
|
||||
|
||||
@@ -101,6 +101,7 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
|
||||
@@ -49,6 +49,10 @@ struct OpenAiChatRequest {
|
||||
#[serde(default)]
|
||||
top_p: Option<f32>,
|
||||
#[serde(default)]
|
||||
think_budget: Option<u32>,
|
||||
#[serde(default)]
|
||||
reasoning_effort: Option<u32>,
|
||||
#[serde(default)]
|
||||
stream: bool,
|
||||
#[serde(default)]
|
||||
tools: Vec<serde_json::Value>,
|
||||
@@ -93,6 +97,7 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: req.think_budget.or(req.reasoning_effort),
|
||||
stop: Vec::new(),
|
||||
passthrough: req.rest,
|
||||
},
|
||||
|
||||
@@ -82,6 +82,7 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
|
||||
@@ -42,9 +42,20 @@ impl OpenAiProvider {
|
||||
if let Some(t) = req.params.temperature {
|
||||
body["temperature"] = t.into();
|
||||
}
|
||||
if let Some(p) = req.params.top_p {
|
||||
body["top_p"] = p.into();
|
||||
}
|
||||
if let Some(m) = req.params.max_tokens {
|
||||
body["max_tokens"] = m.into();
|
||||
}
|
||||
eprintln!("[to_wire] think_budget={:?}", req.params.think_budget);
|
||||
if let Some(tb) = req.params.think_budget {
|
||||
body["think_budget"] = tb.into();
|
||||
body["reasoning_effort"] = tb.into();
|
||||
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
|
||||
body["enable_thinking"] = (tb > 0).into();
|
||||
eprintln!("[to_wire] SET enable_thinking={}", tb > 0);
|
||||
}
|
||||
if !req.tools.is_empty() {
|
||||
body["tools"] = serde_json::Value::Array(req.tools.clone());
|
||||
}
|
||||
@@ -52,6 +63,10 @@ impl OpenAiProvider {
|
||||
for (k, v) in &req.params.passthrough {
|
||||
body[k] = v.clone();
|
||||
}
|
||||
eprintln!(
|
||||
"[to_wire] enable_thinking={:?}",
|
||||
body.get("enable_thinking")
|
||||
);
|
||||
body
|
||||
}
|
||||
}
|
||||
|
||||
@@ -49,6 +49,13 @@ impl Registry {
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
think_budget: s.think_budget,
|
||||
system_prompt: s.system_prompt.clone(),
|
||||
o1: s.o1.clone(),
|
||||
skip_mtp: s.skip_mtp,
|
||||
};
|
||||
by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
|
||||
}
|
||||
|
||||
@@ -145,6 +145,14 @@ impl AppState {
|
||||
let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
|
||||
new_cfg.save(&self.config_path)?;
|
||||
self.live.store(Arc::new(live));
|
||||
|
||||
let cmf_rt = self.cmf.clone();
|
||||
let cmf_cfg = new_cfg.cmf.clone();
|
||||
tokio::spawn(async move {
|
||||
cmf_rt.stop().await;
|
||||
crate::cmf_runtime::manage(cmf_rt, cmf_cfg).await;
|
||||
});
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
Vendored
+197
@@ -397,3 +397,200 @@ tbody tr:hover td { background: var(--hover); }
|
||||
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
|
||||
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
|
||||
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
|
||||
|
||||
/* ---- responsive shell + touch layout ----------------------------------- */
|
||||
.menu-toggle, .nav-close, .mobile-nav-backdrop { display: none; }
|
||||
.hamburger { width: 18px; display: flex; flex-direction: column; gap: 4px; }
|
||||
.hamburger i { display: block; height: 2px; border-radius: 2px; background: currentColor; }
|
||||
body.modal-open { overflow: hidden; }
|
||||
|
||||
@media (max-width: 980px) {
|
||||
.app { display: block; min-height: 100dvh; }
|
||||
.main { min-height: 100dvh; }
|
||||
.menu-toggle { display: inline-flex; flex: 0 0 auto; }
|
||||
.nav-close { display: inline-flex; margin-left: auto; flex: 0 0 auto; font-size: 24px; line-height: 1; }
|
||||
.brand-copy { min-width: 0; }
|
||||
|
||||
.sidebar {
|
||||
display: flex;
|
||||
position: fixed;
|
||||
inset: 0 auto 0 0;
|
||||
width: min(84vw, 310px);
|
||||
height: 100dvh;
|
||||
padding: max(18px, env(safe-area-inset-top)) 14px max(14px, env(safe-area-inset-bottom));
|
||||
z-index: 70;
|
||||
box-shadow: 18px 0 45px rgba(0, 0, 0, .42);
|
||||
transform: translateX(-104%);
|
||||
visibility: hidden;
|
||||
transition: transform .22s ease, visibility 0s linear .22s;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.app.nav-open .sidebar {
|
||||
transform: translateX(0);
|
||||
visibility: visible;
|
||||
transition: transform .22s ease;
|
||||
}
|
||||
.mobile-nav-backdrop {
|
||||
display: block;
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 65;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border: 0;
|
||||
background: rgba(3, 7, 18, .62);
|
||||
opacity: 0;
|
||||
visibility: hidden;
|
||||
pointer-events: none;
|
||||
transition: opacity .22s ease, visibility 0s linear .22s;
|
||||
}
|
||||
.app.nav-open .mobile-nav-backdrop {
|
||||
opacity: 1;
|
||||
visibility: visible;
|
||||
pointer-events: auto;
|
||||
transition: opacity .22s ease;
|
||||
}
|
||||
body.mobile-nav-open { overflow: hidden; }
|
||||
|
||||
.topbar { padding: 10px 18px; min-height: 58px; gap: 10px; }
|
||||
.topbar .title { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.topbar .right { gap: 8px; min-width: 0; }
|
||||
.content { max-width: none; padding: 22px 20px 52px; }
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
body { font-size: 14px; }
|
||||
.status-pills { display: none; }
|
||||
.topbar { padding: 9px 12px; }
|
||||
.content { padding: 18px 14px 42px; }
|
||||
|
||||
.page-head { margin-bottom: 16px; }
|
||||
.page-head > .flex { align-items: stretch; flex-direction: column; }
|
||||
.page-head > .flex > .btn { width: 100%; }
|
||||
.page-head p { line-height: 1.45; }
|
||||
|
||||
.card { padding: 15px; margin-bottom: 14px; border-radius: 12px; }
|
||||
.card .card-head { align-items: flex-start; flex-wrap: wrap; }
|
||||
.card .card-head .right { width: 100%; margin-left: 0; flex-wrap: wrap; }
|
||||
.grid { gap: 14px; }
|
||||
.kpis { gap: 10px; margin-bottom: 14px; }
|
||||
.kpi { padding: 12px; border-radius: 11px; }
|
||||
.kpi .big { font-size: 21px; }
|
||||
|
||||
.row { display: grid; grid-template-columns: minmax(0, 1fr); gap: 0; }
|
||||
.row > * { min-width: 0; width: 100%; }
|
||||
input, select, textarea { min-height: 44px; font-size: 16px; }
|
||||
input[type="checkbox"], input[type="radio"] { min-height: 0; }
|
||||
.check { min-height: 36px; margin-bottom: 8px; }
|
||||
.btn, .icon-btn { min-height: 44px; }
|
||||
.btn { padding: 10px 14px; }
|
||||
.btn.sm, .btn.small { min-height: 38px; padding: 8px 11px; }
|
||||
|
||||
.table-wrap {
|
||||
width: 100%;
|
||||
overflow-x: auto;
|
||||
overscroll-behavior-x: contain;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: thin;
|
||||
}
|
||||
.table-wrap table { min-width: max-content; }
|
||||
th, td { padding: 10px; }
|
||||
|
||||
.tierrow { align-items: flex-start; flex-direction: column; gap: 8px; }
|
||||
.tierrow .tname { width: auto; }
|
||||
.chips { width: 100%; }
|
||||
.togglerow { align-items: flex-start; }
|
||||
|
||||
.pg { gap: 14px; }
|
||||
.routecard .kv { flex-wrap: wrap; }
|
||||
.routecard .kv .v { max-width: 100%; overflow-wrap: anywhere; }
|
||||
.answer { padding: 12px; }
|
||||
.seg { max-width: 100%; overflow-x: auto; }
|
||||
.seg button { min-height: 40px; white-space: nowrap; }
|
||||
|
||||
.hf-grid { grid-template-columns: 1fr; max-height: 56dvh; }
|
||||
.job-head, .actions { align-items: stretch; flex-wrap: wrap; }
|
||||
.actions .btn { flex: 1 1 150px; }
|
||||
.key-row { flex-direction: column; }
|
||||
.key-row .btn { width: 100%; }
|
||||
.cmf-server-row { align-items: stretch !important; padding: 10px; border: 1px solid var(--border-soft); border-radius: 10px; }
|
||||
.cmf-server-row > input, .cmf-server-row > select { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
|
||||
.cmf-server-row > .btn { flex: 1 1 calc(50% - 6px); }
|
||||
.cmf-server-row > .small { flex: 1 1 100%; }
|
||||
|
||||
.modal-backdrop { align-items: flex-end; padding: 8px 0 0; }
|
||||
.modal {
|
||||
max-width: none;
|
||||
max-height: calc(100dvh - 8px);
|
||||
overflow-y: auto;
|
||||
padding: 18px 15px max(14px, env(safe-area-inset-bottom));
|
||||
border-radius: 18px 18px 0 0;
|
||||
}
|
||||
.modal .modal-foot {
|
||||
position: sticky;
|
||||
bottom: calc(-1 * max(14px, env(safe-area-inset-bottom)));
|
||||
margin: 12px -15px 0;
|
||||
padding: 12px 15px max(14px, env(safe-area-inset-bottom));
|
||||
background: color-mix(in srgb, var(--card) 94%, transparent);
|
||||
backdrop-filter: blur(10px);
|
||||
border-top: 1px solid var(--border-soft);
|
||||
}
|
||||
.modal .modal-foot .btn { flex: 1; }
|
||||
|
||||
.toasts { left: 12px; right: 12px; bottom: max(12px, env(safe-area-inset-bottom)); }
|
||||
.toast { min-width: 0; width: 100%; }
|
||||
.login-wrap { align-items: flex-start; padding: max(20px, env(safe-area-inset-top)) 14px; }
|
||||
.login-card { padding: 22px 18px; margin-top: 8dvh; }
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.topbar .title { font-size: 14px; }
|
||||
.topbar .right { gap: 6px; }
|
||||
.lang-select { width: 54px; min-height: 44px; padding: 6px; }
|
||||
.icon-btn { width: 42px; height: 42px; }
|
||||
.kpis { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.kpi .lbl { line-height: 1.25; }
|
||||
.barlist .b { grid-template-columns: minmax(0, 1fr) auto; gap: 6px 10px; }
|
||||
.barlist .b .track { grid-column: 1 / -1; grid-row: 2; }
|
||||
.barlist .b .val { min-width: 0; }
|
||||
.empty { padding: 28px 12px; }
|
||||
}
|
||||
|
||||
@media (max-width: 360px) {
|
||||
.topbar .title { display: none; }
|
||||
.kpis { grid-template-columns: 1fr; }
|
||||
}
|
||||
|
||||
/* ---- onboarding ------------------------------------------------------------ */
|
||||
.onb-shell { min-height: 100vh; display: flex; align-items: flex-start; justify-content: center; padding: 32px 20px; background: var(--bg-grad); }
|
||||
.onb-content { width: 100%; max-width: 860px; }
|
||||
.onb-wrap { max-width: 860px; margin: 0 auto; }
|
||||
.onb-header { display: flex; align-items: center; justify-content: space-between; margin-bottom: 18px; }
|
||||
.onb-dots { display: flex; gap: 8px; }
|
||||
.onb-dots .dot { width: 28px; height: 6px; border-radius: 999px; background: var(--border); }
|
||||
.onb-dots .dot.on { background: var(--accent); }
|
||||
.onb-dots .dot.done { background: color-mix(in srgb, var(--accent) 55%, transparent); }
|
||||
.onb-title { font-size: 12px; font-weight: 700; color: var(--muted); letter-spacing: 0.04em; text-transform: uppercase; }
|
||||
.onb-step h2 { font-size: 22px; margin-bottom: 6px; }
|
||||
.onb-langs { display: grid; grid-template-columns: repeat(auto-fill, minmax(132px, 1fr)); gap: 12px; margin: 18px 0; }
|
||||
.onb-lang {
|
||||
display: flex; flex-direction: column; align-items: center; gap: 10px;
|
||||
padding: 18px 10px; border-radius: 14px; border: 1.5px solid var(--border);
|
||||
background: var(--card); cursor: pointer; transition: .15s; font-family: inherit;
|
||||
}
|
||||
.onb-lang:hover { border-color: color-mix(in srgb, var(--accent) 45%, transparent); transform: translateY(-1px); }
|
||||
.onb-lang.sel { border-color: var(--accent); background: color-mix(in srgb, var(--accent) 10%, var(--card)); box-shadow: 0 6px 18px rgba(99,102,241,.18); }
|
||||
.onb-lang .flag { font-size: 32px; line-height: 1; }
|
||||
.onb-lang b { font-size: 14px; font-weight: 700; color: var(--text); }
|
||||
/* Featured gallery: the list grows as new models are published, so give it more
|
||||
room than the search grid before it starts scrolling. */
|
||||
.featured-section .hf-grid { max-height: min(62vh, 760px); }
|
||||
/* Live download progress inside a featured card (empty until a job runs) */
|
||||
.card-prog:empty { display: none; }
|
||||
.card-prog { margin-top: 6px; }
|
||||
.card-prog .pbar { margin: 4px 0; }
|
||||
.card-prog .btn.sm { padding: 0 6px; line-height: 18px; }
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sidebar, .mobile-nav-backdrop { transition: none; }
|
||||
}
|
||||
|
||||
Vendored
+1
-1
@@ -19,6 +19,6 @@
|
||||
<noscript>This console requires JavaScript.</noscript>
|
||||
<div id="app"><div class="boot">Loading…</div></div>
|
||||
<div id="toasts" class="toasts"></div>
|
||||
<script type="module" src="/admin/js/app.js"></script>
|
||||
<script type="module" src="/admin/js/app.js?v=3"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Vendored
+24
-4
@@ -19,7 +19,10 @@ export function setToken(tok) {
|
||||
}
|
||||
|
||||
async function j(method, path, body) {
|
||||
const opt = { method, headers: { Authorization: "Bearer " + getToken() } };
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const opt = { method, headers };
|
||||
if (body !== undefined) {
|
||||
opt.headers["content-type"] = "application/json";
|
||||
opt.body = JSON.stringify(body);
|
||||
@@ -60,8 +63,7 @@ export const api = {
|
||||
listModels: () => j("GET", "/models"),
|
||||
createModel: (b) => j("POST", "/models", b),
|
||||
updateModel: (id, b) => j("PUT", "/models/" + encodeURIComponent(id), b),
|
||||
deleteModel: (id, deleteFile = true) =>
|
||||
j("DELETE", "/models/" + encodeURIComponent(id) + "?delete_file=" + (deleteFile ? "true" : "false")),
|
||||
deleteModel: (id) => j("DELETE", "/models/" + encodeURIComponent(id)),
|
||||
probeModel: (id) => j("POST", "/models/" + encodeURIComponent(id) + "/probe"),
|
||||
|
||||
// routing
|
||||
@@ -115,14 +117,32 @@ export const api = {
|
||||
deleteImport: (job) => j("DELETE", "/import/" + encodeURIComponent(job)),
|
||||
registerImport: (job) =>
|
||||
j("POST", "/import/" + encodeURIComponent(job) + "/register"),
|
||||
|
||||
// first-run onboarding
|
||||
setupStatus: () => j("GET", "/setup/status"),
|
||||
// Connects every finished download as a local model unless register === false.
|
||||
setupComplete: (lang, register) => j("POST", "/setup/complete", { lang, register }),
|
||||
setupFeatured: () => j("GET", "/setup/featured"),
|
||||
// `name` is the exact output stem from setup/featured — it wins over the
|
||||
// repo/variant guess so the delete always hits the file the download wrote.
|
||||
deleteSetupFile: (repo, variant, name) =>
|
||||
j(
|
||||
"DELETE",
|
||||
`/setup/file?repo=${encodeURIComponent(repo)}` +
|
||||
(variant ? `&variant=${encodeURIComponent(variant)}` : "") +
|
||||
(name ? `&name=${encodeURIComponent(name)}` : "")
|
||||
),
|
||||
};
|
||||
|
||||
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
|
||||
// returns the routing decision read from X-Cortiq-* response headers.
|
||||
export async function testStream(body, onDelta) {
|
||||
const tok = getToken();
|
||||
const headers = { "content-type": "application/json" };
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(BASE + "/test/stream", {
|
||||
method: "POST",
|
||||
headers: { Authorization: "Bearer " + getToken(), "content-type": "application/json" },
|
||||
headers,
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (r.status === 401) {
|
||||
|
||||
Vendored
+71
-11
@@ -4,13 +4,14 @@ import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
|
||||
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
|
||||
import { renderLogin } from "./views/login.js";
|
||||
import { renderDashboard } from "./views/dashboard.js";
|
||||
import { renderModels } from "./views/models.js";
|
||||
import { renderModels } from "./views/models.js?v=3";
|
||||
import { renderImport } from "./views/import.js";
|
||||
import { renderRouting } from "./views/routing.js";
|
||||
import { renderProtocols } from "./views/protocols.js";
|
||||
import { renderKeys } from "./views/keys.js";
|
||||
import { renderPlayground } from "./views/playground.js";
|
||||
import { renderSettings } from "./views/settings.js";
|
||||
import { renderOnboarding } from "./views/onboarding.js";
|
||||
|
||||
const ROUTES = [
|
||||
{ id: "dashboard", icon: "▤", view: renderDashboard },
|
||||
@@ -21,6 +22,7 @@ const ROUTES = [
|
||||
{ id: "keys", icon: "🔑", view: renderKeys },
|
||||
{ id: "playground", icon: "✦", view: renderPlayground },
|
||||
{ id: "settings", icon: "⚙", view: renderSettings },
|
||||
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
|
||||
];
|
||||
|
||||
export const appState = { meta: null, health: null, version: "" };
|
||||
@@ -28,6 +30,17 @@ export const appState = { meta: null, health: null, version: "" };
|
||||
// allaigate site — where router keys are issued and topped up
|
||||
export const SITE_URL = "https://api.allaigate.com/";
|
||||
|
||||
let mobileNavOpen = false;
|
||||
|
||||
function setMobileNav(open) {
|
||||
mobileNavOpen = !!open;
|
||||
const app = document.querySelector(".app");
|
||||
if (app) app.classList.toggle("nav-open", mobileNavOpen);
|
||||
document.body.classList.toggle("mobile-nav-open", mobileNavOpen);
|
||||
const toggle = document.querySelector(".menu-toggle");
|
||||
if (toggle) toggle.setAttribute("aria-expanded", String(mobileNavOpen));
|
||||
}
|
||||
|
||||
function currentRoute() {
|
||||
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
|
||||
return ROUTES.find((r) => r.id === id) || ROUTES[0];
|
||||
@@ -40,19 +53,21 @@ const FAVICON =
|
||||
);
|
||||
|
||||
function sidebar(activeId) {
|
||||
const visibleRoutes = ROUTES.filter((r) => !r.hidden);
|
||||
return h(
|
||||
"aside",
|
||||
{ class: "sidebar", id: "sidebar" },
|
||||
{ class: "sidebar", id: "sidebar", "aria-label": "Main navigation" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "brand" },
|
||||
h("img", { src: FAVICON, alt: "" }),
|
||||
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
|
||||
h("div", { class: "brand-copy" }, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub"))),
|
||||
h("button", { class: "icon-btn nav-close", type: "button", title: "Close menu", "aria-label": "Close menu", onClick: () => setMobileNav(false) }, "×")
|
||||
),
|
||||
...ROUTES.map((r) =>
|
||||
...visibleRoutes.map((r) =>
|
||||
h(
|
||||
"a",
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id, onClick: () => setMobileNav(false) },
|
||||
h("span", { class: "ico" }, r.icon),
|
||||
t("nav." + r.id)
|
||||
)
|
||||
@@ -85,7 +100,7 @@ function statusPills() {
|
||||
: t("status.checking");
|
||||
return h(
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
{ class: "flex status-pills" },
|
||||
h(
|
||||
"span",
|
||||
{ class: "pill" },
|
||||
@@ -109,7 +124,7 @@ function langSelect() {
|
||||
title: "Language",
|
||||
onChange: (e) => setLang(e.target.value),
|
||||
},
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, l.label))
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, `${l.flag} ${l.label}`))
|
||||
);
|
||||
return sel;
|
||||
}
|
||||
@@ -136,6 +151,19 @@ function topbar(activeId) {
|
||||
return h(
|
||||
"header",
|
||||
{ class: "topbar" },
|
||||
h(
|
||||
"button",
|
||||
{
|
||||
class: "icon-btn menu-toggle",
|
||||
type: "button",
|
||||
title: "Menu",
|
||||
"aria-label": "Open menu",
|
||||
"aria-controls": "sidebar",
|
||||
"aria-expanded": String(mobileNavOpen),
|
||||
onClick: () => setMobileNav(!mobileNavOpen),
|
||||
},
|
||||
h("span", { class: "hamburger", "aria-hidden": "true" }, h("i"), h("i"), h("i"))
|
||||
),
|
||||
h("div", { class: "title" }, t("nav." + activeId)),
|
||||
h(
|
||||
"div",
|
||||
@@ -175,12 +203,24 @@ async function renderView() {
|
||||
|
||||
function renderShell() {
|
||||
const route = currentRoute();
|
||||
mobileNavOpen = false;
|
||||
document.body.classList.remove("mobile-nav-open");
|
||||
// onboarding — full screen without sidebar/topbar
|
||||
if (route.id === "onboarding") {
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h("div", { class: "onb-shell" }, h("div", { class: "onb-content", id: "view" }))
|
||||
);
|
||||
renderView();
|
||||
return;
|
||||
}
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "app" },
|
||||
sidebar(route.id),
|
||||
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
|
||||
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
|
||||
)
|
||||
);
|
||||
@@ -213,20 +253,34 @@ export async function boot() {
|
||||
setToken(urlTok);
|
||||
history.replaceState({}, "", location.pathname + location.hash);
|
||||
}
|
||||
if (!getToken()) {
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// try without token first — open mode (no CORTIQ_ADMIN_TOKEN) succeeds,
|
||||
// protected mode returns 401 and we show login
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch (e) {
|
||||
if (String(e.message) === "unauthorized") {
|
||||
// if we have no token stored, show login; with a stale token also show login
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// gateway reachable but error — still show shell
|
||||
// if meta failed for other reason but token was present, fall through
|
||||
if (!getToken()) {
|
||||
// open mode but gateway unreachable — still try shell; health will show
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch {}
|
||||
}
|
||||
}
|
||||
// first-run check — redirect to onboarding if needed (unless already there)
|
||||
try {
|
||||
const st = await api.setupStatus();
|
||||
if (st.needs_setup && location.hash !== "#/onboarding") {
|
||||
location.hash = "#/onboarding";
|
||||
}
|
||||
} catch {}
|
||||
if (!location.hash) location.hash = "#/dashboard";
|
||||
renderShell();
|
||||
refreshHealth();
|
||||
@@ -243,6 +297,12 @@ onLangChange(() => {
|
||||
window.addEventListener("hashchange", () => {
|
||||
if (document.querySelector(".app")) renderShell();
|
||||
});
|
||||
window.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Escape" && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
window.addEventListener("resize", () => {
|
||||
if (window.innerWidth > 980 && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
setInterval(() => {
|
||||
if (document.querySelector(".app")) refreshHealth();
|
||||
}, 20000);
|
||||
|
||||
Vendored
+7
-7
@@ -10,13 +10,13 @@ import tr from "./i18n/tr.js";
|
||||
|
||||
const DICTS = { en, ru, de, fr, es, zh, tr };
|
||||
export const LANGS = [
|
||||
{ code: "en", label: "English" },
|
||||
{ code: "ru", label: "Русский" },
|
||||
{ code: "de", label: "Deutsch" },
|
||||
{ code: "fr", label: "Français" },
|
||||
{ code: "es", label: "Español" },
|
||||
{ code: "zh", label: "中文" },
|
||||
{ code: "tr", label: "Türkçe" },
|
||||
{ code: "en", label: "English", flag: "🇬🇧" },
|
||||
{ code: "ru", label: "Русский", flag: "🇷🇺" },
|
||||
{ code: "de", label: "Deutsch", flag: "🇩🇪" },
|
||||
{ code: "fr", label: "Français", flag: "🇫🇷" },
|
||||
{ code: "es", label: "Español", flag: "🇪🇸" },
|
||||
{ code: "zh", label: "中文", flag: "🇨🇳" },
|
||||
{ code: "tr", label: "Türkçe", flag: "🇹🇷" },
|
||||
];
|
||||
|
||||
const listeners = new Set();
|
||||
|
||||
Vendored
+191
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Routing",
|
||||
"nav.protocols": "Protokolle",
|
||||
"nav.keys": "API-Schlüssel",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Testbereich",
|
||||
"nav.settings": "Einstellungen",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Router-Key & Abrechnung",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "nach Aufgabe",
|
||||
"dash.groupby.account": "nach Account",
|
||||
"dash.recent.title": "Letzte Anfragen",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Playground ausprobieren.",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Testbereich ausprobieren.",
|
||||
"dash.col.time": "Zeit",
|
||||
"dash.col.model": "Modell",
|
||||
"dash.col.tier": "Tier",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "alle",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Testbereich",
|
||||
"pg.subtitle": "Prompt durch die Live-Pipeline senden und Routing-Entscheidung prüfen.",
|
||||
"pg.system": "System-Prompt",
|
||||
"pg.systemPlaceholder": "optionale Systemnachricht",
|
||||
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "Eintrags-TTL",
|
||||
"settings.cache.embed": "Embedding-Modell",
|
||||
"settings.cache.note": "Cache-Einstellungen werden nach einem Neustart wirksam.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Modell importieren",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Logs leeren",
|
||||
"dash.clearConfirm": "Alle erfassten Statistiken und Logs löschen? Das lässt sich nicht rückgängig machen.",
|
||||
"dash.cleared": "Logs geleert",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Gib eine ID oder einen Modellnamen ein",
|
||||
"models.form.apiKey": "API-Schlüssel",
|
||||
"models.form.apiKeyPlaceholder": "API-Schlüssel einfügen",
|
||||
"models.form.apiKeyNotNeeded": "nicht erforderlich (lokaler Server)",
|
||||
"models.form.apiKeyHint": "Wird sicher im Gateway gespeichert und nicht erneut angezeigt.",
|
||||
"models.form.keyStored": "Ein Schlüssel ist gespeichert — leer lassen, um ihn zu behalten, oder einen neuen einfügen.",
|
||||
"models.form.keyFromEnv": "Es wird ein Schlüssel aus einer Umgebungsvariable verwendet.",
|
||||
"models.form.paramsTitle": "Generierungs- und Denkparameter (O(1), MTP, Temperatur)",
|
||||
"models.form.temperature": "Temperatur",
|
||||
"models.form.temperatureHint": "Generierungstemperatur (0.0 = deterministisch, 1.0+ = kreativ).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling-Schwelle (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max. Tokens",
|
||||
"models.form.maxTokensHint": "Maximale Anzahl generierter Tokens.",
|
||||
"models.form.thinkBudget": "Denkmodus / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Token-Budget fürs Nachdenken (0 = aus, N > 0 = max. Reasoning-Tokens).",
|
||||
"models.form.systemPrompt": "Standard-System-Prompt",
|
||||
"models.form.systemPromptHint": "Wird automatisch vorangestellt, wenn die Anfrage keinen System-Prompt enthält.",
|
||||
"models.form.systemPromptPlaceholder": "Du bist ein hilfreicher Assistent…",
|
||||
"models.form.testKey": "Testen & Modelle laden",
|
||||
"models.form.testing": "wird geprüft…",
|
||||
"models.form.foundModels": "{n} Modelle geladen",
|
||||
"models.form.modelPlaceholder": "tippen zum Suchen…",
|
||||
"models.form.modelHint": "Klicke „Testen & Modelle laden“, um die Liste mit deinem Schlüssel abzurufen, und wähle dann eines aus.",
|
||||
"models.managed": "verwaltet",
|
||||
"models.managedHint": "Wird vom verwalteten CMF-Server bereitgestellt (Einstellungen → Lokale Modelle). Dort konfigurieren.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Router verwenden (aus = nur lokale Modelle)",
|
||||
"settings.cmf": "Lokale Modelle (CMF)",
|
||||
"settings.cmf.localOnly": "Nur lokale Modelle verwenden",
|
||||
"settings.cmf.manage": "Lokalen Modellserver starten",
|
||||
"settings.cmf.autoInstall": "cortiq automatisch installieren (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq automatisch aktualisieren",
|
||||
"settings.cmf.model": "Pfad zum lokalen .cmf-Modell",
|
||||
"settings.cmf.port": "Port des lokalen Servers",
|
||||
"settings.cmf.modelId": "Modell-ID im Pool",
|
||||
"settings.cmf.note": "Stellt ein lokales .cmf über cortiq bereit (github.com/infosave2007/cmf). Funktioniert auch, wenn der Router nicht erreichbar ist.",
|
||||
"settings.cmf.threads": "Worker-Threads",
|
||||
"settings.cmf.gpu": "GPU verwenden (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Threads parallelisieren die Decode-Matvecs (~3× schneller; 0 = Standard der Runtime). GPU-Offload hilft vor allem beim Prefill langer Prompts — das Decoding ist speicherbandbreitenbegrenzt und daher auf Apple Silicon ungefähr neutral.",
|
||||
"settings.cmf.pick": "Modell auswählen",
|
||||
"settings.cmf.models": "Lokale Modelle",
|
||||
"settings.cmf.modelsHelp": "Jede Zeile ist ein Modell, das das Gateway ausführt (eigenes cortiq serve auf eigenem Port) und im Pool registriert. Füge so viele hinzu, wie du .cmf-Dateien und freie Ports hast.",
|
||||
"settings.cmf.rowHint": "ID · .cmf-Datei · Port · Threads · GPU. Änderungen greifen nach einem Neustart; das Löschen einer Zeile stoppt diesen Server.",
|
||||
"settings.cmf.addServer": "+ Lokales Modell hinzufügen",
|
||||
"settings.cmf.checkPort": "Port prüfen",
|
||||
"settings.cmf.portFree": "Port ist frei",
|
||||
"settings.cmf.portBusy": "Port ist belegt",
|
||||
"settings.cmf.useSuggested": "{port} verwenden",
|
||||
"settings.cmf.serving": "{model} läuft auf :{port}",
|
||||
"settings.cmf.intro": "Modelle lokal mit der cortiq-Engine ausführen — funktioniert auch offline oder wenn der Router ausfällt.",
|
||||
"settings.cmf.runtime": "cortiq-Runtime",
|
||||
"settings.cmf.runtimeHelp": "Das Kommandozeilenwerkzeug `cortiq`, das .cmf-Modelle lädt und ausführt. Einmal installieren, bevor du einen lokalen Server startest.",
|
||||
"settings.cmf.install": "cortiq installieren",
|
||||
"settings.cmf.reinstall": "Neu installieren / aktualisieren",
|
||||
"settings.cmf.installing": "wird installiert… (kompiliert, einige Minuten)",
|
||||
"settings.cmf.installedOk": "cortiq installiert",
|
||||
"settings.cmf.notInstalled": "nicht installiert",
|
||||
"settings.cmf.addModel": "Modell hinzufügen",
|
||||
"settings.cmf.addModelHelp": "Modelle werden im Tab „Modell importieren“ heruntergeladen und konvertiert (HuggingFace → .cmf). Richte den lokalen Server danach unten auf die .cmf-Datei aus.",
|
||||
"settings.cmf.importNote": "Noch kein Modell? Importiere eines:",
|
||||
"settings.cmf.importLink": "Import öffnen",
|
||||
"settings.cmf.modelHelp": "Pfad zur .cmf-Datei, die bereitgestellt werden soll (z. B. models/qwen.cmf), erzeugt im Import-Tab. Mehrere Modelle für verschiedene Aufgaben? Importiere sie und wechsle diesen Pfad.",
|
||||
"settings.cmf.portHelp": "TCP-Port, auf dem der lokale Modellserver lauscht (Standard 8081). Nur ändern, wenn er mit einem anderen Dienst kollidiert.",
|
||||
"settings.cmf.threadsHelp": "Wie viele CPU-Kerne für die Generierung genutzt werden. Mehr = schneller (~3× gegenüber einem Kern); 0 überlässt die Wahl der Runtime.",
|
||||
"settings.cmf.gpuHelp": "Matrixberechnungen auf die GPU auslagern (Metal/wgpu). Auf Apple Silicon beschleunigt das vor allem lange Prompts — das Decoding ist speicherbandbreitenbegrenzt.",
|
||||
"settings.cmf.localOnlyHelp": "Jede Anfrage an das lokale Modell leiten und dabei Router sowie Cloud-Anbieter ignorieren.",
|
||||
"settings.cmf.manageHelp": "Das Gateway startet beim Hochfahren `cortiq serve` für die untenstehende .cmf-Datei und routet dorthin. Erfordert ein installiertes cortiq (Schritt 1). Greift nach einem Neustart.",
|
||||
"settings.cmf.autoInstallHelp": "Wenn cortiq beim Start fehlt, automatisch von crates.io installieren.",
|
||||
"settings.cmf.autoUpdateHelp": "Beim Start cortiq auf die neueste Version aktualisieren, falls eine neuere existiert.",
|
||||
"settings.router.enabledHelp": "Den cortiq-Router jede Anfrage klassifizieren lassen (Aufgabentyp + Komplexität) und ein Modell wählen. Aus = nur die lokale Routing-Tabelle verwenden.",
|
||||
"settings.router.verifyTlsHelp": "Das TLS-Zertifikat des Routers prüfen. Nur bei einem selbstsignierten oder IP-gebundenen Zertifikat abschalten.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus-Metriken unter GET /metrics bereitstellen.",
|
||||
"settings.cortiq.echoHelp": "Ein Feld `cortiq` (Aufgabe, Modell, Kosten) zu jedem Antwort-Body hinzufügen. Response-Header werden immer gesendet.",
|
||||
"settings.stats.enabledHelp": "Aktuelle Anfragen aufzeichnen, um Dashboard und Diagramme zu speisen.",
|
||||
"settings.cache.enabledHelp": "Semantischer Cache: eine gespeicherte Antwort zurückgeben, wenn ein neuer Prompt einem früheren sehr ähnlich ist, und den Modellaufruf überspringen. Erfordert ein Embedding-Modell im Pool.",
|
||||
"settings.log.bodiesHelp": "Vollständige Nachrichteninhalte von Anfrage und Antwort loggen. Aus Datenschutzgründen standardmäßig aus.",
|
||||
"settings.router.saveKey": "Schlüssel speichern",
|
||||
"settings.router.keySaved": "Router-Schlüssel gespeichert",
|
||||
"settings.router.keyEmpty": "Füge zuerst den Schlüssel ein",
|
||||
"settings.router.secretHelp": "Füge den Schlüssel aus deiner E-Mail von api.allaigate.com ein und klicke auf „Schlüssel speichern“. Das ist KEIN Wert aus „API-Schlüssel“ — die sind für Clients, die dein Gateway aufrufen.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — Schlüssel gespeichert (zum Ersetzen erneut einfügen)",
|
||||
"settings.route.strategyHelp": "Welcher Text zur Klassifizierung an den Router geht: last_user = nur die letzte Nutzernachricht; last_user_plus_system = letzte Nutzernachricht + System-Prompt; concat_all = die gesamte Unterhaltung.",
|
||||
"settings.route.maxCharsHelp": "Maximale Zeichenzahl dieses Textes, die an den Router gesendet wird (längerer Text wird gekürzt). Mehr = genauere Klassifizierung, aber langsamere und teurere Router-Aufrufe.",
|
||||
"settings.route.cacheTtlHelp": "Wie lange eine Routing-Entscheidung für eine identische Anfrage zwischengespeichert wird, bevor der Router erneut gefragt wird (z. B. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Standard-Routing-Profil, wenn eine Anfrage keines angibt: cost-saver (am günstigsten), balanced oder quality-first (bestes Modell). Beeinflusst, welches Modell der Router wählt.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Suche auf HuggingFace, konvertiere in ein lokales .cmf und stelle es bereit.",
|
||||
"import.searchPlaceholder": "HuggingFace-Modelle suchen (z. B. qwen3, llama)…",
|
||||
"import.searching": "Suche läuft…",
|
||||
"import.noResults": "Keine Modelle gefunden.",
|
||||
"import.configure": "Konvertierung einrichten",
|
||||
"import.outName": "Ausgabename",
|
||||
"import.outNameHint": "Name der .cmf-Datei (Buchstaben, Ziffern, - und _).",
|
||||
"import.quant": "Quantisierung",
|
||||
"import.convert": "Konvertieren & laden",
|
||||
"import.converting": "Konvertierung",
|
||||
"import.started": "Konvertierung gestartet",
|
||||
"import.failed": "Konvertierung fehlgeschlagen — siehe Log.",
|
||||
"import.register": "Als lokales Modell registrieren",
|
||||
"import.registered": "Als '{id}' registriert. Starte cortiq-server, um es bereitzustellen.",
|
||||
"import.jobs": "Downloads & Konvertierungen",
|
||||
"import.cancel": "Abbrechen",
|
||||
"import.cancelling": "Wird abgebrochen — unvollständige Ausgabe wird entfernt…",
|
||||
"import.delete": "Löschen",
|
||||
"import.deleteConfirm": "Dieses Modell und seine konvertierte .cmf-Datei von der Festplatte löschen?",
|
||||
"import.deleted": "Mitsamt Dateien gelöscht",
|
||||
"import.state.running": "läuft",
|
||||
"import.state.done": "fertig",
|
||||
"import.state.error": "Fehler",
|
||||
"import.state.cancelled": "abgebrochen",
|
||||
"import.featuredTitle": "Empfohlene fertige .cmf-Modelle",
|
||||
"import.featuredSub": "Vorkonvertierte .cmf-Modelle vom infosave-Konto auf HuggingFace — die Liste wird live gelesen, neue Modelle erscheinen von selbst.",
|
||||
"import.readyCmfBadge": "Fertiges .cmf",
|
||||
"import.directDownload": "Herunterladen",
|
||||
"import.q.q8_2f": "8-Bit Zwei-Feld (𝒲×θ) — empfohlen: bestes Verhältnis von Qualität und Größe.",
|
||||
"import.q.q8_row": "8-Bit pro Zeile — einfach und robust.",
|
||||
"import.q.q4": "4-Bit Block — am kleinsten, geringere Qualität.",
|
||||
"import.q.q4_tiled": "4-Bit gekachelt (Q4_TILED) — 1,66× schneller auf der Metal-GPU.",
|
||||
"import.q.vbit": "Variabel 3–8 Bit — Bit-Budget pro Experte.",
|
||||
"import.q.q1t": "1,58-Bit ternär (Q1T) — BitNet b1.58 mit spärlichen Ausreißern.",
|
||||
"import.q.q1s": "1-Bit spärlich (Q1S) — 1-Bit-Basis mit holografischen Ausreißern.",
|
||||
"import.q.q1p": "1-Bit PTQ (Q1P) — 1 Bit mit Fehlerdiffusion.",
|
||||
"import.q.q1": "1-Bit (Q1) — rein binäres Format.",
|
||||
"import.q.f16": "16-Bit — keine Quantisierung.",
|
||||
"import.q.f32": "32-Bit — volle Präzision (große Datei).",
|
||||
"import.adv.title": "Erweiterte Einstellungen",
|
||||
"import.adv.auto": "automatisch",
|
||||
"import.adv.linearCore": "Linearer Kern",
|
||||
"import.adv.linearHint": "Wie Layer mit linearer Attention übernommen bzw. gefaltet werden.",
|
||||
"import.adv.nphase": "vmf_phase Phasen/Kopf",
|
||||
"import.adv.vbitShape": "VBIT-Budgetform",
|
||||
"import.adv.meanBits": "VBIT mittlere Bits",
|
||||
"import.adv.shard": "Shard max. (GB)",
|
||||
"import.adv.noShard": "kein Sharding",
|
||||
"import.adv.skipMtp": "MTP-Kopf überspringen",
|
||||
"import.adv.o1": "O(1)-Attention",
|
||||
"import.adv.o1Hint": "Streaming-Attention mit konstantem Speicher: der KV-Cache wird nach dem Prefill freigegeben, lange Kontexte lassen den Speicher also nicht wachsen. Die Gewichte bleiben Byte-identisch — gespeichert wird nur ein Hinweis für die Ladezeit; du kannst jederzeit zurückwechseln (mit --o1 off starten). Auf Qwen3-0.6B validiert: alle Layer ≈ ×1,18 Perplexität.",
|
||||
"import.adv.o1Off": "aus — exakte Attention (Standard)",
|
||||
"import.adv.o1All": "alle — jeden Softmax-Layer konvertieren",
|
||||
"import.adv.o1Deep": "deep N — nur die N tiefsten Layer",
|
||||
"import.adv.o1Custom": "benutzerdefiniert — explizite Layer-Liste",
|
||||
"import.adv.o1DeepN": "N tiefe Layer",
|
||||
"import.adv.o1Layers": "Layer (kommagetrennt)",
|
||||
"import.adv.o1mHint": "Landmark-Budget (Standard 32)",
|
||||
"import.adv.o1windowHint": "Breite des exakten Fensters (Standard 128)",
|
||||
"import.adv.o1sinkHint": "permanente exakte Sink-Keys (Standard 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Ersteinrichtung",
|
||||
"onboarding.welcome": "Willkommen bei Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Wähle deine Sprache — du kannst sie jederzeit in der Kopfzeile ändern.",
|
||||
"onboarding.next": "Weiter",
|
||||
"onboarding.back": "Zurück",
|
||||
"onboarding.skip": "Vorerst überspringen",
|
||||
"onboarding.skipSetup": "Überspringen",
|
||||
"onboarding.finish": "Einrichtung abschließen",
|
||||
"onboarding.featuredTitle": "Fertige Modelle (ein Klick)",
|
||||
"onboarding.featuredSub": "Das sind bereits konvertierte .cmf-Modelle von infosave auf HuggingFace — einfach herunterladen, keine Konvertierung nötig. Wähle eines oder mehrere für den Sofortstart.",
|
||||
"onboarding.download": "Herunterladen",
|
||||
"onboarding.hfTitle": "Jedes HuggingFace-Modell — automatisch konvertiert",
|
||||
"onboarding.hfSub": "Suche ein beliebiges Modell, wähle eine Quantisierung, und wir konvertieren es automatisch nach .cmf. Keine manuellen Config-Änderungen.",
|
||||
"onboarding.noJobs": "Noch keine Downloads.",
|
||||
"onboarding.doneTitle": "Alles bereit",
|
||||
"onboarding.doneSub": "Heruntergeladene Modelle werden automatisch registriert. Weitere kannst du unter Import oder Modelle hinzufügen.",
|
||||
"onboarding.doneHint": "„Einrichtung abschließen“ verbindet jeden fertigen Download als lokales Modell, startet dessen .cmf-Server und richtet das Routing darauf aus. Alles lässt sich später unter Modelle ändern.",
|
||||
"onboarding.willRegister": "Werden als lokale Modelle verbunden",
|
||||
"onboarding.nothingToRegister": "Noch keine abgeschlossenen Downloads — nichts zu verbinden. Geh einen Schritt zurück und wähle ein Modell.",
|
||||
"onboarding.registeredN": "{n} Modell(e) verbunden: {ids}",
|
||||
"onboarding.runtimeMissing": "Die cortiq-Runtime ist nicht installiert — installiere sie unter Einstellungen → Lokale Modelle, um diese Modelle bereitzustellen.",
|
||||
"onboarding.downloaded": "heruntergeladen",
|
||||
"onboarding.update": "Aktualisieren",
|
||||
"onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen",
|
||||
"onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.",
|
||||
"onboarding.unregistered": "Ebenfalls abgemeldet: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+52
-13
@@ -132,8 +132,6 @@ export default {
|
||||
"models.form.secretMissing": "not set",
|
||||
"models.form.secretPlaceholder": "set or replace key (write-only)",
|
||||
"models.deleteConfirm": "Delete model \"{id}\"?",
|
||||
"models.deleteFile": "also delete the model file from disk",
|
||||
"models.deletedWithFile": "Deleted (file removed from disk)",
|
||||
"models.form.idOrModel": "Enter an id or a model name",
|
||||
"models.form.apiKey": "API Key",
|
||||
"models.form.apiKeyPlaceholder": "paste your API key",
|
||||
@@ -141,6 +139,18 @@ export default {
|
||||
"models.form.apiKeyHint": "Stored securely on the gateway; it is not shown again.",
|
||||
"models.form.keyStored": "A key is saved — leave blank to keep it, or paste a new one.",
|
||||
"models.form.keyFromEnv": "Using a key from an environment variable.",
|
||||
"models.form.paramsTitle": "Generation & Thinking Parameters (O(1), MTP, Temperature)",
|
||||
"models.form.temperature": "Temperature",
|
||||
"models.form.temperatureHint": "Generation temperature (0.0 = deterministic, 1.0+ = creative).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling threshold (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max Tokens",
|
||||
"models.form.maxTokensHint": "Maximum completion tokens to generate.",
|
||||
"models.form.thinkBudget": "Thinking Mode / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Reasoning token budget (0 = disabled, N > 0 = max reasoning tokens).",
|
||||
"models.form.systemPrompt": "Default System Prompt",
|
||||
"models.form.systemPromptHint": "Prepend automatically if request lacks a system prompt.",
|
||||
"models.form.systemPromptPlaceholder": "You are a helpful assistant...",
|
||||
"models.form.testKey": "Test & load models",
|
||||
"models.form.testing": "checking…",
|
||||
"models.form.foundModels": "{n} models loaded",
|
||||
@@ -264,14 +274,11 @@ export default {
|
||||
"settings.cmf.pick": "select a model",
|
||||
"settings.cmf.models": "Local models",
|
||||
"settings.cmf.modelsHelp": "Each row is one model the gateway runs (its own cortiq serve on its own port) and registers in the pool. Add as many as you have .cmf files and free ports.",
|
||||
"settings.cmf.rowHint": "id · .cmf file · port · threads · GPU · O(1). Applied on save: a removed row stops its server (memory freed), a new row starts one.",
|
||||
"settings.cmf.o1Auto": "O(1): auto",
|
||||
"settings.cmf.o1Help": "O(1) attention at serve time: auto — follow the hint recorded in the .cmf; off — force exact attention (fixes incoherent output after a too-aggressive --o1 conversion, no re-convert needed); all/deepN — force-enable on those layers.",
|
||||
"settings.cmf.rowHint": "id · .cmf file · port · threads · GPU. Changes apply on restart; deleting a row stops that server.",
|
||||
"settings.cmf.addServer": "+ Add local model",
|
||||
"settings.cmf.checkPort": "Check port",
|
||||
"settings.cmf.portFree": "port is free",
|
||||
"settings.cmf.portBusy": "port is in use",
|
||||
"settings.cmf.portOwn": "in use by this gateway's current server — freed on restart, replacing is fine",
|
||||
"settings.cmf.useSuggested": "use {port}",
|
||||
"settings.cmf.serving": "serving {model} on :{port}",
|
||||
"settings.cmf.intro": "Run models locally with the cortiq engine — works even offline or when the router is down.",
|
||||
@@ -379,7 +386,7 @@ export default {
|
||||
"import.failed": "Conversion failed — see the log.",
|
||||
"import.register": "Register as local model",
|
||||
"import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
|
||||
"import.jobs": "Conversions",
|
||||
"import.jobs": "Downloads & Conversions",
|
||||
"import.cancel": "Cancel",
|
||||
"import.cancelling": "Cancelling — cleaning up partial output…",
|
||||
"import.delete": "Delete",
|
||||
@@ -389,12 +396,22 @@ export default {
|
||||
"import.state.done": "done",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelled",
|
||||
"import.featuredTitle": "Featured Ready-to-Run .cmf Models",
|
||||
"import.featuredSub": "Pre-converted .cmf models from the infosave account on HuggingFace — the list is read live, so newly published models appear here on their own.",
|
||||
"import.readyCmfBadge": "Ready .cmf",
|
||||
"import.directDownload": "Download",
|
||||
"import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
|
||||
"import.q.q8_row": "8-bit per-row — simple, robust.",
|
||||
"import.q.q4": "4-bit block — smallest, lower quality.",
|
||||
"import.q.q4_tiled": "4-bit tiled (Q4_TILED) — 1.66x faster on Metal GPU.",
|
||||
"import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
|
||||
"import.q.q1t": "1.58-bit ternary (Q1T) — BitNet b1.58 with sparse outliers.",
|
||||
"import.q.q1s": "1-bit sparse (Q1S) — 1-bit base with holographic outliers.",
|
||||
"import.q.q1p": "1-bit PTQ (Q1P) — 1-bit with error diffusion.",
|
||||
"import.q.q1": "1-bit (Q1) — pure binary format.",
|
||||
"import.q.f16": "16-bit — no quantization.",
|
||||
"import.q.f32": "32-bit — full precision (large file).",
|
||||
|
||||
"import.adv.title": "Advanced settings",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Linear core",
|
||||
@@ -414,12 +431,34 @@ export default {
|
||||
"import.adv.o1DeepN": "N deep layers",
|
||||
"import.adv.o1Layers": "Layers (comma-separated)",
|
||||
"import.adv.o1mHint": "landmark budget (default 32)",
|
||||
"import.adv.o1mLabel": "Landmarks (--o1-m)",
|
||||
"import.adv.o1mHelp": "How many landmark tokens summarize the whole older history when attention is approximated. More = closer to exact attention but slower and more memory; fewer = faster but rougher. 32 is the validated sweet spot — change only if you know why.",
|
||||
"import.adv.o1windowLabel": "Exact window (--o1-window)",
|
||||
"import.adv.o1windowHelp": "The last N tokens always get exact (non-approximated) attention. This protects quality on the text the model just wrote/read. Bigger = better local quality, more memory. Default 128.",
|
||||
"import.adv.o1sinkLabel": "Sink keys (--o1-sink)",
|
||||
"import.adv.o1sinkHelp": "The first N tokens of the context are kept exact forever ('attention sinks'). Models lean on these anchors — keeping them exact stabilizes long generations. Default 4; rarely needs changing.",
|
||||
"import.adv.o1windowHint": "exact-window width (default 128)",
|
||||
"import.adv.o1sinkHint": "permanent exact sink keys (default 4)",
|
||||
|
||||
// onboarding (first run)
|
||||
"onboarding.title": "Welcome setup",
|
||||
"onboarding.welcome": "Welcome to Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choose your language — you can change it anytime in the header.",
|
||||
"onboarding.next": "Next",
|
||||
"onboarding.back": "Back",
|
||||
"onboarding.skip": "Skip for now",
|
||||
"onboarding.skipSetup": "Skip",
|
||||
"onboarding.finish": "Finish setup",
|
||||
"onboarding.featuredTitle": "Ready-made models (one click)",
|
||||
"onboarding.featuredSub": "These are pre-converted .cmf models from infosave on HuggingFace — just Download, no conversion needed. Pick one or more to get started instantly.",
|
||||
"onboarding.download": "Download",
|
||||
"onboarding.hfTitle": "Any HuggingFace model — auto convert",
|
||||
"onboarding.hfSub": "Search any model, pick a quantization, and we convert it to .cmf automatically. No manual config edits.",
|
||||
"onboarding.noJobs": "No downloads yet.",
|
||||
"onboarding.doneTitle": "You're all set",
|
||||
"onboarding.doneSub": "Models you downloaded will be registered automatically. You can add more in Import or Models.",
|
||||
"onboarding.doneHint": "Finish connects every completed download as a local model, starts its .cmf server and points routing at it. You can change all of it later in Models.",
|
||||
"onboarding.willRegister": "Will be connected as local models",
|
||||
"onboarding.nothingToRegister": "No finished downloads yet — nothing to connect. Go back a step to pick a model.",
|
||||
"onboarding.registeredN": "Connected {n} model(s): {ids}",
|
||||
"onboarding.runtimeMissing": "The cortiq runtime isn't installed — install it in Settings → Local models to serve these.",
|
||||
"onboarding.downloaded": "downloaded",
|
||||
"onboarding.update": "Update",
|
||||
"onboarding.deleteFile": "Delete the .cmf file from disk",
|
||||
"onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.",
|
||||
"onboarding.unregistered": "Also unregistered: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+191
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Enrutamiento",
|
||||
"nav.protocols": "Protocolos",
|
||||
"nav.keys": "Claves API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Zona de pruebas",
|
||||
"nav.settings": "Configuración",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clave del router y pagos",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "por tarea",
|
||||
"dash.groupby.account": "por cuenta",
|
||||
"dash.recent.title": "Solicitudes recientes",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba el Playground.",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba la Zona de pruebas.",
|
||||
"dash.col.time": "Hora",
|
||||
"dash.col.model": "Modelo",
|
||||
"dash.col.tier": "Nivel",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "todos",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Zona de pruebas",
|
||||
"pg.subtitle": "Envía un prompt por el pipeline en vivo e inspecciona la decisión de enrutamiento.",
|
||||
"pg.system": "Prompt del sistema",
|
||||
"pg.systemPlaceholder": "mensaje de sistema opcional",
|
||||
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "TTL de entrada",
|
||||
"settings.cache.embed": "Modelo de embeddings",
|
||||
"settings.cache.note": "Los ajustes del caché se aplican tras reiniciar.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importar modelo",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Borrar registros",
|
||||
"dash.clearConfirm": "¿Borrar todas las estadísticas y registros guardados? Esta acción no se puede deshacer.",
|
||||
"dash.cleared": "Registros borrados",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Introduce un id o un nombre de modelo",
|
||||
"models.form.apiKey": "Clave API",
|
||||
"models.form.apiKeyPlaceholder": "pega tu clave API",
|
||||
"models.form.apiKeyNotNeeded": "no necesaria (servidor local)",
|
||||
"models.form.apiKeyHint": "Se guarda de forma segura en la pasarela; no se vuelve a mostrar.",
|
||||
"models.form.keyStored": "Hay una clave guardada — déjalo vacío para conservarla o pega una nueva.",
|
||||
"models.form.keyFromEnv": "Se está usando una clave de una variable de entorno.",
|
||||
"models.form.paramsTitle": "Parámetros de generación y razonamiento (O(1), MTP, temperatura)",
|
||||
"models.form.temperature": "Temperatura",
|
||||
"models.form.temperatureHint": "Temperatura de generación (0.0 = determinista, 1.0+ = creativa).",
|
||||
"models.form.topP": "Top-P (muestreo nucleus)",
|
||||
"models.form.topPHint": "Umbral de muestreo (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens máx.",
|
||||
"models.form.maxTokensHint": "Número máximo de tokens generados.",
|
||||
"models.form.thinkBudget": "Modo razonamiento / presupuesto de pensamiento",
|
||||
"models.form.thinkBudgetHint": "Presupuesto de tokens de razonamiento (0 = desactivado, N > 0 = máximo de tokens de razonamiento).",
|
||||
"models.form.systemPrompt": "Prompt de sistema por defecto",
|
||||
"models.form.systemPromptHint": "Se antepone automáticamente si la petición no trae prompt de sistema.",
|
||||
"models.form.systemPromptPlaceholder": "Eres un asistente útil…",
|
||||
"models.form.testKey": "Probar y cargar modelos",
|
||||
"models.form.testing": "comprobando…",
|
||||
"models.form.foundModels": "{n} modelos cargados",
|
||||
"models.form.modelPlaceholder": "escribe para buscar…",
|
||||
"models.form.modelHint": "Pulsa «Probar y cargar modelos» para traer la lista con tu clave y luego elige uno.",
|
||||
"models.managed": "gestionado",
|
||||
"models.managedHint": "Lo proporciona el servidor CMF gestionado (Ajustes → Modelos locales). Configúralo allí.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Usar el router (desactivado = solo modelos locales)",
|
||||
"settings.cmf": "Modelos locales (CMF)",
|
||||
"settings.cmf.localOnly": "Usar solo modelos locales",
|
||||
"settings.cmf.manage": "Ejecutar un servidor de modelo local",
|
||||
"settings.cmf.autoInstall": "Instalar cortiq automáticamente (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Actualizar cortiq automáticamente",
|
||||
"settings.cmf.model": "Ruta del modelo .cmf local",
|
||||
"settings.cmf.port": "Puerto del servidor local",
|
||||
"settings.cmf.modelId": "Id del modelo en el pool",
|
||||
"settings.cmf.note": "Sirve un .cmf local mediante cortiq (github.com/infosave2007/cmf). Funciona incluso cuando el router no está disponible.",
|
||||
"settings.cmf.threads": "Hilos de trabajo",
|
||||
"settings.cmf.gpu": "Usar GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Los hilos paralelizan los matvec de decodificación (~3× más rápido; 0 = valor por defecto del runtime). Descargar en GPU ayuda sobre todo al prefill de prompts largos — la decodificación está limitada por el ancho de banda de memoria, así que en Apple Silicon es prácticamente neutro.",
|
||||
"settings.cmf.pick": "selecciona un modelo",
|
||||
"settings.cmf.models": "Modelos locales",
|
||||
"settings.cmf.modelsHelp": "Cada fila es un modelo que la pasarela ejecuta (su propio cortiq serve en su propio puerto) y registra en el pool. Añade tantos como archivos .cmf y puertos libres tengas.",
|
||||
"settings.cmf.rowHint": "id · archivo .cmf · puerto · hilos · GPU. Los cambios se aplican al reiniciar; borrar una fila detiene ese servidor.",
|
||||
"settings.cmf.addServer": "+ Añadir modelo local",
|
||||
"settings.cmf.checkPort": "Comprobar puerto",
|
||||
"settings.cmf.portFree": "puerto libre",
|
||||
"settings.cmf.portBusy": "puerto ocupado",
|
||||
"settings.cmf.useSuggested": "usar {port}",
|
||||
"settings.cmf.serving": "sirviendo {model} en :{port}",
|
||||
"settings.cmf.intro": "Ejecuta modelos en local con el motor cortiq — funciona incluso sin conexión o cuando el router está caído.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "La herramienta de línea de comandos `cortiq` que carga y ejecuta modelos .cmf. Instálala una vez antes de arrancar un servidor local.",
|
||||
"settings.cmf.install": "Instalar cortiq",
|
||||
"settings.cmf.reinstall": "Reinstalar / actualizar",
|
||||
"settings.cmf.installing": "instalando… (compila, unos minutos)",
|
||||
"settings.cmf.installedOk": "cortiq instalado",
|
||||
"settings.cmf.notInstalled": "no instalado",
|
||||
"settings.cmf.addModel": "Añadir un modelo",
|
||||
"settings.cmf.addModelHelp": "Los modelos se descargan y convierten en la pestaña Importar (HuggingFace → .cmf). Después apunta el servidor local al archivo .cmf de abajo.",
|
||||
"settings.cmf.importNote": "¿Aún no tienes modelo? Importa uno:",
|
||||
"settings.cmf.importLink": "Abrir Importar",
|
||||
"settings.cmf.modelHelp": "Ruta del archivo .cmf que se va a servir (p. ej. models/qwen.cmf), generado en la pestaña Importar. ¿Varios modelos para distintas tareas? Impórtalos y cambia esta ruta.",
|
||||
"settings.cmf.portHelp": "Puerto TCP en el que escucha el servidor de modelo local (8081 por defecto). Cámbialo solo si choca con otro servicio.",
|
||||
"settings.cmf.threadsHelp": "Cuántos núcleos de CPU usar para la generación. Más = más rápido (~3× frente a un núcleo); 0 deja que decida el runtime.",
|
||||
"settings.cmf.gpuHelp": "Descargar el cálculo matricial en la GPU (Metal/wgpu). En Apple Silicon acelera sobre todo los prompts largos — la decodificación está limitada por el ancho de banda de memoria.",
|
||||
"settings.cmf.localOnlyHelp": "Enviar cada petición al modelo local, ignorando el router y cualquier proveedor en la nube.",
|
||||
"settings.cmf.manageHelp": "La pasarela arranca `cortiq serve` para el .cmf de abajo al iniciarse y enruta hacia él. Requiere cortiq instalado (paso 1). Se aplica al reiniciar.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq falta al arrancar, instalarlo automáticamente desde crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Al arrancar, actualizar cortiq a la versión más reciente si existe una más nueva.",
|
||||
"settings.router.enabledHelp": "Dejar que el router de cortiq clasifique cada petición (tipo de tarea + complejidad) y elija un modelo. Desactivado = usar solo la tabla de enrutado local.",
|
||||
"settings.router.verifyTlsHelp": "Verificar el certificado TLS del router. Desactívalo solo con un certificado autofirmado o ligado a una IP.",
|
||||
"settings.telemetry.metricsHelp": "Exponer métricas Prometheus en GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Añadir un campo `cortiq` (tarea, modelo, coste) al cuerpo de cada respuesta. Las cabeceras de respuesta se envían siempre.",
|
||||
"settings.stats.enabledHelp": "Registrar las peticiones recientes para alimentar el panel y las gráficas.",
|
||||
"settings.cache.enabledHelp": "Caché semántica: devolver una respuesta guardada cuando un prompt nuevo se parece mucho a uno anterior, evitando la llamada al modelo. Requiere un modelo de embeddings en el pool.",
|
||||
"settings.log.bodiesHelp": "Registrar el cuerpo completo de los mensajes de petición y respuesta. Desactivado por defecto por privacidad.",
|
||||
"settings.router.saveKey": "Guardar clave",
|
||||
"settings.router.keySaved": "Clave del router guardada",
|
||||
"settings.router.keyEmpty": "Pega antes la clave",
|
||||
"settings.router.secretHelp": "Pega la clave del correo de api.allaigate.com y pulsa Guardar clave. NO es un valor de «Claves API» — esas son para los clientes que llaman a tu pasarela.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clave guardada (pega otra para reemplazarla)",
|
||||
"settings.route.strategyHelp": "Qué texto se envía al router para clasificar: last_user = solo el último mensaje del usuario; last_user_plus_system = último mensaje del usuario + prompt de sistema; concat_all = toda la conversación.",
|
||||
"settings.route.maxCharsHelp": "Máximo de caracteres de ese texto enviados al router (lo que exceda se recorta). Más = clasificación más precisa, pero llamadas al router más lentas y caras.",
|
||||
"settings.route.cacheTtlHelp": "Cuánto tiempo se cachea una decisión de enrutado para una petición idéntica antes de volver a preguntar al router (p. ej. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Perfil de enrutado por defecto cuando la petición no indica ninguno: cost-saver (el más barato), balanced o quality-first (el mejor modelo). Influye en qué modelo elige el router.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Busca en HuggingFace, convierte a un .cmf local y sírvelo.",
|
||||
"import.searchPlaceholder": "Buscar modelos en HuggingFace (p. ej. qwen3, llama)…",
|
||||
"import.searching": "Buscando…",
|
||||
"import.noResults": "No se encontraron modelos.",
|
||||
"import.configure": "Configurar la conversión",
|
||||
"import.outName": "Nombre de salida",
|
||||
"import.outNameHint": "Nombre del archivo .cmf (letras, dígitos, - y _).",
|
||||
"import.quant": "Cuantización",
|
||||
"import.convert": "Convertir y cargar",
|
||||
"import.converting": "Convirtiendo",
|
||||
"import.started": "Conversión iniciada",
|
||||
"import.failed": "La conversión falló — revisa el registro.",
|
||||
"import.register": "Registrar como modelo local",
|
||||
"import.registered": "Registrado como «{id}». Ejecuta cortiq-server para servirlo.",
|
||||
"import.jobs": "Descargas y conversiones",
|
||||
"import.cancel": "Cancelar",
|
||||
"import.cancelling": "Cancelando — limpiando el archivo incompleto…",
|
||||
"import.delete": "Eliminar",
|
||||
"import.deleteConfirm": "¿Eliminar este modelo y su archivo .cmf convertido del disco?",
|
||||
"import.deleted": "Eliminado junto con sus archivos",
|
||||
"import.state.running": "en curso",
|
||||
"import.state.done": "listo",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelado",
|
||||
"import.featuredTitle": "Modelos .cmf listos para usar",
|
||||
"import.featuredSub": "Modelos .cmf ya convertidos de la cuenta infosave en HuggingFace — la lista se lee en vivo, los modelos nuevos aparecen solos.",
|
||||
"import.readyCmfBadge": ".cmf listo",
|
||||
"import.directDownload": "Descargar",
|
||||
"import.q.q8_2f": "8 bits de dos campos (𝒲×θ) — recomendado: mejor relación calidad/tamaño.",
|
||||
"import.q.q8_row": "8 bits por fila — simple y robusto.",
|
||||
"import.q.q4": "4 bits por bloque — el más pequeño, menor calidad.",
|
||||
"import.q.q4_tiled": "4 bits en mosaico (Q4_TILED) — 1,66× más rápido en GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — presupuesto de bits por experto.",
|
||||
"import.q.q1t": "Ternario de 1,58 bits (Q1T) — BitNet b1.58 con valores atípicos dispersos.",
|
||||
"import.q.q1s": "1 bit disperso (Q1S) — base de 1 bit con valores atípicos holográficos.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit con difusión de error.",
|
||||
"import.q.q1": "1 bit (Q1) — formato puramente binario.",
|
||||
"import.q.f16": "16 bits — sin cuantización.",
|
||||
"import.q.f32": "32 bits — precisión completa (archivo grande).",
|
||||
"import.adv.title": "Ajustes avanzados",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Núcleo lineal",
|
||||
"import.adv.linearHint": "Cómo se trasladan o pliegan las capas de atención lineal.",
|
||||
"import.adv.nphase": "vmf_phase fases/cabeza",
|
||||
"import.adv.vbitShape": "Forma del presupuesto VBIT",
|
||||
"import.adv.meanBits": "Bits medios VBIT",
|
||||
"import.adv.shard": "Tamaño máx. de shard (GB)",
|
||||
"import.adv.noShard": "sin fragmentar",
|
||||
"import.adv.skipMtp": "omitir la cabeza MTP",
|
||||
"import.adv.o1": "Atención O(1)",
|
||||
"import.adv.o1Hint": "Atención en streaming con memoria constante: la caché KV se libera tras el prefill, así que los contextos largos no hacen crecer la memoria. Los pesos quedan idénticos byte a byte — solo se guarda una indicación para el momento de carga; puedes volver atrás cuando quieras (servir con --o1 off). Validado en Qwen3-0.6B: todas las capas ≈ ×1,18 de perplejidad.",
|
||||
"import.adv.o1Off": "desactivada — atención exacta (por defecto)",
|
||||
"import.adv.o1All": "all — convertir todas las capas softmax",
|
||||
"import.adv.o1Deep": "deep N — solo las N capas más profundas",
|
||||
"import.adv.o1Custom": "personalizado — lista explícita de capas",
|
||||
"import.adv.o1DeepN": "N capas profundas",
|
||||
"import.adv.o1Layers": "Capas (separadas por comas)",
|
||||
"import.adv.o1mHint": "presupuesto de landmarks (32 por defecto)",
|
||||
"import.adv.o1windowHint": "ancho de la ventana exacta (128 por defecto)",
|
||||
"import.adv.o1sinkHint": "claves sink exactas permanentes (4 por defecto)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuración inicial",
|
||||
"onboarding.welcome": "Bienvenido a Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Elige tu idioma — puedes cambiarlo cuando quieras en la cabecera.",
|
||||
"onboarding.next": "Siguiente",
|
||||
"onboarding.back": "Atrás",
|
||||
"onboarding.skip": "Omitir por ahora",
|
||||
"onboarding.skipSetup": "Omitir",
|
||||
"onboarding.finish": "Finalizar configuración",
|
||||
"onboarding.featuredTitle": "Modelos listos para usar (un clic)",
|
||||
"onboarding.featuredSub": "Son modelos .cmf ya convertidos de infosave en HuggingFace — basta con descargarlos, sin conversión. Elige uno o varios para empezar al instante.",
|
||||
"onboarding.download": "Descargar",
|
||||
"onboarding.hfTitle": "Cualquier modelo de HuggingFace — conversión automática",
|
||||
"onboarding.hfSub": "Busca cualquier modelo, elige una cuantización y lo convertimos a .cmf automáticamente. Sin editar la configuración a mano.",
|
||||
"onboarding.noJobs": "Todavía no hay descargas.",
|
||||
"onboarding.doneTitle": "Todo listo",
|
||||
"onboarding.doneSub": "Los modelos descargados se registrarán automáticamente. Puedes añadir más en Importar o Modelos.",
|
||||
"onboarding.doneHint": "«Finalizar configuración» conecta cada descarga completada como modelo local, arranca su servidor .cmf y apunta el enrutado hacia él. Todo esto se puede cambiar luego en Modelos.",
|
||||
"onboarding.willRegister": "Se conectarán como modelos locales",
|
||||
"onboarding.nothingToRegister": "Aún no hay descargas terminadas — no hay nada que conectar. Vuelve un paso atrás y elige un modelo.",
|
||||
"onboarding.registeredN": "{n} modelo(s) conectado(s): {ids}",
|
||||
"onboarding.runtimeMissing": "El runtime cortiq no está instalado — instálalo en Ajustes → Modelos locales para poder servirlos.",
|
||||
"onboarding.downloaded": "descargado",
|
||||
"onboarding.update": "Actualizar",
|
||||
"onboarding.deleteFile": "Eliminar el archivo .cmf del disco",
|
||||
"onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.",
|
||||
"onboarding.unregistered": "También se dio de baja: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+191
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Routage",
|
||||
"nav.protocols": "Protocoles",
|
||||
"nav.keys": "Clés API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Bac à sable",
|
||||
"nav.settings": "Paramètres",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clé routeur & facturation",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "par tâche",
|
||||
"dash.groupby.account": "par compte",
|
||||
"dash.recent.title": "Requêtes récentes",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Playground.",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Bac à sable.",
|
||||
"dash.col.time": "Heure",
|
||||
"dash.col.model": "Modèle",
|
||||
"dash.col.tier": "Niveau",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "tous",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Bac à sable",
|
||||
"pg.subtitle": "Envoyez une invite via le pipeline en direct et inspectez la décision de routage.",
|
||||
"pg.system": "Invite système",
|
||||
"pg.systemPlaceholder": "message système optionnel",
|
||||
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "TTL d'entrée",
|
||||
"settings.cache.embed": "Modèle d'embeddings",
|
||||
"settings.cache.note": "Les paramètres du cache prennent effet après un redémarrage.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importer un modèle",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Effacer les journaux",
|
||||
"dash.clearConfirm": "Effacer toutes les statistiques et les journaux enregistrés ? Action irréversible.",
|
||||
"dash.cleared": "Journaux effacés",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Saisissez un identifiant ou un nom de modèle",
|
||||
"models.form.apiKey": "Clé API",
|
||||
"models.form.apiKeyPlaceholder": "collez votre clé API",
|
||||
"models.form.apiKeyNotNeeded": "non requise (serveur local)",
|
||||
"models.form.apiKeyHint": "Stockée de façon sécurisée sur la passerelle ; elle n'est plus affichée ensuite.",
|
||||
"models.form.keyStored": "Une clé est enregistrée — laissez vide pour la conserver, ou collez-en une nouvelle.",
|
||||
"models.form.keyFromEnv": "Une clé issue d'une variable d'environnement est utilisée.",
|
||||
"models.form.paramsTitle": "Paramètres de génération et de raisonnement (O(1), MTP, température)",
|
||||
"models.form.temperature": "Température",
|
||||
"models.form.temperatureHint": "Température de génération (0.0 = déterministe, 1.0+ = créatif).",
|
||||
"models.form.topP": "Top-P (échantillonnage nucleus)",
|
||||
"models.form.topPHint": "Seuil d'échantillonnage (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens max.",
|
||||
"models.form.maxTokensHint": "Nombre maximal de tokens générés.",
|
||||
"models.form.thinkBudget": "Mode raisonnement / budget de réflexion",
|
||||
"models.form.thinkBudgetHint": "Budget de tokens de raisonnement (0 = désactivé, N > 0 = maximum de tokens de raisonnement).",
|
||||
"models.form.systemPrompt": "Prompt système par défaut",
|
||||
"models.form.systemPromptHint": "Ajouté automatiquement si la requête ne contient pas de prompt système.",
|
||||
"models.form.systemPromptPlaceholder": "Vous êtes un assistant utile…",
|
||||
"models.form.testKey": "Tester et charger les modèles",
|
||||
"models.form.testing": "vérification…",
|
||||
"models.form.foundModels": "{n} modèles chargés",
|
||||
"models.form.modelPlaceholder": "tapez pour rechercher…",
|
||||
"models.form.modelHint": "Cliquez sur « Tester et charger les modèles » pour récupérer la liste liée à votre clé, puis choisissez-en un.",
|
||||
"models.managed": "géré",
|
||||
"models.managedHint": "Fourni par le serveur CMF géré (Paramètres → Modèles locaux). Configurez-le là-bas.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Utiliser le routeur (désactivé = modèles locaux uniquement)",
|
||||
"settings.cmf": "Modèles locaux (CMF)",
|
||||
"settings.cmf.localOnly": "Utiliser uniquement les modèles locaux",
|
||||
"settings.cmf.manage": "Lancer un serveur de modèle local",
|
||||
"settings.cmf.autoInstall": "Installer cortiq automatiquement (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Mettre cortiq à jour automatiquement",
|
||||
"settings.cmf.model": "Chemin du modèle .cmf local",
|
||||
"settings.cmf.port": "Port du serveur local",
|
||||
"settings.cmf.modelId": "Identifiant du modèle dans le pool",
|
||||
"settings.cmf.note": "Sert un .cmf local via cortiq (github.com/infosave2007/cmf). Fonctionne même quand le routeur est indisponible.",
|
||||
"settings.cmf.threads": "Threads de travail",
|
||||
"settings.cmf.gpu": "Utiliser le GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Les threads parallélisent les matvecs de décodage (~3× plus rapide ; 0 = valeur par défaut du runtime). Le déport GPU aide surtout au prefill des longs prompts — le décodage est limité par la bande passante mémoire, donc sur Apple Silicon l'effet est à peu près neutre.",
|
||||
"settings.cmf.pick": "choisir un modèle",
|
||||
"settings.cmf.models": "Modèles locaux",
|
||||
"settings.cmf.modelsHelp": "Chaque ligne est un modèle que la passerelle exécute (son propre cortiq serve sur son propre port) et enregistre dans le pool. Ajoutez-en autant que vous avez de fichiers .cmf et de ports libres.",
|
||||
"settings.cmf.rowHint": "id · fichier .cmf · port · threads · GPU. Les changements prennent effet au redémarrage ; supprimer une ligne arrête ce serveur.",
|
||||
"settings.cmf.addServer": "+ Ajouter un modèle local",
|
||||
"settings.cmf.checkPort": "Vérifier le port",
|
||||
"settings.cmf.portFree": "port libre",
|
||||
"settings.cmf.portBusy": "port occupé",
|
||||
"settings.cmf.useSuggested": "utiliser {port}",
|
||||
"settings.cmf.serving": "{model} servi sur :{port}",
|
||||
"settings.cmf.intro": "Exécutez des modèles en local avec le moteur cortiq — fonctionne même hors ligne ou quand le routeur est en panne.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "L'outil en ligne de commande `cortiq` qui charge et exécute les modèles .cmf. Installez-le une fois avant de démarrer un serveur local.",
|
||||
"settings.cmf.install": "Installer cortiq",
|
||||
"settings.cmf.reinstall": "Réinstaller / mettre à jour",
|
||||
"settings.cmf.installing": "installation… (compilation, quelques minutes)",
|
||||
"settings.cmf.installedOk": "cortiq installé",
|
||||
"settings.cmf.notInstalled": "non installé",
|
||||
"settings.cmf.addModel": "Ajouter un modèle",
|
||||
"settings.cmf.addModelHelp": "Les modèles se téléchargent et se convertissent dans l'onglet Importer (HuggingFace → .cmf). Faites ensuite pointer le serveur local vers le fichier .cmf ci-dessous.",
|
||||
"settings.cmf.importNote": "Pas encore de modèle ? Importez-en un :",
|
||||
"settings.cmf.importLink": "Ouvrir Importer",
|
||||
"settings.cmf.modelHelp": "Chemin du fichier .cmf à servir (par ex. models/qwen.cmf), produit par l'onglet Importer. Plusieurs modèles pour des tâches différentes ? Importez-les et changez ce chemin.",
|
||||
"settings.cmf.portHelp": "Port TCP sur lequel écoute le serveur de modèle local (8081 par défaut). À changer seulement en cas de conflit avec un autre service.",
|
||||
"settings.cmf.threadsHelp": "Nombre de cœurs CPU utilisés pour la génération. Plus = plus rapide (~3× par rapport à un seul cœur) ; 0 laisse le runtime décider.",
|
||||
"settings.cmf.gpuHelp": "Déporter les calculs matriciels sur le GPU (Metal/wgpu). Sur Apple Silicon, cela accélère surtout les longs prompts — le décodage est limité par la bande passante mémoire.",
|
||||
"settings.cmf.localOnlyHelp": "Router chaque requête vers le modèle local, en ignorant le routeur et tous les fournisseurs cloud.",
|
||||
"settings.cmf.manageHelp": "Au démarrage, la passerelle lance `cortiq serve` pour le .cmf ci-dessous et y route les requêtes. Nécessite cortiq installé (étape 1). Prend effet au redémarrage.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq est absent au démarrage, l'installer automatiquement depuis crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Au démarrage, mettre cortiq à niveau vers la dernière version si elle est plus récente.",
|
||||
"settings.router.enabledHelp": "Laisser le routeur cortiq classer chaque requête (type de tâche + complexité) et choisir un modèle. Désactivé = utiliser uniquement la table de routage locale.",
|
||||
"settings.router.verifyTlsHelp": "Vérifier le certificat TLS du routeur. À désactiver uniquement pour un certificat auto-signé ou lié à une IP.",
|
||||
"settings.telemetry.metricsHelp": "Exposer les métriques Prometheus sur GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Ajouter un champ `cortiq` (tâche, modèle, coût) au corps de chaque réponse. Les en-têtes de réponse sont toujours envoyés.",
|
||||
"settings.stats.enabledHelp": "Enregistrer les requêtes récentes pour alimenter le tableau de bord et les graphiques.",
|
||||
"settings.cache.enabledHelp": "Cache sémantique : renvoyer une réponse enregistrée quand un nouveau prompt ressemble beaucoup à un précédent, en évitant l'appel au modèle. Nécessite un modèle d'embedding dans le pool.",
|
||||
"settings.log.bodiesHelp": "Journaliser le corps complet des messages de requête et de réponse. Désactivé par défaut pour la confidentialité.",
|
||||
"settings.router.saveKey": "Enregistrer la clé",
|
||||
"settings.router.keySaved": "Clé du routeur enregistrée",
|
||||
"settings.router.keyEmpty": "Collez d'abord la clé",
|
||||
"settings.router.secretHelp": "Collez la clé reçue par e-mail depuis api.allaigate.com, puis cliquez sur Enregistrer la clé. Ce n'est PAS une valeur de « Clés API » — celles-ci servent aux clients qui appellent votre passerelle.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clé enregistrée (recollez-en une pour la remplacer)",
|
||||
"settings.route.strategyHelp": "Texte envoyé au routeur pour la classification : last_user = uniquement le dernier message utilisateur ; last_user_plus_system = dernier message utilisateur + prompt système ; concat_all = toute la conversation.",
|
||||
"settings.route.maxCharsHelp": "Nombre maximal de caractères de ce texte envoyés au routeur (au-delà, il est tronqué). Plus = classification plus précise, mais appels au routeur plus lents et plus coûteux.",
|
||||
"settings.route.cacheTtlHelp": "Durée de mise en cache d'une décision de routage pour une requête identique avant de réinterroger le routeur (par ex. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Profil de routage par défaut quand la requête n'en précise aucun : cost-saver (le moins cher), balanced ou quality-first (meilleur modèle). Influence le modèle choisi par le routeur.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Cherchez sur HuggingFace, convertissez en .cmf local et servez-le.",
|
||||
"import.searchPlaceholder": "Rechercher des modèles HuggingFace (par ex. qwen3, llama)…",
|
||||
"import.searching": "Recherche…",
|
||||
"import.noResults": "Aucun modèle trouvé.",
|
||||
"import.configure": "Configurer la conversion",
|
||||
"import.outName": "Nom de sortie",
|
||||
"import.outNameHint": "Nom du fichier .cmf (lettres, chiffres, - et _).",
|
||||
"import.quant": "Quantification",
|
||||
"import.convert": "Convertir et charger",
|
||||
"import.converting": "Conversion",
|
||||
"import.started": "Conversion lancée",
|
||||
"import.failed": "Échec de la conversion — voir le journal.",
|
||||
"import.register": "Enregistrer comme modèle local",
|
||||
"import.registered": "Enregistré sous « {id} ». Lancez cortiq-server pour le servir.",
|
||||
"import.jobs": "Téléchargements et conversions",
|
||||
"import.cancel": "Annuler",
|
||||
"import.cancelling": "Annulation — nettoyage du fichier incomplet…",
|
||||
"import.delete": "Supprimer",
|
||||
"import.deleteConfirm": "Supprimer ce modèle et son fichier .cmf converti du disque ?",
|
||||
"import.deleted": "Supprimé avec ses fichiers",
|
||||
"import.state.running": "en cours",
|
||||
"import.state.done": "terminé",
|
||||
"import.state.error": "erreur",
|
||||
"import.state.cancelled": "annulé",
|
||||
"import.featuredTitle": "Modèles .cmf prêts à l'emploi",
|
||||
"import.featuredSub": "Modèles .cmf déjà convertis du compte infosave sur HuggingFace — la liste est lue en direct, les nouveaux modèles apparaissent d'eux-mêmes.",
|
||||
"import.readyCmfBadge": ".cmf prêt",
|
||||
"import.directDownload": "Télécharger",
|
||||
"import.q.q8_2f": "8 bits à deux champs (𝒲×θ) — recommandé : meilleur rapport qualité/taille.",
|
||||
"import.q.q8_row": "8 bits par ligne — simple et robuste.",
|
||||
"import.q.q4": "4 bits par bloc — le plus compact, qualité moindre.",
|
||||
"import.q.q4_tiled": "4 bits en tuiles (Q4_TILED) — 1,66× plus rapide sur GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — budget de bits par expert.",
|
||||
"import.q.q1t": "Ternaire 1,58 bit (Q1T) — BitNet b1.58 avec valeurs aberrantes éparses.",
|
||||
"import.q.q1s": "1 bit épars (Q1S) — base 1 bit avec valeurs aberrantes holographiques.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit avec diffusion d'erreur.",
|
||||
"import.q.q1": "1 bit (Q1) — format purement binaire.",
|
||||
"import.q.f16": "16 bits — sans quantification.",
|
||||
"import.q.f32": "32 bits — pleine précision (fichier volumineux).",
|
||||
"import.adv.title": "Paramètres avancés",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Cœur linéaire",
|
||||
"import.adv.linearHint": "Comment les couches d'attention linéaire sont reportées ou fusionnées.",
|
||||
"import.adv.nphase": "vmf_phase phases/tête",
|
||||
"import.adv.vbitShape": "Forme du budget VBIT",
|
||||
"import.adv.meanBits": "Bits moyens VBIT",
|
||||
"import.adv.shard": "Taille max. des shards (Go)",
|
||||
"import.adv.noShard": "pas de découpage",
|
||||
"import.adv.skipMtp": "ignorer la tête MTP",
|
||||
"import.adv.o1": "Attention O(1)",
|
||||
"import.adv.o1Hint": "Attention en flux à mémoire constante : le cache KV est libéré après le prefill, la mémoire ne croît donc pas avec de longs contextes. Les poids restent identiques octet pour octet — seule une indication de chargement est enregistrée ; vous pouvez revenir en arrière à tout moment (servir avec --o1 off). Validé sur Qwen3-0.6B : toutes couches ≈ ×1,18 de perplexité.",
|
||||
"import.adv.o1Off": "désactivé — attention exacte (par défaut)",
|
||||
"import.adv.o1All": "all — convertir chaque couche softmax",
|
||||
"import.adv.o1Deep": "deep N — seulement les N couches les plus profondes",
|
||||
"import.adv.o1Custom": "personnalisé — liste explicite de couches",
|
||||
"import.adv.o1DeepN": "N couches profondes",
|
||||
"import.adv.o1Layers": "Couches (séparées par des virgules)",
|
||||
"import.adv.o1mHint": "budget de landmarks (32 par défaut)",
|
||||
"import.adv.o1windowHint": "largeur de la fenêtre exacte (128 par défaut)",
|
||||
"import.adv.o1sinkHint": "clés sink exactes permanentes (4 par défaut)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuration initiale",
|
||||
"onboarding.welcome": "Bienvenue dans Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choisissez votre langue — vous pourrez la changer à tout moment dans l'en-tête.",
|
||||
"onboarding.next": "Suivant",
|
||||
"onboarding.back": "Retour",
|
||||
"onboarding.skip": "Passer pour l'instant",
|
||||
"onboarding.skipSetup": "Passer",
|
||||
"onboarding.finish": "Terminer la configuration",
|
||||
"onboarding.featuredTitle": "Modèles prêts à l'emploi (en un clic)",
|
||||
"onboarding.featuredSub": "Ce sont des modèles .cmf déjà convertis, publiés par infosave sur HuggingFace — il suffit de les télécharger, aucune conversion nécessaire. Choisissez-en un ou plusieurs pour démarrer immédiatement.",
|
||||
"onboarding.download": "Télécharger",
|
||||
"onboarding.hfTitle": "N'importe quel modèle HuggingFace — conversion automatique",
|
||||
"onboarding.hfSub": "Cherchez un modèle, choisissez une quantification, et nous le convertissons automatiquement en .cmf. Aucune modification manuelle de la configuration.",
|
||||
"onboarding.noJobs": "Aucun téléchargement pour l'instant.",
|
||||
"onboarding.doneTitle": "Tout est prêt",
|
||||
"onboarding.doneSub": "Les modèles téléchargés seront enregistrés automatiquement. Vous pourrez en ajouter d'autres dans Importer ou Modèles.",
|
||||
"onboarding.doneHint": "« Terminer la configuration » connecte chaque téléchargement achevé comme modèle local, démarre son serveur .cmf et y dirige le routage. Tout cela reste modifiable ensuite dans Modèles.",
|
||||
"onboarding.willRegister": "Seront connectés comme modèles locaux",
|
||||
"onboarding.nothingToRegister": "Aucun téléchargement terminé — rien à connecter. Revenez à l'étape précédente pour choisir un modèle.",
|
||||
"onboarding.registeredN": "{n} modèle(s) connecté(s) : {ids}",
|
||||
"onboarding.runtimeMissing": "Le runtime cortiq n'est pas installé — installez-le dans Paramètres → Modèles locaux pour les servir.",
|
||||
"onboarding.downloaded": "téléchargé",
|
||||
"onboarding.update": "Mettre à jour",
|
||||
"onboarding.deleteFile": "Supprimer le fichier .cmf du disque",
|
||||
"onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.",
|
||||
"onboarding.unregistered": "Également désenregistré : {ids}",
|
||||
};
|
||||
|
||||
Vendored
+55
-16
@@ -31,7 +31,7 @@ export default {
|
||||
"nav.routing": "Роутинг",
|
||||
"nav.protocols": "Протоколы",
|
||||
"nav.keys": "API-ключи",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Песочница",
|
||||
"nav.settings": "Настройки",
|
||||
"nav.foot": "Шлюз · v{v}",
|
||||
"nav.billing": "Ключ роутера и оплата",
|
||||
@@ -80,7 +80,7 @@ export default {
|
||||
"dash.groupby.label": "по задаче",
|
||||
"dash.groupby.account": "по аккаунту",
|
||||
"dash.recent.title": "Последние запросы",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Playground.",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Песочницу.",
|
||||
"dash.col.time": "Время",
|
||||
"dash.col.model": "Модель",
|
||||
"dash.col.tier": "Уровень",
|
||||
@@ -131,8 +131,6 @@ export default {
|
||||
"models.form.secretMissing": "не задан",
|
||||
"models.form.secretPlaceholder": "задать или заменить ключ (только запись)",
|
||||
"models.deleteConfirm": "Удалить модель \"{id}\"?",
|
||||
"models.deleteFile": "также удалить файл модели с диска",
|
||||
"models.deletedWithFile": "Удалено (файл стёрт с диска)",
|
||||
"models.form.idOrModel": "Укажите id или имя модели",
|
||||
"models.form.apiKey": "API-ключ",
|
||||
"models.form.apiKeyPlaceholder": "вставьте API-ключ",
|
||||
@@ -140,6 +138,18 @@ export default {
|
||||
"models.form.apiKeyHint": "Хранится безопасно на шлюзе; больше не показывается.",
|
||||
"models.form.keyStored": "Ключ сохранён — оставьте пустым, чтобы не менять, или вставьте новый.",
|
||||
"models.form.keyFromEnv": "Используется ключ из переменной окружения.",
|
||||
"models.form.paramsTitle": "Параметры генерации, O(1) и режим размышления",
|
||||
"models.form.temperature": "Температура",
|
||||
"models.form.temperatureHint": "Температура генерации (0.0 — точные факты, 1.0+ — творчество).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Порог сэмплирования токенов (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Макс. токенов (Max Tokens)",
|
||||
"models.form.maxTokensHint": "Максимальное количество генерируемых токенов.",
|
||||
"models.form.thinkBudget": "Режим размышления / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Бюджет токенов мыслей (0 = выключен, N > 0 = лимит рассуждений).",
|
||||
"models.form.systemPrompt": "Системный промпт по умолчанию",
|
||||
"models.form.systemPromptHint": "Автоматически подставляется, если запрос не содержит системное сообщение.",
|
||||
"models.form.systemPromptPlaceholder": "Вы — полезный ассистент...",
|
||||
"models.form.testKey": "Проверить и загрузить модели",
|
||||
"models.form.testing": "проверка…",
|
||||
"models.form.foundModels": "загружено моделей: {n}",
|
||||
@@ -209,7 +219,7 @@ export default {
|
||||
"keys.allowAll": "все",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Песочница",
|
||||
"pg.subtitle": "Отправьте запрос через активный пайплайн и изучите решение роутера.",
|
||||
"pg.system": "Системный промпт",
|
||||
"pg.systemPlaceholder": "необязательное системное сообщение",
|
||||
@@ -263,14 +273,11 @@ export default {
|
||||
"settings.cmf.pick": "выберите модель",
|
||||
"settings.cmf.models": "Локальные модели",
|
||||
"settings.cmf.modelsHelp": "Каждая строка — отдельная модель: шлюз поднимает свой cortiq serve на своём порту и регистрирует её в пуле. Добавляйте столько, сколько есть .cmf-файлов и свободных портов.",
|
||||
"settings.cmf.rowHint": "id · файл .cmf · порт · потоки · GPU · O(1). Применяется сразу при сохранении: удалённая строка останавливает сервер (память освобождается), новая — запускает.",
|
||||
"settings.cmf.o1Auto": "O(1): авто",
|
||||
"settings.cmf.o1Help": "O(1)-внимание при запуске: авто — использовать подсказку из .cmf; off — принудительно точное внимание (лечит бессвязный вывод после слишком агрессивной --o1 конвертации, без переконвертации); all/deepN — принудительно включить на этих слоях.",
|
||||
"settings.cmf.rowHint": "id · файл .cmf · порт · потоки · GPU. Изменения применяются после перезапуска; удаление строки останавливает сервер.",
|
||||
"settings.cmf.addServer": "+ Добавить модель",
|
||||
"settings.cmf.checkPort": "Проверить порт",
|
||||
"settings.cmf.portFree": "порт свободен",
|
||||
"settings.cmf.portBusy": "порт занят",
|
||||
"settings.cmf.portOwn": "занят текущим сервером шлюза — освободится после перезапуска, замена допустима",
|
||||
"settings.cmf.useSuggested": "занять {port}",
|
||||
"settings.cmf.serving": "работает: {model} на :{port}",
|
||||
"settings.cmf.intro": "Запускайте модели локально движком cortiq — работает даже офлайн или когда роутер недоступен.",
|
||||
@@ -378,7 +385,7 @@ export default {
|
||||
"import.failed": "Конвертация не удалась — смотрите лог.",
|
||||
"import.register": "Подключить как локальную модель",
|
||||
"import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
|
||||
"import.jobs": "Конвертации",
|
||||
"import.jobs": "Загрузки / Конвертации",
|
||||
"import.cancel": "Отменить",
|
||||
"import.cancelling": "Отмена — удаляю недописанный файл…",
|
||||
"import.delete": "Удалить",
|
||||
@@ -388,12 +395,22 @@ export default {
|
||||
"import.state.done": "готово",
|
||||
"import.state.error": "ошибка",
|
||||
"import.state.cancelled": "отменено",
|
||||
"import.featuredTitle": "Рекомендуемые готовые .cmf модели",
|
||||
"import.featuredSub": "Готовые .cmf модели из аккаунта infosave на HuggingFace — список читается вживую, новые модели появляются здесь сами.",
|
||||
"import.readyCmfBadge": "Готовая .cmf",
|
||||
"import.directDownload": "Скачать",
|
||||
"import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
|
||||
"import.q.q8_row": "8-бит построчно — просто, надёжно.",
|
||||
"import.q.q4": "4-бит блочно — компактнее, ниже качество.",
|
||||
"import.q.q4_tiled": "4-бит тайловый (Q4_TILED) — х1.66 быстрее на Metal GPU.",
|
||||
"import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
|
||||
"import.q.q1t": "1.58-бит тернарный (Q1T) — BitNet b1.58 с оутлайерами.",
|
||||
"import.q.q1s": "1-бит спарс (Q1S) — 1-бит база с голографическими оутлайерами.",
|
||||
"import.q.q1p": "1-бит PTQ (Q1P) — 1-бит с переносом ошибки (error diffusion).",
|
||||
"import.q.q1": "1-бит (Q1) — чистый бинарный формат.",
|
||||
"import.q.f16": "16-бит — без квантования.",
|
||||
"import.q.f32": "32-бит — полная точность (большой файл).",
|
||||
|
||||
"import.adv.title": "Расширенные настройки",
|
||||
"import.adv.auto": "авто",
|
||||
"import.adv.linearCore": "Линейное ядро",
|
||||
@@ -413,12 +430,34 @@ export default {
|
||||
"import.adv.o1DeepN": "N глубоких слоёв",
|
||||
"import.adv.o1Layers": "Слои (через запятую)",
|
||||
"import.adv.o1mHint": "число опорных точек (по умолчанию 32)",
|
||||
"import.adv.o1mLabel": "Опорные точки (--o1-m)",
|
||||
"import.adv.o1mHelp": "Сколько опорных токенов представляют всю старую историю при аппроксимации внимания. Больше — ближе к точному вниманию, но медленнее и больше памяти; меньше — быстрее, но грубее. 32 — проверенное значение; меняйте, только если понимаете зачем.",
|
||||
"import.adv.o1windowLabel": "Точное окно (--o1-window)",
|
||||
"import.adv.o1windowHelp": "Последние N токенов всегда обрабатываются точным (не аппроксимированным) вниманием — это защищает качество на только что написанном/прочитанном тексте. Больше — лучше локальное качество, больше памяти. По умолчанию 128.",
|
||||
"import.adv.o1sinkLabel": "Sink-ключи (--o1-sink)",
|
||||
"import.adv.o1sinkHelp": "Первые N токенов контекста навсегда остаются точными («якоря внимания»). Модели опираются на эти якоря — их точность стабилизирует длинную генерацию. По умолчанию 4; менять почти никогда не нужно.",
|
||||
"import.adv.o1windowHint": "ширина точного окна (по умолчанию 128)",
|
||||
"import.adv.o1sinkHint": "постоянные точные sink-ключи (по умолчанию 4)",
|
||||
|
||||
// онбординг (первый запуск)
|
||||
"onboarding.title": "Первоначальная настройка",
|
||||
"onboarding.welcome": "Добро пожаловать в Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Выберите язык — его можно сменить в шапке в любой момент.",
|
||||
"onboarding.next": "Далее",
|
||||
"onboarding.back": "Назад",
|
||||
"onboarding.skip": "Пропустить",
|
||||
"onboarding.skipSetup": "Пропустить",
|
||||
"onboarding.finish": "Завершить",
|
||||
"onboarding.featuredTitle": "Готовые модели в один клик",
|
||||
"onboarding.featuredSub": "Это уже сконвертированные .cmf модели от infosave на HuggingFace — просто Скачать, без конвертации. Выберите одну или несколько для мгновенного старта.",
|
||||
"onboarding.download": "Скачать",
|
||||
"onboarding.hfTitle": "Любая модель с HuggingFace — автоконвертация",
|
||||
"onboarding.hfSub": "Найдите любую модель, выберите квантизацию — мы автоматически сконвертируем её в .cmf. Никаких ручных правок конфига.",
|
||||
"onboarding.noJobs": "Пока нет загрузок.",
|
||||
"onboarding.doneTitle": "Готово",
|
||||
"onboarding.doneSub": "Скачанные модели зарегистрируются автоматически. Добавить ещё можно во вкладке Импорт или Модели.",
|
||||
"onboarding.doneHint": "Кнопка «Завершить» подключит все скачанные модели как локальные, поднимет для них .cmf сервер и настроит роутинг. Всё это потом можно изменить во вкладке Модели.",
|
||||
"onboarding.willRegister": "Будут подключены как локальные модели",
|
||||
"onboarding.nothingToRegister": "Завершённых загрузок пока нет — подключать нечего. Вернитесь на шаг назад и выберите модель.",
|
||||
"onboarding.registeredN": "Подключено моделей: {n} — {ids}",
|
||||
"onboarding.runtimeMissing": "Среда cortiq не установлена — установите её в Настройки → Локальные модели, чтобы запускать эти модели.",
|
||||
"onboarding.downloaded": "скачана",
|
||||
"onboarding.update": "Обновить",
|
||||
"onboarding.deleteFile": "Удалить файл .cmf с диска",
|
||||
"onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.",
|
||||
"onboarding.unregistered": "Также отключено из моделей: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+188
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "Giriş TTL",
|
||||
"settings.cache.embed": "Gömme modeli",
|
||||
"settings.cache.note": "Anlamsal önbellek ayarları yeniden başlatmadan sonra geçerli olur.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Model içe aktar",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Günlükleri temizle",
|
||||
"dash.clearConfirm": "Kaydedilmiş tüm istatistikler ve günlükler silinsin mi? Bu işlem geri alınamaz.",
|
||||
"dash.cleared": "Günlükler temizlendi",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Bir id ya da model adı girin",
|
||||
"models.form.apiKey": "API anahtarı",
|
||||
"models.form.apiKeyPlaceholder": "API anahtarınızı yapıştırın",
|
||||
"models.form.apiKeyNotNeeded": "gerekmiyor (yerel sunucu)",
|
||||
"models.form.apiKeyHint": "Ağ geçidinde güvenle saklanır; bir daha gösterilmez.",
|
||||
"models.form.keyStored": "Kayıtlı bir anahtar var — korumak için boş bırakın ya da yenisini yapıştırın.",
|
||||
"models.form.keyFromEnv": "Ortam değişkenindeki anahtar kullanılıyor.",
|
||||
"models.form.paramsTitle": "Üretim ve düşünme parametreleri (O(1), MTP, sıcaklık)",
|
||||
"models.form.temperature": "Sıcaklık",
|
||||
"models.form.temperatureHint": "Üretim sıcaklığı (0.0 = belirlenimci, 1.0+ = yaratıcı).",
|
||||
"models.form.topP": "Top-P (nucleus örnekleme)",
|
||||
"models.form.topPHint": "Örnekleme eşiği (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Maks. token",
|
||||
"models.form.maxTokensHint": "Üretilecek en fazla token sayısı.",
|
||||
"models.form.thinkBudget": "Düşünme modu / düşünme bütçesi",
|
||||
"models.form.thinkBudgetHint": "Akıl yürütme token bütçesi (0 = kapalı, N > 0 = en fazla akıl yürütme tokeni).",
|
||||
"models.form.systemPrompt": "Varsayılan sistem istemi",
|
||||
"models.form.systemPromptHint": "İstekte sistem istemi yoksa otomatik olarak başa eklenir.",
|
||||
"models.form.systemPromptPlaceholder": "Yardımsever bir asistansın…",
|
||||
"models.form.testKey": "Test et ve modelleri yükle",
|
||||
"models.form.testing": "kontrol ediliyor…",
|
||||
"models.form.foundModels": "{n} model yüklendi",
|
||||
"models.form.modelPlaceholder": "aramak için yazın…",
|
||||
"models.form.modelHint": "Anahtarınızla listeyi çekmek için “Test et ve modelleri yükle”ye tıklayın, sonra birini seçin.",
|
||||
"models.managed": "yönetilen",
|
||||
"models.managedHint": "Yönetilen CMF sunucusu tarafından sağlanır (Ayarlar → Yerel modeller). Yapılandırmayı orada yapın.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Yönlendiriciyi kullan (kapalı = yalnızca yerel modeller)",
|
||||
"settings.cmf": "Yerel modeller (CMF)",
|
||||
"settings.cmf.localOnly": "Yalnızca yerel modelleri kullan",
|
||||
"settings.cmf.manage": "Yerel model sunucusu çalıştır",
|
||||
"settings.cmf.autoInstall": "cortiq'i otomatik kur (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq'i otomatik güncelle",
|
||||
"settings.cmf.model": "Yerel .cmf model yolu",
|
||||
"settings.cmf.port": "Yerel sunucu portu",
|
||||
"settings.cmf.modelId": "Havuzdaki model id'si",
|
||||
"settings.cmf.note": "Yerel bir .cmf dosyasını cortiq ile sunar (github.com/infosave2007/cmf). Yönlendirici erişilemez olsa bile çalışır.",
|
||||
"settings.cmf.threads": "İş parçacıkları",
|
||||
"settings.cmf.gpu": "GPU kullan (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "İş parçacıkları çözme aşamasındaki matris-vektör çarpımlarını paralelleştirir (~3× hızlı; 0 = çalışma zamanı varsayılanı). GPU'ya aktarım daha çok uzun istemlerin prefill aşamasına yarar — çözme bellek bant genişliğine bağlı olduğundan Apple Silicon'da etkisi yaklaşık nötrdür.",
|
||||
"settings.cmf.pick": "bir model seçin",
|
||||
"settings.cmf.models": "Yerel modeller",
|
||||
"settings.cmf.modelsHelp": "Her satır, ağ geçidinin çalıştırdığı (kendi portunda kendi cortiq serve süreci olan) ve havuza kaydettiği bir modeldir. Kaç .cmf dosyanız ve boş portunuz varsa o kadar ekleyebilirsiniz.",
|
||||
"settings.cmf.rowHint": "id · .cmf dosyası · port · iş parçacığı · GPU. Değişiklikler yeniden başlatınca geçerli olur; satırı silmek o sunucuyu durdurur.",
|
||||
"settings.cmf.addServer": "+ Yerel model ekle",
|
||||
"settings.cmf.checkPort": "Portu kontrol et",
|
||||
"settings.cmf.portFree": "port boş",
|
||||
"settings.cmf.portBusy": "port kullanımda",
|
||||
"settings.cmf.useSuggested": "{port} kullan",
|
||||
"settings.cmf.serving": "{model} :{port} üzerinde sunuluyor",
|
||||
"settings.cmf.intro": "Modelleri cortiq motoruyla yerelde çalıştırın — çevrimdışıyken ya da yönlendirici çöktüğünde bile çalışır.",
|
||||
"settings.cmf.runtime": "cortiq çalışma zamanı",
|
||||
"settings.cmf.runtimeHelp": ".cmf modellerini yükleyip çalıştıran `cortiq` komut satırı aracı. Yerel sunucuyu başlatmadan önce bir kez kurun.",
|
||||
"settings.cmf.install": "cortiq'i kur",
|
||||
"settings.cmf.reinstall": "Yeniden kur / güncelle",
|
||||
"settings.cmf.installing": "kuruluyor… (derleniyor, birkaç dakika)",
|
||||
"settings.cmf.installedOk": "cortiq kuruldu",
|
||||
"settings.cmf.notInstalled": "kurulu değil",
|
||||
"settings.cmf.addModel": "Model ekle",
|
||||
"settings.cmf.addModelHelp": "Modeller “Model içe aktar” sekmesinde indirilip dönüştürülür (HuggingFace → .cmf). Sonra aşağıdaki yerel sunucuyu o .cmf dosyasına yönlendirin.",
|
||||
"settings.cmf.importNote": "Henüz model yok mu? Bir tane içe aktarın:",
|
||||
"settings.cmf.importLink": "İçe aktarmayı aç",
|
||||
"settings.cmf.modelHelp": "Sunulacak .cmf dosyasının yolu (ör. models/qwen.cmf), içe aktarma sekmesinde üretilir. Farklı görevler için birden çok model mi? Hepsini içe aktarıp bu yolu değiştirin.",
|
||||
"settings.cmf.portHelp": "Yerel model sunucusunun dinlediği TCP portu (varsayılan 8081). Yalnızca başka bir servisle çakışırsa değiştirin.",
|
||||
"settings.cmf.threadsHelp": "Üretim için kaç CPU çekirdeği kullanılacağı. Daha fazlası = daha hızlı (tek çekirdeğe göre ~3×); 0 seçimi çalışma zamanına bırakır.",
|
||||
"settings.cmf.gpuHelp": "Matris hesaplarını GPU'ya aktarır (Metal/wgpu). Apple Silicon'da çoğunlukla uzun istemleri hızlandırır — çözme bellek bant genişliğine bağlıdır.",
|
||||
"settings.cmf.localOnlyHelp": "Her isteği yerel modele yönlendirir; yönlendiriciyi ve tüm bulut sağlayıcılarını yok sayar.",
|
||||
"settings.cmf.manageHelp": "Ağ geçidi açılışta aşağıdaki .cmf için `cortiq serve` başlatır ve istekleri oraya yönlendirir. cortiq'in kurulu olması gerekir (adım 1). Yeniden başlatınca geçerli olur.",
|
||||
"settings.cmf.autoInstallHelp": "Açılışta cortiq yoksa crates.io üzerinden otomatik kurar.",
|
||||
"settings.cmf.autoUpdateHelp": "Açılışta daha yeni bir sürüm varsa cortiq'i en son sürüme yükseltir.",
|
||||
"settings.router.enabledHelp": "Her isteği cortiq yönlendiricisi sınıflandırsın (görev türü + karmaşıklık) ve model seçsin. Kapalı = yalnızca yerel yönlendirme tablosunu kullan.",
|
||||
"settings.router.verifyTlsHelp": "Yönlendiricinin TLS sertifikasını doğrula. Yalnızca kendinden imzalı ya da IP'ye bağlı sertifikada kapatın.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus metriklerini GET /metrics adresinde yayınla.",
|
||||
"settings.cortiq.echoHelp": "Her yanıt gövdesine `cortiq` alanı ekle (görev, model, maliyet). Yanıt başlıkları her zaman gönderilir.",
|
||||
"settings.stats.enabledHelp": "Panoyu ve grafikleri beslemek için son istekleri kaydet.",
|
||||
"settings.cache.enabledHelp": "Anlamsal önbellek: yeni bir istem geçmiştekine çok benziyorsa kaydedilmiş yanıtı döndürür ve model çağrısını atlar. Havuzda bir gömme (embedding) modeli gerekir.",
|
||||
"settings.log.bodiesHelp": "İstek/yanıt mesaj gövdelerinin tamamını günlüğe yaz. Gizlilik için varsayılan olarak kapalı.",
|
||||
"settings.router.saveKey": "Anahtarı kaydet",
|
||||
"settings.router.keySaved": "Yönlendirici anahtarı kaydedildi",
|
||||
"settings.router.keyEmpty": "Önce anahtarı yapıştırın",
|
||||
"settings.router.secretHelp": "api.allaigate.com e-postanızdaki anahtarı yapıştırıp Anahtarı kaydet'e basın. Bu bir “API anahtarları” değeri DEĞİLDİR — onlar ağ geçidinizi çağıran istemciler içindir.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — anahtar kayıtlı (değiştirmek için yenisini yapıştırın)",
|
||||
"settings.route.strategyHelp": "Sınıflandırma için yönlendiriciye gönderilen metin: last_user = yalnızca son kullanıcı mesajı; last_user_plus_system = son kullanıcı mesajı + sistem istemi; concat_all = tüm konuşma.",
|
||||
"settings.route.maxCharsHelp": "Bu metnin yönlendiriciye gönderilen en fazla karakter sayısı (fazlası kırpılır). Daha fazlası = daha isabetli sınıflandırma, ama daha yavaş ve pahalı yönlendirici çağrıları.",
|
||||
"settings.route.cacheTtlHelp": "Aynı istek için yönlendirme kararının, yönlendiriciye yeniden sorulmadan önce ne kadar önbellekte tutulacağı (ör. 60s, 5m).",
|
||||
"settings.route.profileHelp": "İstek belirtmediğinde kullanılacak varsayılan yönlendirme profili: cost-saver (en ucuz), balanced ya da quality-first (en iyi model). Yönlendiricinin hangi modeli seçeceğini etkiler.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "HuggingFace'te arayın, yerel bir .cmf dosyasına dönüştürün ve sunun.",
|
||||
"import.searchPlaceholder": "HuggingFace modellerinde ara (ör. qwen3, llama)…",
|
||||
"import.searching": "Aranıyor…",
|
||||
"import.noResults": "Model bulunamadı.",
|
||||
"import.configure": "Dönüştürmeyi yapılandır",
|
||||
"import.outName": "Çıktı adı",
|
||||
"import.outNameHint": ".cmf dosya adı (harf, rakam, - ve _).",
|
||||
"import.quant": "Niceleme",
|
||||
"import.convert": "Dönüştür ve yükle",
|
||||
"import.converting": "Dönüştürülüyor",
|
||||
"import.started": "Dönüştürme başladı",
|
||||
"import.failed": "Dönüştürme başarısız — günlüğe bakın.",
|
||||
"import.register": "Yerel model olarak kaydet",
|
||||
"import.registered": "“{id}” olarak kaydedildi. Sunmak için cortiq-server'ı çalıştırın.",
|
||||
"import.jobs": "İndirmeler ve dönüştürmeler",
|
||||
"import.cancel": "İptal",
|
||||
"import.cancelling": "İptal ediliyor — yarım kalan dosya temizleniyor…",
|
||||
"import.delete": "Sil",
|
||||
"import.deleteConfirm": "Bu model ve dönüştürülmüş .cmf dosyası diskten silinsin mi?",
|
||||
"import.deleted": "Dosyalarıyla birlikte silindi",
|
||||
"import.state.running": "çalışıyor",
|
||||
"import.state.done": "tamam",
|
||||
"import.state.error": "hata",
|
||||
"import.state.cancelled": "iptal edildi",
|
||||
"import.featuredTitle": "Öne çıkan hazır .cmf modelleri",
|
||||
"import.featuredSub": "HuggingFace'teki infosave hesabından hazır .cmf modelleri — liste canlı okunur, yeni modeller kendiliğinden görünür.",
|
||||
"import.readyCmfBadge": "Hazır .cmf",
|
||||
"import.directDownload": "İndir",
|
||||
"import.q.q8_2f": "8 bit çift alan (𝒲×θ) — önerilen: en iyi kalite/boyut dengesi.",
|
||||
"import.q.q8_row": "Satır başına 8 bit — basit ve sağlam.",
|
||||
"import.q.q4": "4 bit blok — en küçük, kalite daha düşük.",
|
||||
"import.q.q4_tiled": "4 bit döşeli (Q4_TILED) — Metal GPU'da 1,66× daha hızlı.",
|
||||
"import.q.vbit": "Değişken 3–8 bit — uzman başına bit bütçesi.",
|
||||
"import.q.q1t": "1,58 bit üçlü (Q1T) — seyrek aykırı değerlerle BitNet b1.58.",
|
||||
"import.q.q1s": "1 bit seyrek (Q1S) — holografik aykırı değerlerle 1 bit taban.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — hata yayılımlı 1 bit.",
|
||||
"import.q.q1": "1 bit (Q1) — saf ikili biçim.",
|
||||
"import.q.f16": "16 bit — niceleme yok.",
|
||||
"import.q.f32": "32 bit — tam hassasiyet (büyük dosya).",
|
||||
"import.adv.title": "Gelişmiş ayarlar",
|
||||
"import.adv.auto": "otomatik",
|
||||
"import.adv.linearCore": "Doğrusal çekirdek",
|
||||
"import.adv.linearHint": "Doğrusal dikkat katmanlarının nasıl taşındığı/katlandığı.",
|
||||
"import.adv.nphase": "vmf_phase faz/başlık",
|
||||
"import.adv.vbitShape": "VBIT bütçe biçimi",
|
||||
"import.adv.meanBits": "VBIT ortalama bit",
|
||||
"import.adv.shard": "Parça üst sınırı (GB)",
|
||||
"import.adv.noShard": "parçalama yok",
|
||||
"import.adv.skipMtp": "MTP başlığını atla",
|
||||
"import.adv.o1": "O(1) dikkat",
|
||||
"import.adv.o1Hint": "Sabit bellekli akış dikkati: KV önbelleği prefill sonrası serbest bırakılır, böylece uzun bağlamlar belleği büyütmez. Ağırlıklar bayt bayt aynı kalır — yalnızca yükleme anına dair bir ipucu kaydedilir; istediğiniz zaman geri dönebilirsiniz (--o1 off ile sunun). Qwen3-0.6B üzerinde doğrulandı: tüm katmanlar ≈ ×1,18 şaşkınlık (perplexity).",
|
||||
"import.adv.o1Off": "kapalı — tam dikkat (varsayılan)",
|
||||
"import.adv.o1All": "all — her softmax katmanını dönüştür",
|
||||
"import.adv.o1Deep": "deep N — yalnızca en derin N katman",
|
||||
"import.adv.o1Custom": "özel — açık katman listesi",
|
||||
"import.adv.o1DeepN": "N derin katman",
|
||||
"import.adv.o1Layers": "Katmanlar (virgülle ayrılmış)",
|
||||
"import.adv.o1mHint": "landmark bütçesi (varsayılan 32)",
|
||||
"import.adv.o1windowHint": "tam pencere genişliği (varsayılan 128)",
|
||||
"import.adv.o1sinkHint": "kalıcı tam sink anahtarları (varsayılan 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "İlk kurulum",
|
||||
"onboarding.welcome": "Cortiq Gateway'e hoş geldiniz",
|
||||
"onboarding.welcomeSub": "Dilinizi seçin — üst çubuktan istediğiniz zaman değiştirebilirsiniz.",
|
||||
"onboarding.next": "İleri",
|
||||
"onboarding.back": "Geri",
|
||||
"onboarding.skip": "Şimdilik atla",
|
||||
"onboarding.skipSetup": "Atla",
|
||||
"onboarding.finish": "Kurulumu tamamla",
|
||||
"onboarding.featuredTitle": "Hazır modeller (tek tık)",
|
||||
"onboarding.featuredSub": "Bunlar HuggingFace'te infosave tarafından yayınlanan, önceden dönüştürülmüş .cmf modelleridir — sadece indirin, dönüştürme gerekmez. Hemen başlamak için bir ya da birkaçını seçin.",
|
||||
"onboarding.download": "İndir",
|
||||
"onboarding.hfTitle": "Herhangi bir HuggingFace modeli — otomatik dönüştürme",
|
||||
"onboarding.hfSub": "İstediğiniz modeli arayın, bir niceleme seçin; biz onu otomatik olarak .cmf biçimine dönüştürelim. Elle yapılandırma düzenlemesi yok.",
|
||||
"onboarding.noJobs": "Henüz indirme yok.",
|
||||
"onboarding.doneTitle": "Her şey hazır",
|
||||
"onboarding.doneSub": "İndirdiğiniz modeller otomatik olarak kaydedilir. Daha fazlasını İçe aktar ya da Modeller sekmesinden ekleyebilirsiniz.",
|
||||
"onboarding.doneHint": "“Kurulumu tamamla”, biten her indirmeyi yerel model olarak bağlar, .cmf sunucusunu başlatır ve yönlendirmeyi ona çevirir. Tümünü sonradan Modeller sekmesinden değiştirebilirsiniz.",
|
||||
"onboarding.willRegister": "Yerel model olarak bağlanacak",
|
||||
"onboarding.nothingToRegister": "Henüz tamamlanmış indirme yok — bağlanacak bir şey bulunmuyor. Bir adım geri dönüp model seçin.",
|
||||
"onboarding.registeredN": "{n} model bağlandı: {ids}",
|
||||
"onboarding.runtimeMissing": "cortiq çalışma zamanı kurulu değil — bunları sunabilmek için Ayarlar → Yerel modeller bölümünden kurun.",
|
||||
"onboarding.downloaded": "indirildi",
|
||||
"onboarding.update": "Güncelle",
|
||||
"onboarding.deleteFile": ".cmf dosyasını diskten sil",
|
||||
"onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.",
|
||||
"onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+188
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "条目 TTL",
|
||||
"settings.cache.embed": "嵌入模型",
|
||||
"settings.cache.note": "缓存设置将在重启后生效。",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "导入模型",
|
||||
|
||||
// dash
|
||||
"dash.clear": "清空日志",
|
||||
"dash.clearConfirm": "清空所有已记录的统计与日志?此操作无法撤销。",
|
||||
"dash.cleared": "日志已清空",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "请输入 id 或模型名称",
|
||||
"models.form.apiKey": "API 密钥",
|
||||
"models.form.apiKeyPlaceholder": "粘贴你的 API 密钥",
|
||||
"models.form.apiKeyNotNeeded": "无需密钥(本地服务器)",
|
||||
"models.form.apiKeyHint": "安全保存在网关中,之后不会再次显示。",
|
||||
"models.form.keyStored": "已保存一个密钥 — 留空表示保留,或粘贴新的密钥替换。",
|
||||
"models.form.keyFromEnv": "正在使用环境变量中的密钥。",
|
||||
"models.form.paramsTitle": "生成与思考参数(O(1)、MTP、温度)",
|
||||
"models.form.temperature": "温度",
|
||||
"models.form.temperatureHint": "生成温度(0.0 = 确定性,1.0+ = 更有创造性)。",
|
||||
"models.form.topP": "Top-P(核采样)",
|
||||
"models.form.topPHint": "采样阈值(0.0 .. 1.0)。",
|
||||
"models.form.maxTokens": "最大 token 数",
|
||||
"models.form.maxTokensHint": "生成内容的最大 token 数。",
|
||||
"models.form.thinkBudget": "思考模式 / 思考预算",
|
||||
"models.form.thinkBudgetHint": "推理 token 预算(0 = 关闭,N > 0 = 推理 token 上限)。",
|
||||
"models.form.systemPrompt": "默认系统提示词",
|
||||
"models.form.systemPromptHint": "当请求中没有系统提示词时自动添加到前面。",
|
||||
"models.form.systemPromptPlaceholder": "你是一个乐于助人的助手……",
|
||||
"models.form.testKey": "测试并加载模型",
|
||||
"models.form.testing": "检查中……",
|
||||
"models.form.foundModels": "已加载 {n} 个模型",
|
||||
"models.form.modelPlaceholder": "输入以搜索……",
|
||||
"models.form.modelHint": "点击“测试并加载模型”用你的密钥拉取列表,然后选择一个。",
|
||||
"models.managed": "托管",
|
||||
"models.managedHint": "由托管的 CMF 服务器提供(设置 → 本地模型)。请在那里配置。",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "使用路由器(关闭 = 仅本地模型)",
|
||||
"settings.cmf": "本地模型(CMF)",
|
||||
"settings.cmf.localOnly": "仅使用本地模型",
|
||||
"settings.cmf.manage": "运行本地模型服务器",
|
||||
"settings.cmf.autoInstall": "自动安装 cortiq(crates.io)",
|
||||
"settings.cmf.autoUpdate": "自动更新 cortiq",
|
||||
"settings.cmf.model": "本地 .cmf 模型路径",
|
||||
"settings.cmf.port": "本地服务器端口",
|
||||
"settings.cmf.modelId": "模型在池中的 id",
|
||||
"settings.cmf.note": "通过 cortiq 提供本地 .cmf 服务(github.com/infosave2007/cmf)。即使路由器不可用也能工作。",
|
||||
"settings.cmf.threads": "工作线程",
|
||||
"settings.cmf.gpu": "使用 GPU(Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "线程会并行化解码阶段的矩阵向量乘法(约快 3 倍;0 = 运行时默认值)。GPU 卸载主要加速长提示词的 prefill — 解码受内存带宽限制,因此在 Apple Silicon 上基本持平。",
|
||||
"settings.cmf.pick": "选择一个模型",
|
||||
"settings.cmf.models": "本地模型",
|
||||
"settings.cmf.modelsHelp": "每一行都是网关运行的一个模型(各自独立的 cortiq serve,各占一个端口)并注册到模型池。有多少 .cmf 文件和空闲端口就可以加多少行。",
|
||||
"settings.cmf.rowHint": "id · .cmf 文件 · 端口 · 线程 · GPU。改动在重启后生效;删除一行会停止对应的服务器。",
|
||||
"settings.cmf.addServer": "+ 添加本地模型",
|
||||
"settings.cmf.checkPort": "检查端口",
|
||||
"settings.cmf.portFree": "端口空闲",
|
||||
"settings.cmf.portBusy": "端口被占用",
|
||||
"settings.cmf.useSuggested": "使用 {port}",
|
||||
"settings.cmf.serving": "{model} 正在 :{port} 提供服务",
|
||||
"settings.cmf.intro": "用 cortiq 引擎在本地运行模型 — 即使离线或路由器故障也能工作。",
|
||||
"settings.cmf.runtime": "cortiq 运行时",
|
||||
"settings.cmf.runtimeHelp": "用于加载和运行 .cmf 模型的命令行工具 `cortiq`。启动本地服务器前先安装一次。",
|
||||
"settings.cmf.install": "安装 cortiq",
|
||||
"settings.cmf.reinstall": "重新安装 / 更新",
|
||||
"settings.cmf.installing": "安装中……(需要编译,几分钟)",
|
||||
"settings.cmf.installedOk": "cortiq 已安装",
|
||||
"settings.cmf.notInstalled": "未安装",
|
||||
"settings.cmf.addModel": "添加模型",
|
||||
"settings.cmf.addModelHelp": "模型在“导入模型”标签页下载并转换(HuggingFace → .cmf)。然后把下面的本地服务器指向该 .cmf 文件。",
|
||||
"settings.cmf.importNote": "还没有模型?去导入一个:",
|
||||
"settings.cmf.importLink": "打开导入",
|
||||
"settings.cmf.modelHelp": "要提供服务的 .cmf 文件路径(例如 models/qwen.cmf),由导入标签页生成。要为不同任务准备多个模型?导入它们并切换此路径。",
|
||||
"settings.cmf.portHelp": "本地模型服务器监听的 TCP 端口(默认 8081)。仅在与其他服务冲突时修改。",
|
||||
"settings.cmf.threadsHelp": "生成时使用多少个 CPU 核心。越多越快(相比单核约快 3 倍);0 表示交给运行时决定。",
|
||||
"settings.cmf.gpuHelp": "把矩阵运算卸载到 GPU(Metal/wgpu)。在 Apple Silicon 上主要加速长提示词 — 解码受内存带宽限制。",
|
||||
"settings.cmf.localOnlyHelp": "把每个请求都发往本地模型,忽略路由器和所有云端供应商。",
|
||||
"settings.cmf.manageHelp": "网关启动时会为下面的 .cmf 启动 `cortiq serve` 并把请求路由过去。需要已安装 cortiq(第 1 步)。重启后生效。",
|
||||
"settings.cmf.autoInstallHelp": "如果启动时缺少 cortiq,自动从 crates.io 安装。",
|
||||
"settings.cmf.autoUpdateHelp": "启动时如有更新版本,将 cortiq 升级到最新发布版。",
|
||||
"settings.router.enabledHelp": "让 cortiq 路由器对每个请求分类(任务类型 + 复杂度)并挑选模型。关闭 = 仅使用本地路由表。",
|
||||
"settings.router.verifyTlsHelp": "校验路由器的 TLS 证书。仅在使用自签名或绑定 IP 的证书时关闭。",
|
||||
"settings.telemetry.metricsHelp": "在 GET /metrics 暴露 Prometheus 指标。",
|
||||
"settings.cortiq.echoHelp": "在每个响应体中加入 `cortiq` 字段(任务、模型、成本)。响应头始终会发送。",
|
||||
"settings.stats.enabledHelp": "记录最近的请求,为仪表盘和图表提供数据。",
|
||||
"settings.cache.enabledHelp": "语义缓存:当新提示词与过去的某个非常相似时直接返回已保存的回答,跳过模型调用。需要池中有一个嵌入模型。",
|
||||
"settings.log.bodiesHelp": "记录请求/响应的完整消息体。出于隐私考虑默认关闭。",
|
||||
"settings.router.saveKey": "保存密钥",
|
||||
"settings.router.keySaved": "路由器密钥已保存",
|
||||
"settings.router.keyEmpty": "请先粘贴密钥",
|
||||
"settings.router.secretHelp": "粘贴 api.allaigate.com 邮件中的密钥,然后点击“保存密钥”。这不是“API 密钥”页中的值 — 那些是给调用你网关的客户端用的。",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — 密钥已保存(再次粘贴可替换)",
|
||||
"settings.route.strategyHelp": "发送给路由器用于分类的文本:last_user = 仅最后一条用户消息;last_user_plus_system = 最后一条用户消息 + 系统提示词;concat_all = 整段对话。",
|
||||
"settings.route.maxCharsHelp": "发送给路由器的文本最大字符数(超出会截断)。越多分类越准,但路由器调用更慢也更贵。",
|
||||
"settings.route.cacheTtlHelp": "相同请求的路由决策缓存多久后才再次询问路由器(例如 60s、5m)。",
|
||||
"settings.route.profileHelp": "请求未指定时使用的默认路由档位:cost-saver(最省钱)、balanced 或 quality-first(最好的模型)。会影响路由器挑选哪个模型。",
|
||||
|
||||
// import
|
||||
"import.subtitle": "在 HuggingFace 上搜索模型,转换为本地 .cmf 并提供服务。",
|
||||
"import.searchPlaceholder": "搜索 HuggingFace 模型(例如 qwen3、llama)……",
|
||||
"import.searching": "搜索中……",
|
||||
"import.noResults": "未找到模型。",
|
||||
"import.configure": "配置转换",
|
||||
"import.outName": "输出名称",
|
||||
"import.outNameHint": ".cmf 文件名(字母、数字、- 和 _)。",
|
||||
"import.quant": "量化",
|
||||
"import.convert": "转换并加载",
|
||||
"import.converting": "转换中",
|
||||
"import.started": "转换已开始",
|
||||
"import.failed": "转换失败 — 请查看日志。",
|
||||
"import.register": "注册为本地模型",
|
||||
"import.registered": "已注册为 “{id}”。运行 cortiq-server 即可提供服务。",
|
||||
"import.jobs": "下载与转换",
|
||||
"import.cancel": "取消",
|
||||
"import.cancelling": "正在取消 — 清理未完成的文件……",
|
||||
"import.delete": "删除",
|
||||
"import.deleteConfirm": "从磁盘删除该模型及其转换好的 .cmf 文件?",
|
||||
"import.deleted": "已连同文件一起删除",
|
||||
"import.state.running": "进行中",
|
||||
"import.state.done": "完成",
|
||||
"import.state.error": "错误",
|
||||
"import.state.cancelled": "已取消",
|
||||
"import.featuredTitle": "推荐的开箱即用 .cmf 模型",
|
||||
"import.featuredSub": "来自 HuggingFace 上 infosave 账号的预转换 .cmf 模型 — 列表实时读取,新模型会自动出现。",
|
||||
"import.readyCmfBadge": "现成 .cmf",
|
||||
"import.directDownload": "下载",
|
||||
"import.q.q8_2f": "8 位双场(𝒲×θ)— 推荐:质量与体积的最佳平衡。",
|
||||
"import.q.q8_row": "8 位按行 — 简单稳健。",
|
||||
"import.q.q4": "4 位分块 — 体积最小,质量较低。",
|
||||
"import.q.q4_tiled": "4 位分块平铺(Q4_TILED)— 在 Metal GPU 上快 1.66 倍。",
|
||||
"import.q.vbit": "可变 3–8 位 — 按专家分配位宽预算。",
|
||||
"import.q.q1t": "1.58 位三值(Q1T)— BitNet b1.58,带稀疏离群值。",
|
||||
"import.q.q1s": "1 位稀疏(Q1S)— 1 位基底,带全息离群值。",
|
||||
"import.q.q1p": "1 位 PTQ(Q1P)— 1 位,带误差扩散。",
|
||||
"import.q.q1": "1 位(Q1)— 纯二值格式。",
|
||||
"import.q.f16": "16 位 — 不量化。",
|
||||
"import.q.f32": "32 位 — 全精度(文件很大)。",
|
||||
"import.adv.title": "高级设置",
|
||||
"import.adv.auto": "自动",
|
||||
"import.adv.linearCore": "线性核",
|
||||
"import.adv.linearHint": "线性注意力层如何被保留或折叠。",
|
||||
"import.adv.nphase": "vmf_phase 每头相位数",
|
||||
"import.adv.vbitShape": "VBIT 预算形状",
|
||||
"import.adv.meanBits": "VBIT 平均位宽",
|
||||
"import.adv.shard": "分片上限(GB)",
|
||||
"import.adv.noShard": "不分片",
|
||||
"import.adv.skipMtp": "跳过 MTP 头",
|
||||
"import.adv.o1": "O(1) 注意力",
|
||||
"import.adv.o1Hint": "常量内存的流式注意力:prefill 之后释放 KV 缓存,因此长上下文不会持续增加内存。权重保持逐字节一致 — 只记录一个加载时的提示;随时可以切回(启动时加 --o1 off)。已在 Qwen3-0.6B 上验证:全部层约为 ×1.18 困惑度。",
|
||||
"import.adv.o1Off": "关闭 — 精确注意力(默认)",
|
||||
"import.adv.o1All": "all — 转换所有 softmax 层",
|
||||
"import.adv.o1Deep": "deep N — 仅最深的 N 层",
|
||||
"import.adv.o1Custom": "自定义 — 显式指定层列表",
|
||||
"import.adv.o1DeepN": "最深的 N 层",
|
||||
"import.adv.o1Layers": "层(用逗号分隔)",
|
||||
"import.adv.o1mHint": "landmark 预算(默认 32)",
|
||||
"import.adv.o1windowHint": "精确窗口宽度(默认 128)",
|
||||
"import.adv.o1sinkHint": "常驻精确 sink key 数(默认 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "初始设置",
|
||||
"onboarding.welcome": "欢迎使用 Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "选择你的语言 — 随时可以在顶栏更改。",
|
||||
"onboarding.next": "下一步",
|
||||
"onboarding.back": "上一步",
|
||||
"onboarding.skip": "暂时跳过",
|
||||
"onboarding.skipSetup": "跳过",
|
||||
"onboarding.finish": "完成设置",
|
||||
"onboarding.featuredTitle": "开箱即用的模型(一键)",
|
||||
"onboarding.featuredSub": "这些是 HuggingFace 上 infosave 发布的、已经转换好的 .cmf 模型 — 直接下载即可,无需转换。选择一个或多个即可立即开始。",
|
||||
"onboarding.download": "下载",
|
||||
"onboarding.hfTitle": "任意 HuggingFace 模型 — 自动转换",
|
||||
"onboarding.hfSub": "搜索任意模型,选择量化方式,我们会自动把它转换为 .cmf。无需手动修改配置。",
|
||||
"onboarding.noJobs": "还没有下载任务。",
|
||||
"onboarding.doneTitle": "全部就绪",
|
||||
"onboarding.doneSub": "下载好的模型会自动注册。之后可以在“导入模型”或“模型”中添加更多。",
|
||||
"onboarding.doneHint": "“完成设置”会把每个下载完成的模型接入为本地模型,启动它的 .cmf 服务器并把路由指向它。这些之后都可以在“模型”中修改。",
|
||||
"onboarding.willRegister": "将接入为本地模型",
|
||||
"onboarding.nothingToRegister": "还没有下载完成的模型 — 没有可接入的内容。返回上一步选一个模型。",
|
||||
"onboarding.registeredN": "已接入 {n} 个模型:{ids}",
|
||||
"onboarding.runtimeMissing": "cortiq 运行时尚未安装 — 请在 设置 → 本地模型 中安装后才能运行这些模型。",
|
||||
"onboarding.downloaded": "已下载",
|
||||
"onboarding.update": "更新",
|
||||
"onboarding.deleteFile": "从磁盘删除该 .cmf 文件",
|
||||
"onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。",
|
||||
"onboarding.unregistered": "同时已取消注册:{ids}",
|
||||
};
|
||||
|
||||
Vendored
+14
-1
@@ -23,6 +23,10 @@ export function clear(node) {
|
||||
while (node && node.firstChild) node.removeChild(node.firstChild);
|
||||
}
|
||||
|
||||
export function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
export function mount(node, ...children) {
|
||||
clear(node);
|
||||
for (const c of children.flat())
|
||||
@@ -46,7 +50,14 @@ export function toast(msg, kind = "info", ms = 3200) {
|
||||
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
|
||||
export function modal(title, body, onSave, saveLabel) {
|
||||
const backdrop = h("div", { class: "modal-backdrop" });
|
||||
const close = () => backdrop.remove();
|
||||
const onKeydown = (e) => {
|
||||
if (e.key === "Escape") close();
|
||||
};
|
||||
const close = () => {
|
||||
backdrop.remove();
|
||||
document.body.classList.remove("modal-open");
|
||||
document.removeEventListener("keydown", onKeydown);
|
||||
};
|
||||
backdrop.addEventListener("click", (e) => {
|
||||
if (e.target === backdrop) close();
|
||||
});
|
||||
@@ -73,6 +84,8 @@ export function modal(title, body, onSave, saveLabel) {
|
||||
)
|
||||
);
|
||||
backdrop.appendChild(card);
|
||||
document.body.classList.add("modal-open");
|
||||
document.addEventListener("keydown", onKeydown);
|
||||
document.body.appendChild(backdrop);
|
||||
return { close };
|
||||
}
|
||||
|
||||
Vendored
+191
@@ -0,0 +1,191 @@
|
||||
// Ready-to-run .cmf models, read live from the infosave HuggingFace account via
|
||||
// /setup/featured — so a newly published model shows up without a code change.
|
||||
// Shared by the first-run wizard and the Import view: same cards, same download
|
||||
// / progress / delete behaviour in both places.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export async function loadFeatured() {
|
||||
try {
|
||||
const r = await api.setupFeatured();
|
||||
return r.models || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export function quantFromVariant(name) {
|
||||
const s = (name || "").toLowerCase();
|
||||
if (s.includes("q1t")) return "Q1T";
|
||||
if (s.includes("q2tp")) return "Q2TP";
|
||||
if (s.includes("q4tp")) return "Q4TP";
|
||||
if (s.includes("q4t")) return "Q4T";
|
||||
if (s.includes("q1")) return "Q1";
|
||||
if (s.includes("q4")) return "Q4";
|
||||
if (s.includes("q8")) return "Q8";
|
||||
return "";
|
||||
}
|
||||
|
||||
// Same thresholds as the backend's format_bytes, so card and job agree.
|
||||
export function fmtBytes(n) {
|
||||
if (!n && n !== 0) return "";
|
||||
if (n >= 1e9) return (n / 1e9).toFixed(1) + " GB";
|
||||
if (n >= 1e6) return Math.round(n / 1e6) + " MB";
|
||||
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
|
||||
return n + " B";
|
||||
}
|
||||
|
||||
export function jobPct(j) {
|
||||
if (j.progress == null) return null;
|
||||
return Math.max(0, Math.min(100, Math.round(j.progress * 100)));
|
||||
}
|
||||
|
||||
/// "downloading · 42% · 1.4 / 3.2 GB" — whatever of it the job actually knows.
|
||||
/// A settled job shows its result, not the phase it stopped in.
|
||||
export function jobStatusText(j) {
|
||||
const parts = [];
|
||||
if (j.state === "running") {
|
||||
if (j.phase) parts.push(j.phase);
|
||||
const pct = jobPct(j);
|
||||
if (pct != null) parts.push(pct + "%");
|
||||
if (j.total_bytes) parts.push(`${fmtBytes(j.done_bytes || 0)} / ${fmtBytes(j.total_bytes)}`);
|
||||
} else if (j.state === "done" && j.size_bytes) {
|
||||
parts.push(fmtBytes(j.size_bytes));
|
||||
}
|
||||
return parts.join(" · ");
|
||||
}
|
||||
|
||||
export function jobBar(j) {
|
||||
const pct = jobPct(j);
|
||||
if (j.state === "done") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill ok", style: "width:100%" }));
|
||||
if (j.state !== "running") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill err", style: "width:100%" }));
|
||||
// known percentage → determinate fill; unknown (listing files) → animated slider
|
||||
return h("div", { class: "pbar" },
|
||||
pct != null
|
||||
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
|
||||
: h("div", { class: "pbar-fill run" })
|
||||
);
|
||||
}
|
||||
|
||||
/// HF gives `desc` as a bare pipeline tag ("text-generation") for live entries and
|
||||
/// a real sentence for curated ones. A one-word tag belongs in the badge row, not
|
||||
/// on a line of its own under every card.
|
||||
function descBadge(desc) {
|
||||
return desc && !desc.includes(" ") ? h("span", { class: "badge" }, desc) : null;
|
||||
}
|
||||
function descLine(desc) {
|
||||
return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null;
|
||||
}
|
||||
|
||||
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v) }.
|
||||
/// Live progress lands in the per-variant host so a poll can repaint it in place,
|
||||
/// without re-rendering the grid under the user's cursor.
|
||||
export function featuredCard(m, ctx) {
|
||||
const badge = m.badge || "CMF";
|
||||
const label = m.label || m.id;
|
||||
const desc = m.desc || "";
|
||||
const variants = m.variants && m.variants.length ? m.variants : null;
|
||||
const size = m.size || (m.size_bytes ? fmtBytes(m.size_bytes) : "");
|
||||
const isDownloaded = !!m.downloaded;
|
||||
const progressHost = (v) => {
|
||||
const host = h("div", { class: "card-prog" });
|
||||
const jid = (v && v.job) || m.job;
|
||||
if (jid) ctx.hosts.set(jid, host);
|
||||
return host;
|
||||
};
|
||||
// multi-variant (e.g. DeepSeek with parts-q2tp / parts-q4tp) → separate buttons per variant
|
||||
if (variants && variants.length > 1) {
|
||||
return h("div", { class: "hf-card featured-card" + (isDownloaded ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, isDownloaded ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
descBadge(desc),
|
||||
isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; flex-direction:column; gap:6px; margin-top:8px" },
|
||||
...variants.map((v) => {
|
||||
const q = quantFromVariant(v.name) || badge;
|
||||
const done = !!v.downloaded;
|
||||
const upd = !!v.update_available;
|
||||
const lab = done
|
||||
? (upd ? `↻ ${t("onboarding.update")} ${q} · ${v.size}` : `✓ ${q} · ${v.size}`)
|
||||
: `⬇ ${q} · ${v.size}`;
|
||||
const btn = h("button", {
|
||||
class: "btn sm " + (done && !upd ? "ghost" : "primary"),
|
||||
disabled: done && !upd ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v),
|
||||
title: v.name + (done ? " — " + t("onboarding.downloaded") : ""),
|
||||
}, lab);
|
||||
const row = h("div", { style: "display:flex; gap:6px; align-items:center; flex-wrap:wrap" }, btn);
|
||||
if (done) {
|
||||
const del = h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑");
|
||||
del.addEventListener("click", () => ctx.onDelete(m, v));
|
||||
row.appendChild(del);
|
||||
}
|
||||
return h("div", {}, row, progressHost(v));
|
||||
})
|
||||
)
|
||||
);
|
||||
}
|
||||
const v0 = variants && variants[0];
|
||||
const done0 = !!(v0 && v0.downloaded) || isDownloaded;
|
||||
const upd0 = !!(v0 && v0.update_available) || !!m.update_available;
|
||||
const q0 = quantFromVariant((v0 && v0.name) || "") || badge;
|
||||
const size0 = (v0 && v0.size) || size;
|
||||
const btn0 = h("button", {
|
||||
class: "btn sm " + (done0 && !upd0 ? "ghost" : "primary"),
|
||||
disabled: done0 && !upd0 ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v0),
|
||||
title: done0 && !upd0 ? t("onboarding.downloaded") : ""
|
||||
}, done0
|
||||
? (upd0 ? `↻ ${t("onboarding.update")} ${q0} · ${size0}` : `✓ ${q0} · ${size0}`)
|
||||
: `⬇ ${q0}${size0 ? " · " + size0 : ""}`);
|
||||
const del0 = done0 ? h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete")) : null;
|
||||
if (del0) del0.addEventListener("click", () => ctx.onDelete(m, v0));
|
||||
return h("div", { class: "hf-card featured-card" + (done0 ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, done0 ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null,
|
||||
descBadge(desc)
|
||||
),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; gap:6px; margin-top:8px; align-items:center; flex-wrap:wrap" }, btn0, del0),
|
||||
progressHost(v0)
|
||||
);
|
||||
}
|
||||
|
||||
/// Repaint the per-variant progress hosts from the current job list.
|
||||
export function paintFeaturedHosts(hosts, jobs, onCancel) {
|
||||
const byId = new Map(jobs.map((j) => [j.id, j]));
|
||||
for (const [id, host] of hosts) {
|
||||
const j = byId.get(id);
|
||||
if (!j || j.state !== "running") { mount(host); continue; }
|
||||
const cancel = h("button", { class: "btn sm ghost", style: "margin-left:auto", title: t("import.cancel") }, "✕");
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await onCancel(j); } catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
mount(host,
|
||||
jobBar(j),
|
||||
h("div", { style: "display:flex; align-items:center; gap:8px" },
|
||||
h("span", { class: "muted small" }, jobStatusText(j)),
|
||||
cancel
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Start the download for a featured variant. `local_name` is the stem the
|
||||
/// gateway itself derived, sent back verbatim so download / progress / delete
|
||||
/// can never disagree about which file this is.
|
||||
export async function startFeaturedDownload(m, v) {
|
||||
const fallbackBase = m.id.split("/").pop().toLowerCase().replace(/[^a-z0-9_-]/g, "-");
|
||||
const name = (v && v.local_name) || m.local_name || fallbackBase;
|
||||
const variant = v && v.name ? v.name : undefined;
|
||||
await api.startImport({ repo: m.id, quant: "Q1", name, variant });
|
||||
return variant;
|
||||
}
|
||||
Vendored
+93
-16
@@ -2,32 +2,36 @@
|
||||
// live progress view. Search HF, pick quantization + advanced params, watch
|
||||
// the conversion (multiple in parallel, resumable across tab switches), then
|
||||
// register the result as a local model.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { h, mount, toast, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobStatusText,
|
||||
} from "./featured.js";
|
||||
|
||||
const QUANTS = [
|
||||
["Q8_2F", "import.q.q8_2f"],
|
||||
["Q8_ROW", "import.q.q8_row"],
|
||||
["Q4_BLOCK", "import.q.q4"],
|
||||
["Q4_TILED", "import.q.q4_tiled"],
|
||||
["VBIT", "import.q.vbit"],
|
||||
["Q1T", "import.q.q1t"],
|
||||
["Q1S", "import.q.q1s"],
|
||||
["Q1P", "import.q.q1p"],
|
||||
["Q1", "import.q.q1"],
|
||||
["F16", "import.q.f16"],
|
||||
["F32", "import.q.f32"],
|
||||
];
|
||||
|
||||
// A small "?" badge with a hover/focus tooltip (same style as Settings).
|
||||
function help(text) {
|
||||
return text ? h("span", { class: "help", title: text, tabindex: "0", role: "img", "aria-label": text }, "?") : null;
|
||||
}
|
||||
function field(label, control, hint, helpText) {
|
||||
return h("label", { class: "field" },
|
||||
h("span", {}, label, helpText ? " " : null, help(helpText)),
|
||||
control,
|
||||
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control,
|
||||
hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
function fmtNum(n) {
|
||||
if (n == null) return "—";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
|
||||
@@ -50,6 +54,10 @@ export function renderImport() {
|
||||
let selected = null;
|
||||
let lastModels = []; // cache of the last search, so re-highlighting needs no refetch
|
||||
let jobsTimer = null;
|
||||
let featured = []; // ready-made .cmf models from the infosave HF account
|
||||
let featuredHosts = new Map(); // job id → live progress slot inside a featured card
|
||||
let lastJobs = [];
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
|
||||
const searchIn = h("input", {
|
||||
class: "hf-search",
|
||||
@@ -147,9 +155,9 @@ export function renderImport() {
|
||||
field(t("import.adv.o1DeepN"), o1DeepIn),
|
||||
field(t("import.adv.o1Layers"), o1CustomIn));
|
||||
const o1Knobs = h("div", { class: "row", style: "display:none" },
|
||||
field(t("import.adv.o1mLabel"), o1M, t("import.adv.o1mHint"), t("import.adv.o1mHelp")),
|
||||
field(t("import.adv.o1windowLabel"), o1Window, t("import.adv.o1windowHint"), t("import.adv.o1windowHelp")),
|
||||
field(t("import.adv.o1sinkLabel"), o1Sink, t("import.adv.o1sinkHint"), t("import.adv.o1sinkHelp")));
|
||||
field("--o1-m", o1M, t("import.adv.o1mHint")),
|
||||
field("--o1-window", o1Window, t("import.adv.o1windowHint")),
|
||||
field("--o1-sink", o1Sink, t("import.adv.o1sinkHint")));
|
||||
o1Sel.addEventListener("change", () => {
|
||||
const on = !!o1Sel.value;
|
||||
o1Knobs.style.display = on ? "" : "none";
|
||||
@@ -251,6 +259,12 @@ export function renderImport() {
|
||||
}
|
||||
// Load + render all jobs. Runs on mount (before `root` is attached — so no
|
||||
// isConnected guard here) and on each poll tick.
|
||||
async function cancelJob(j) {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "warn");
|
||||
await refreshJobs();
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
let jobs = [];
|
||||
try {
|
||||
@@ -259,7 +273,14 @@ export function renderImport() {
|
||||
return;
|
||||
}
|
||||
jobs = jobs.filter((j) => j.state !== "cancelled"); // hide user-cancelled jobs
|
||||
lastJobs = jobs;
|
||||
renderJobs(jobs);
|
||||
// A settled download changes what the featured cards offer (✓ + delete), so
|
||||
// re-read the list on that transition — not on every poll tick.
|
||||
const settled = jobs.some((j) => jobStates.get(j.id) === "running" && j.state !== "running");
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) await renderFeatured();
|
||||
else paintFeaturedHosts(featuredHosts, jobs, cancelJob);
|
||||
if (jobs.some((j) => j.state === "running")) ensurePolling();
|
||||
else stopPolling();
|
||||
}
|
||||
@@ -286,7 +307,7 @@ export function renderImport() {
|
||||
(running && pct != null ? " · " + pct + "%" : "") +
|
||||
(j.size_bytes ? " · " + fmtBytes(j.size_bytes) : "")
|
||||
);
|
||||
const phase = h("div", { class: "muted small" }, j.phase || "");
|
||||
const phase = h("div", { class: "muted small" }, jobStatusText(j));
|
||||
const logEl = h("pre", { class: "conv-log" }, (j.log || []).slice(-10).join("\n"));
|
||||
|
||||
const actions = [];
|
||||
@@ -364,6 +385,59 @@ export function renderImport() {
|
||||
);
|
||||
}
|
||||
|
||||
// ── featured: the infosave account, read live so new releases just show up ──
|
||||
const featuredGrid = h("div", { class: "hf-grid" }, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
const reloadBtn = h("button", { class: "btn ghost sm" }, "↻ " + t("common.refresh"));
|
||||
const featuredHost = h(
|
||||
"div",
|
||||
{ class: "card featured-section" },
|
||||
h("div", { class: "card-head", style: "display:flex; align-items:center; gap:10px" },
|
||||
h("div", { class: "card-title", style: "margin:0" }, "⚡ " + t("import.featuredTitle")),
|
||||
h("div", { style: "flex:1" }),
|
||||
reloadBtn),
|
||||
h("div", { class: "muted small", style: "margin:2px 0 10px" }, t("import.featuredSub")),
|
||||
featuredGrid
|
||||
);
|
||||
|
||||
async function startFeatured(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function deleteFeatured(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!window.confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await renderFeatured();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function renderFeatured(reload = true) {
|
||||
if (reload) featured = await loadFeatured();
|
||||
featuredHosts = new Map();
|
||||
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured };
|
||||
mount(featuredGrid, ...(featured.length
|
||||
? featured.map((m) => featuredCard(m, ctx))
|
||||
: [h("div", { class: "muted pad" }, t("import.noResults"))]));
|
||||
paintFeaturedHosts(featuredHosts, lastJobs, cancelJob);
|
||||
}
|
||||
reloadBtn.addEventListener("click", async () => {
|
||||
reloadBtn.disabled = true;
|
||||
mount(featuredGrid, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
await renderFeatured();
|
||||
reloadBtn.disabled = false;
|
||||
});
|
||||
|
||||
const root = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -371,10 +445,13 @@ export function renderImport() {
|
||||
h("h2", {}, t("nav.import")),
|
||||
h("div", { class: "muted" }, t("import.subtitle"))),
|
||||
jobsHost, // progress panel on top, above the search
|
||||
featuredHost, // featured ready-to-run models
|
||||
h("div", { class: "card" }, searchIn, results),
|
||||
formHost
|
||||
);
|
||||
doSearch(); // initial trending list (mounts into `results`)
|
||||
renderFeatured(); // ready-made .cmf models, fetched live
|
||||
refreshJobs(); // resume: show any running/recent conversions immediately
|
||||
return root;
|
||||
}
|
||||
|
||||
|
||||
Vendored
+112
-47
@@ -1,5 +1,5 @@
|
||||
// Models — pool table with add/edit modal, probe, and per-model secret entry.
|
||||
import { h, mount, modal, toast, money } from "../ui.js";
|
||||
import { h, mount, modal, toast, confirmDialog, money, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
@@ -7,9 +7,6 @@ import { appState } from "../app.js";
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
function keyBadge(src) {
|
||||
const map = { store: "store", env: "env", missing: "bad", none: "" };
|
||||
@@ -54,7 +51,16 @@ function modelForm(meta, existing) {
|
||||
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
|
||||
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
|
||||
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
|
||||
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
|
||||
const CAP_EMOJIS = {
|
||||
tools: "🔧 tools",
|
||||
vision: "👁️ vision",
|
||||
audio: "🎙️ audio",
|
||||
reasoning: "🧠 reasoning",
|
||||
json: "📄 json",
|
||||
caching: "⚡ caching"
|
||||
};
|
||||
|
||||
const capInputs = (meta.caps || Object.keys(CAP_EMOJIS)).map((c) => {
|
||||
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
|
||||
return { c, cb };
|
||||
});
|
||||
@@ -73,25 +79,55 @@ function modelForm(meta, existing) {
|
||||
capInputs.forEach(({ c, cb }) => { cb.checked = d.caps.includes(c); });
|
||||
keyIn.placeholder = d.needsKey ? t("models.form.apiKeyPlaceholder") : t("models.form.apiKeyNotNeeded");
|
||||
mount(keyStatus, "");
|
||||
// Automatically pre-fetch models (especially useful for OpenRouter which doesn't need a key)
|
||||
testKey(true);
|
||||
}
|
||||
providerSel.addEventListener("change", applyDefaults);
|
||||
if (!existing) applyDefaults();
|
||||
|
||||
// Auto-fill the id from the model name (until the user edits id by hand).
|
||||
if (!existing) {
|
||||
modelIn.addEventListener("input", () => {
|
||||
if (!idIn.value || idIn._auto) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
let cachedModels = [];
|
||||
|
||||
const autofillData = () => {
|
||||
console.log("autofillData triggered. modelIn.value:", modelIn.value);
|
||||
console.log("cachedModels length:", cachedModels.length);
|
||||
if (!existing && (!idIn.value || idIn._auto)) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
console.log("auto-filled idIn:", idIn.value);
|
||||
}
|
||||
const found = cachedModels.find((m) => (m.id || m) === modelIn.value);
|
||||
console.log("found model:", found);
|
||||
if (found) {
|
||||
if (found.price_in !== undefined) {
|
||||
priceInIn.value = parseFloat(Number(found.price_in).toFixed(4));
|
||||
priceOutIn.value = parseFloat(Number(found.price_out).toFixed(4));
|
||||
console.log("Updated prices:", priceInIn.value, priceOutIn.value);
|
||||
}
|
||||
});
|
||||
if (found.caps && Array.isArray(found.caps)) {
|
||||
capInputs.forEach(({ c, cb }) => {
|
||||
cb.checked = found.caps.includes(c);
|
||||
});
|
||||
console.log("Updated caps:", found.caps);
|
||||
}
|
||||
}
|
||||
};
|
||||
modelIn.addEventListener("input", autofillData);
|
||||
modelIn.addEventListener("change", autofillData);
|
||||
|
||||
if (!existing) {
|
||||
idIn.addEventListener("input", () => { idIn._auto = false; });
|
||||
} else {
|
||||
console.log("Editing existing model, fetching testKey(true)");
|
||||
testKey(true);
|
||||
}
|
||||
|
||||
async function testKey() {
|
||||
if (!baseUrlIn.value.trim()) { mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?")); return; }
|
||||
async function testKey(silent = false) {
|
||||
if (!baseUrlIn.value.trim()) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?"));
|
||||
return;
|
||||
}
|
||||
testBtn.disabled = true;
|
||||
mount(keyStatus, t("models.form.testing"));
|
||||
if (!silent) mount(keyStatus, t("models.form.testing"));
|
||||
try {
|
||||
const r = await api.providerModels({
|
||||
provider: providerSel.value,
|
||||
@@ -100,24 +136,61 @@ function modelForm(meta, existing) {
|
||||
api_key_env: envFor(),
|
||||
});
|
||||
if (r.ok) {
|
||||
dl.replaceChildren(...(r.models || []).map((mm) => h("option", { value: mm })));
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, "✓ " + t("models.form.foundModels", { n: r.count })));
|
||||
modelIn.focus();
|
||||
cachedModels = r.models || [];
|
||||
dl.replaceChildren(...cachedModels.map((mm) => h("option", { value: mm.id || mm })));
|
||||
|
||||
let msg = "✓ " + t("models.form.foundModels", { n: r.count });
|
||||
if (r.balance !== undefined) {
|
||||
msg += ` (Balance: $${r.balance.toFixed(4)})`;
|
||||
}
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, msg));
|
||||
if (!silent) modelIn.focus();
|
||||
} else {
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
}
|
||||
} catch (e) {
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
}
|
||||
testBtn.disabled = false;
|
||||
}
|
||||
testBtn.addEventListener("click", testKey);
|
||||
testBtn.addEventListener("click", () => testKey(false));
|
||||
keyIn.addEventListener("change", () => testKey(true));
|
||||
|
||||
const keyHint =
|
||||
existing && existing.key_source === "store" ? t("models.form.keyStored")
|
||||
: existing && existing.key_source === "env" ? t("models.form.keyFromEnv")
|
||||
: t("models.form.apiKeyHint");
|
||||
|
||||
// ⚙ Generation & Thinking Parameters
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: m.temperature ?? "" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: m.top_p ?? "" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: m.max_tokens ?? "" });
|
||||
const thinkIn = h("input", { type: "number", min: "0", placeholder: "1024", value: m.think_budget ?? "" });
|
||||
const o1Sel = h("select", {},
|
||||
h("option", { value: "" }, t("import.adv.auto")),
|
||||
h("option", { value: "all" }, "all — O(1) Attention"),
|
||||
h("option", { value: "deep" }, "deep — O(1) Deep"),
|
||||
h("option", { value: "off" }, "off — Standard"));
|
||||
if (m.o1) o1Sel.value = m.o1;
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: m.skip_mtp ? true : null });
|
||||
const sysPromptIn = h("textarea", { rows: 2, placeholder: t("models.form.systemPromptPlaceholder"), value: m.system_prompt || "" });
|
||||
|
||||
const advParams = h(
|
||||
"details",
|
||||
{ class: "adv-params", style: "margin-top:12px;padding:10px;border:1px solid var(--border);border-radius:6px;" },
|
||||
h("summary", { style: "font-weight:600;cursor:pointer;" }, "⚙ " + t("models.form.paramsTitle")),
|
||||
h("div", { class: "row", style: "margin-top:8px" },
|
||||
field(t("models.form.temperature"), tempIn, t("models.form.temperatureHint")),
|
||||
field(t("models.form.topP"), topPIn, t("models.form.topPHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("models.form.maxTokens"), maxTokIn, t("models.form.maxTokensHint")),
|
||||
field(t("models.form.thinkBudget"), thinkIn, t("models.form.thinkBudgetHint"))),
|
||||
h("div", { class: "row" },
|
||||
field("O(1) Attention", o1Sel, t("import.adv.o1Hint")),
|
||||
field("MTP", h("label", { class: "check" }, skipMtpCb, t("import.adv.skipMtp")))),
|
||||
field(t("models.form.systemPrompt"), sysPromptIn, t("models.form.systemPromptHint"))
|
||||
);
|
||||
|
||||
const node = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -128,7 +201,8 @@ function modelForm(meta, existing) {
|
||||
dl,
|
||||
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
|
||||
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c))))
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap", style: "gap:10px" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check", style: "margin:0" }, cb, CAP_EMOJIS[c] || c)))),
|
||||
advParams
|
||||
);
|
||||
|
||||
const getValue = () => {
|
||||
@@ -147,6 +221,13 @@ function modelForm(meta, existing) {
|
||||
price_out: parseFloat(priceOutIn.value) || 0,
|
||||
api_key_env: hasKey ? keyEnv : null,
|
||||
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: thinkIn.value !== "" ? parseInt(thinkIn.value) : null,
|
||||
o1: o1Sel.value || null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysPromptIn.value.trim() || null,
|
||||
},
|
||||
secret: typedKey,
|
||||
keyEnv,
|
||||
@@ -280,31 +361,15 @@ function row(m, meta, reload) {
|
||||
"button",
|
||||
{
|
||||
class: "btn sm danger",
|
||||
onClick: () => {
|
||||
// local models may have a .cmf on disk — let the user decide its fate
|
||||
const isLocal = m.managed || /127\.0\.0\.1|localhost/.test(m.base_url || "");
|
||||
const fileCb = h("input", { type: "checkbox", checked: isLocal ? true : null });
|
||||
const body = h(
|
||||
"div",
|
||||
{},
|
||||
h("p", {}, t("models.deleteConfirm", { id: m.id })),
|
||||
isLocal
|
||||
? h("label", { class: "check" }, fileCb, t("models.deleteFile"))
|
||||
: null
|
||||
);
|
||||
modal(t("common.delete") + ": " + m.id, body, async () => {
|
||||
try {
|
||||
const r = await api.deleteModel(m.id, isLocal && fileCb.checked);
|
||||
toast(
|
||||
r.file_removed ? t("models.deletedWithFile") : t("toast.deleted"),
|
||||
"good"
|
||||
);
|
||||
await reload();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
return false;
|
||||
}
|
||||
}, t("common.delete"));
|
||||
onClick: async () => {
|
||||
if (!(await confirmDialog(t("models.deleteConfirm", { id: m.id })))) return;
|
||||
try {
|
||||
await api.deleteModel(m.id);
|
||||
toast(t("toast.deleted"), "good");
|
||||
await reload();
|
||||
} catch (e) {
|
||||
toast(String(e.message || e), "bad");
|
||||
}
|
||||
},
|
||||
},
|
||||
t("common.delete")
|
||||
|
||||
Vendored
+332
@@ -0,0 +1,332 @@
|
||||
// First-run onboarding wizard: language → featured .cmf models (infosave) → HF search+convert → done.
|
||||
// No manual TOML edits: every action goes through the admin API.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t, getLang, setLang, LANGS } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured as fetchFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobBar,
|
||||
jobStatusText,
|
||||
fmtBytes,
|
||||
} from "./featured.js";
|
||||
|
||||
let step = 0; // 0 lang, 1 featured, 2 hf convert, 3 done
|
||||
let featured = [];
|
||||
let featuredLoaded = false;
|
||||
|
||||
function langStep(onNext) {
|
||||
const cards = LANGS.map((l) =>
|
||||
h("button", {
|
||||
class: "onb-lang" + (getLang() === l.code ? " sel" : ""),
|
||||
onClick: () => {
|
||||
setLang(l.code);
|
||||
cards.forEach((c) => c.classList.toggle("sel", c.dataset.code === l.code));
|
||||
toast(l.flag + " " + l.label, "ok");
|
||||
},
|
||||
"data-code": l.code,
|
||||
title: l.label,
|
||||
"aria-label": l.label,
|
||||
}, h("span", { class: "flag" }, l.flag), h("b", {}, l.label))
|
||||
);
|
||||
return h("div", { class: "onb-step" },
|
||||
h("h2", {}, t("onboarding.welcome")),
|
||||
h("p", { class: "muted" }, t("onboarding.welcomeSub")),
|
||||
h("div", { class: "onb-langs" }, ...cards),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn primary", onClick: onNext }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
export function renderOnboarding() {
|
||||
const root = h("div", { class: "onb-wrap" });
|
||||
let jobsPoll = null;
|
||||
let jobs = [];
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
let hosts = new Map(); // job id → card progress host (rebuilt on every render)
|
||||
let jobsHost = h("div", {});
|
||||
|
||||
// Job rows: bar, live status, and the actions that finish or undo a download.
|
||||
function jobsMini(list) {
|
||||
if (!list.length) return h("div", { class: "muted small" }, t("onboarding.noJobs"));
|
||||
return h("div", {},
|
||||
...list.map((j) => {
|
||||
const act = [];
|
||||
if (j.state === "done") {
|
||||
const btn = h("button", { class: "btn primary sm" }, "✓ " + t("import.register"));
|
||||
btn.addEventListener("click", async () => {
|
||||
btn.disabled = true;
|
||||
try {
|
||||
const r = await api.registerImport(j.id);
|
||||
toast(t("import.registered", { id: r.model_id }), "ok");
|
||||
} catch (e) { toast(e.message, "err"); btn.disabled = false; }
|
||||
});
|
||||
act.push(btn);
|
||||
}
|
||||
if (j.state === "running") {
|
||||
const cancel = h("button", { class: "btn ghost sm" }, "✕ " + t("import.cancel"));
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await api.cancelImport(j.id); toast(t("import.cancelling"), "ok"); await refreshJobs(); }
|
||||
catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
act.push(cancel);
|
||||
} else {
|
||||
// done / error / cancelled — remove the .cmf from disk and redo the step
|
||||
const del = h("button", { class: "btn ghost sm danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete"));
|
||||
del.addEventListener("click", () => deleteJob(j));
|
||||
act.push(del);
|
||||
}
|
||||
const known = ["running", "done", "error", "cancelled"].includes(j.state);
|
||||
return h("div", { class: "job", style: "margin:8px 0" },
|
||||
h("div", { class: "job-head" },
|
||||
h("b", {}, j.name || j.repo || j.id),
|
||||
h("span", { class: "badge" + (j.state === "done" ? " ok" : j.state === "running" ? "" : " bad") }, known ? t("import.state." + j.state) : j.state)
|
||||
),
|
||||
jobBar(j),
|
||||
h("div", { class: "muted small" }, jobStatusText(j)),
|
||||
h("pre", { class: "conv-log", style: "max-height:80px" }, (j.log || []).slice(-6).join("\n")),
|
||||
act.length ? h("div", { class: "actions" }, ...act) : null
|
||||
);
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
// Repaint only the live bits (card hosts + job list) — cheap enough for 1.5s.
|
||||
function paintJobs() {
|
||||
mount(jobsHost, jobs.length
|
||||
? h("div", { class: "card" }, h("div", { class: "card-title" }, t("import.jobs")), jobsMini(jobs))
|
||||
: h("div", {}));
|
||||
paintFeaturedHosts(hosts, jobs, async (j) => {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "ok");
|
||||
await refreshJobs();
|
||||
});
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
try { jobs = (await api.listImports()).jobs || []; } catch { jobs = []; }
|
||||
// A job that just settled changes what the cards offer (✓ + delete), so the
|
||||
// featured list has to be re-read — but only on the transition, not per poll.
|
||||
const settled = jobs.some((j) => {
|
||||
const prev = jobStates.get(j.id);
|
||||
return prev === "running" && j.state !== "running";
|
||||
});
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) {
|
||||
await loadFeatured();
|
||||
render(); // rebuilds the cards (✓ + delete) and repaints the job rows
|
||||
} else {
|
||||
paintJobs();
|
||||
}
|
||||
if (jobs.some((j) => j.state === "running")) {
|
||||
if (!jobsPoll) jobsPoll = setInterval(refreshJobs, 1500);
|
||||
} else {
|
||||
if (jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; }
|
||||
}
|
||||
}
|
||||
|
||||
async function loadFeatured() {
|
||||
featured = await fetchFeatured();
|
||||
featuredLoaded = true;
|
||||
}
|
||||
|
||||
function topNav(prev, next, extra) {
|
||||
return h("div", { class: "actions", style: "margin-bottom:12px; position:sticky; top:0; background:var(--bg); z-index:2; padding:8px 0; border-bottom:1px solid var(--border-soft); backdrop-filter:blur(4px)" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
extra ? extra : h("div", { style: "flex:1" }),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
}
|
||||
|
||||
async function startDownload(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function handleDelete(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
// Delete straight from a job row: the gateway knows that job's output path.
|
||||
async function deleteJob(j) {
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: j.name || j.repo }))) return;
|
||||
try {
|
||||
await api.deleteImport(j.id);
|
||||
toast(t("import.deleted"), "ok");
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
function render() {
|
||||
const dots = [0, 1, 2, 3].map((i) => h("span", { class: "dot" + (i === step ? " on" : i < step ? " done" : "") }));
|
||||
const header = h("div", { class: "onb-header" },
|
||||
h("div", { class: "onb-dots" }, ...dots),
|
||||
h("div", { class: "onb-title" }, t("onboarding.title") + ` — ${step + 1}/4`)
|
||||
);
|
||||
|
||||
let body;
|
||||
const next = () => { step = Math.min(3, step + 1); render(); };
|
||||
const prev = () => { step = Math.max(0, step - 1); render(); };
|
||||
hosts = new Map(); // rebuilt by featuredCard below
|
||||
|
||||
if (step === 0) {
|
||||
const top = h("div", { class: "actions", style: "margin-bottom:12px; justify-content:flex-end" },
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
body = h("div", {},
|
||||
top,
|
||||
langStep(next)
|
||||
);
|
||||
} else if (step === 1) {
|
||||
const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip")));
|
||||
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete };
|
||||
// the list is fetched from HuggingFace — say so instead of showing a blank grid
|
||||
const grid = featured.length
|
||||
? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx)))
|
||||
: h("div", { class: "muted pad" }, featuredLoaded ? t("import.noResults") : t("import.searching"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
top,
|
||||
h("h2", {}, "📦 " + t("onboarding.featuredTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.featuredSub")),
|
||||
grid,
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn", onClick: async () => { await loadFeatured(); render(); await refreshJobs(); } }, t("common.refresh")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →"),
|
||||
h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip"))
|
||||
)
|
||||
);
|
||||
} else if (step === 2) {
|
||||
const top2 = topNav(prev, next);
|
||||
// lightweight HF search inline
|
||||
const inp = h("input", { placeholder: t("import.searchPlaceholder"), style: "width:100%" });
|
||||
const res = h("div", { class: "hf-grid" });
|
||||
const quantSel = h("select", {},
|
||||
h("option", { value: "Q8_2F" }, "Q8_2F — " + t("import.q.q8_2f")),
|
||||
h("option", { value: "Q4_BLOCK" }, "Q4 — " + t("import.q.q4")),
|
||||
h("option", { value: "Q1T" }, "Q1T — " + t("import.q.q1t")),
|
||||
h("option", { value: "Q1" }, "Q1 — " + t("import.q.q1")),
|
||||
);
|
||||
let selected = null;
|
||||
let debounce = null;
|
||||
async function doSearch() {
|
||||
const q = inp.value.trim();
|
||||
mount(res, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
try {
|
||||
const r = await api.hfSearch(q, 16);
|
||||
const models = r.models || [];
|
||||
if (!models.length) { mount(res, h("div", { class: "muted pad" }, t("import.noResults"))); return; }
|
||||
mount(res, ...models.map((m) => {
|
||||
const card = h("div", { class: "hf-card" + (selected && selected.id === m.id ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, m.id),
|
||||
h("div", { class: "hf-meta" }, h("span", { class: "badge" }, m.pipeline_tag || "model"), h("span", { class: "muted" }, "♥ " + (m.likes ?? 0)))
|
||||
);
|
||||
card.addEventListener("click", () => { selected = m; doSearch(); });
|
||||
return card;
|
||||
}));
|
||||
} catch (e) { mount(res, h("div", { class: "err" }, String(e.message))); }
|
||||
}
|
||||
inp.addEventListener("input", () => { clearTimeout(debounce); debounce = setTimeout(doSearch, 300); });
|
||||
const go = h("button", { class: "btn primary" }, "⬇ " + t("import.convert"));
|
||||
go.addEventListener("click", async () => {
|
||||
if (!selected) { toast(t("import.noResults"), "warn"); return; }
|
||||
go.disabled = true;
|
||||
try {
|
||||
await api.startImport({ repo: selected.id, quant: quantSel.value, name: selected.id.split("/").pop().toLowerCase() });
|
||||
toast(t("import.started"), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
go.disabled = false;
|
||||
});
|
||||
// auto initial search
|
||||
setTimeout(doSearch, 100);
|
||||
body = h("div", { class: "onb-step" },
|
||||
top2,
|
||||
h("h2", {}, "🔍 " + t("onboarding.hfTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.hfSub")),
|
||||
h("div", { class: "card" }, inp, res),
|
||||
h("div", { class: "card", style: "margin-top:12px" },
|
||||
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("import.quant")), quantSel)),
|
||||
h("div", { class: "actions" }, go)
|
||||
),
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
} else {
|
||||
// done — finishing connects every completed download as a local model
|
||||
const ready = jobs.filter((j) => j.state === "done");
|
||||
const doneBtn = h("button", { class: "btn primary" }, t("onboarding.finish"));
|
||||
doneBtn.addEventListener("click", async () => {
|
||||
doneBtn.disabled = true;
|
||||
let r = null;
|
||||
try { r = await api.setupComplete(getLang(), true); } catch (e) { toast(e.message, "err"); }
|
||||
if (r && r.registered && r.registered.length) {
|
||||
toast(t("onboarding.registeredN", { n: r.registered.length, ids: r.registered.join(", ") }), "ok");
|
||||
if (!r.runtime_installed) toast(t("onboarding.runtimeMissing"), "warn");
|
||||
// let the toast be readable before the reload swaps the page
|
||||
await new Promise((res) => setTimeout(res, 900));
|
||||
}
|
||||
location.hash = "#/models";
|
||||
location.reload();
|
||||
});
|
||||
const skipBtn = h("button", { class: "btn ghost", onClick: async () => {
|
||||
try { await api.setupComplete(getLang(), false); } catch {}
|
||||
location.hash = "#/dashboard";
|
||||
} }, t("onboarding.skipSetup"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
h("h2", {}, "✅ " + t("onboarding.doneTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.doneSub")),
|
||||
ready.length
|
||||
? h("div", { class: "card" },
|
||||
h("div", { class: "card-title" }, t("onboarding.willRegister")),
|
||||
...ready.map((j) => h("div", { class: "muted small" }, "• cmf-" + (j.name || j.repo) + (j.size_bytes ? ` · ${fmtBytes(j.size_bytes)}` : "")))
|
||||
)
|
||||
: h("div", { class: "muted small" }, t("onboarding.nothingToRegister")),
|
||||
jobsHost,
|
||||
h("div", { class: "callout" }, t("onboarding.doneHint")),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
skipBtn,
|
||||
doneBtn
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
mount(root, h("div", { class: "onb" }, header, body));
|
||||
paintJobs(); // fill the freshly built hosts with whatever we already know
|
||||
if (jobs.some((j) => j.state === "running") && !jobsPoll) {
|
||||
jobsPoll = setInterval(refreshJobs, 1500);
|
||||
}
|
||||
}
|
||||
|
||||
loadFeatured().then(() => { render(); refreshJobs(); });
|
||||
render();
|
||||
// cleanup poll on detach
|
||||
const obs = new MutationObserver(() => { if (!root.isConnected && jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; obs.disconnect(); } });
|
||||
obs.observe(document.body, { childList: true, subtree: true });
|
||||
return root;
|
||||
}
|
||||
Vendored
+98
-14
@@ -234,12 +234,6 @@ export async function renderSettings() {
|
||||
const portIn = h("input", { type: "number", value: sv.port ?? 8090, style: "width:82px" });
|
||||
const threadsIn = h("input", { type: "number", min: 0, value: sv.threads ?? 8, style: "width:60px", title: t("settings.cmf.threadsHelp") });
|
||||
const gpuIn = h("input", { type: "checkbox", checked: sv.gpu ? true : null, title: t("settings.cmf.gpuHelp") });
|
||||
// serve-time O(1) override: off cures a too-aggressive --o1 conversion
|
||||
const o1Opts = ["", "off", "all", "deep4", "deep8", "deep12"];
|
||||
const o1In = h("select", { style: "width:110px", title: t("settings.cmf.o1Help") },
|
||||
...o1Opts.map((v) => h("option", { value: v, selected: (sv.o1 || "") === v ? true : null },
|
||||
v === "" ? t("settings.cmf.o1Auto") : v)));
|
||||
if (sv.o1 && !o1Opts.includes(sv.o1)) o1In.appendChild(h("option", { value: sv.o1, selected: true }, sv.o1));
|
||||
const checkBtn = h("button", { class: "btn small", type: "button" }, t("settings.cmf.checkPort"));
|
||||
const removeBtn = h("button", { class: "btn small danger", type: "button", title: t("common.delete") }, "✕");
|
||||
const st = h("span", { class: "small" });
|
||||
@@ -251,11 +245,7 @@ export async function renderSettings() {
|
||||
try {
|
||||
const r = await api.cmfPortCheck(p);
|
||||
if (r.available) mount(st, h("span", { style: "color:var(--good)" }, "✓ " + t("settings.cmf.portFree")));
|
||||
else if (r.managed) {
|
||||
// held by this gateway's own managed server — replacing the model on
|
||||
// this port is fine, it frees up on restart
|
||||
mount(st, h("span", { style: "color:var(--accent)" }, "ℹ " + t("settings.cmf.portOwn")));
|
||||
} else {
|
||||
else {
|
||||
const kids = [h("span", { style: "color:var(--bad)" }, "✗ " + t("settings.cmf.portBusy"))];
|
||||
if (r.suggested) { const b = h("button", { class: "btn small", type: "button" }, t("settings.cmf.useSuggested", { port: r.suggested })); b.addEventListener("click", () => { portIn.value = r.suggested; mount(st, ""); }); kids.push(b); }
|
||||
mount(st, h("span", {}, ...kids));
|
||||
@@ -263,9 +253,103 @@ export async function renderSettings() {
|
||||
} catch (e) { mount(st, String(e.message || e)); }
|
||||
checkBtn.disabled = false;
|
||||
});
|
||||
const row = h("div", { class: "flex wrap", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), o1In, removeBtn, st);
|
||||
const entry = { node: row, read: () => ({ id: idIn.value.trim() || autoId(modelSel.value), model: modelSel.value.trim(), port: parseInt(portIn.value) || 8090, threads: parseInt(threadsIn.value) || 0, gpu: gpuIn.checked, o1: o1In.value }) };
|
||||
|
||||
// ── O(1) Nyström Attention ──
|
||||
// NOT a "reasoning mode". This is a memory-efficient streaming attention
|
||||
// kernel (Nyström/landmark) that replaces standard KV-cache attention,
|
||||
// giving O(1) memory per decode step instead of O(n).
|
||||
const o1Sel = h("select", { style: "width:120px", title: "O(1) Nyström attention layer spec: which layers replace KV-cache with streaming Nyström attention" },
|
||||
h("option", { value: "" }, "O(1) авто"),
|
||||
h("option", { value: "all" }, "O(1) все слои"),
|
||||
h("option", { value: "deep" }, "O(1) глубокие"),
|
||||
h("option", { value: "off" }, "O(1) выкл."));
|
||||
if (sv.o1) o1Sel.value = sv.o1;
|
||||
const o1mIn = h("input", { type: "number", min: "4", placeholder: "32", value: sv.o1_m ?? "", style: "width:55px", title: "Landmark budget (≥4, default 32). More landmarks ≠ better: m=64 measured worse." });
|
||||
const o1wIn = h("input", { type: "number", min: "1", placeholder: "128", value: sv.o1_window ?? "", style: "width:60px", title: "Exact-window width — main quality lever (default 128)" });
|
||||
const o1sinkIn = h("input", { type: "number", min: "0", placeholder: "4", value: sv.o1_sink ?? "", style: "width:50px", title: "Permanent exact sink keys, StreamingLLM discipline (default 4)" });
|
||||
|
||||
// ── Generation parameters ──
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: sv.temperature ?? "", style: "width:60px" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: sv.top_p ?? "", style: "width:60px" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: sv.max_tokens ?? "", style: "width:75px" });
|
||||
// think_budget = 0 means reasoning disabled (enable_thinking=false);
|
||||
// null = model decides; N>0 = token budget for the <think> block.
|
||||
const noThinkCb = h("input", { type: "checkbox", checked: (sv.think_budget === 0) ? true : null, title: "Отключить режим размышлений (think_budget=0, enable_thinking=false для Qwen3/3.5)" });
|
||||
const thinkIn = h("input", { type: "number", min: "1", placeholder: "авто", value: (sv.think_budget && sv.think_budget > 0) ? sv.think_budget : "", style: "width:75px", title: "Бюджет токенов на размышления (think_budget). Пустое = модель решает сама." });
|
||||
if (sv.think_budget === 0) thinkIn.disabled = true;
|
||||
noThinkCb.addEventListener("change", () => {
|
||||
thinkIn.disabled = noThinkCb.checked;
|
||||
if (noThinkCb.checked) thinkIn.value = "";
|
||||
});
|
||||
const sysIn = h("input", { placeholder: "System prompt (optional)", value: sv.system_prompt || "", style: "flex:1;min-width:200px" });
|
||||
|
||||
// ── MTP (Multi-Token Prediction): speculative decoding, NOT reasoning ──
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: sv.skip_mtp ? true : null, title: "Disable MTP speculative decoding (env CMF_MTP=0)" });
|
||||
|
||||
// ── Settings drawer ──
|
||||
const drawerStyle = "display:none;width:100%;margin-top:6px;padding:10px 12px;background:var(--bg-subtle);border-radius:6px;border:1px solid var(--border)";
|
||||
const groupStyle = "display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:4px 0";
|
||||
const labelStyle = "font-size:11px;color:var(--text-muted);font-weight:600;text-transform:uppercase;letter-spacing:0.5px;width:100%;margin-bottom:2px";
|
||||
|
||||
const drawer = h("div", { style: drawerStyle },
|
||||
// Group 1: O(1) Nyström Attention
|
||||
h("div", { style: labelStyle }, "O(1) Nyström Attention"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Слои:"), o1Sel,
|
||||
h("span", { class: "small muted" }, "m:"), o1mIn,
|
||||
h("span", { class: "small muted" }, "Window:"), o1wIn,
|
||||
h("span", { class: "small muted" }, "Sink:"), o1sinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 2: Generation
|
||||
h("div", { style: labelStyle }, "Генерация"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Temp:"), tempIn,
|
||||
h("span", { class: "small muted" }, "Top-P:"), topPIn,
|
||||
h("span", { class: "small muted" }, "Max токенов:"), maxTokIn
|
||||
),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "System prompt:"), sysIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 3: Reasoning (thinking)
|
||||
h("div", { style: labelStyle }, "Режим размышлений"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "Для моделей Qwen3/3.5: enable_thinking=false — модель отвечает напрямую без блока <think>" }, noThinkCb, "Отключить размышления"),
|
||||
h("span", { class: "small muted" }, "Think budget:"), thinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 4: Runtime
|
||||
h("div", { style: labelStyle }, "Runtime"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "MTP — Multi-Token Prediction (спекулятивное декодирование). Отключение замедляет генерацию, но стабилизирует output." }, skipMtpCb, "Отключить MTP")
|
||||
)
|
||||
);
|
||||
|
||||
const gearBtn = h("button", { class: "btn small", type: "button", title: "⚙ Настройки модели" }, "⚙");
|
||||
gearBtn.addEventListener("click", () => {
|
||||
drawer.style.display = drawer.style.display === "none" ? "block" : "none";
|
||||
});
|
||||
|
||||
const row = h("div", { class: "flex wrap cmf-server-row", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), gearBtn, removeBtn, st, drawer);
|
||||
const entry = { node: row, read: () => ({
|
||||
id: idIn.value.trim() || autoId(modelSel.value),
|
||||
model: modelSel.value.trim(),
|
||||
port: parseInt(portIn.value) || 8090,
|
||||
threads: parseInt(threadsIn.value) || 0,
|
||||
gpu: gpuIn.checked,
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: noThinkCb.checked ? 0 : (thinkIn.value !== "" ? parseInt(thinkIn.value) : null),
|
||||
o1: o1Sel.value || null,
|
||||
o1_m: o1mIn.value !== "" ? parseInt(o1mIn.value) : null,
|
||||
o1_window: o1wIn.value !== "" ? parseInt(o1wIn.value) : null,
|
||||
o1_sink: o1sinkIn.value !== "" ? parseInt(o1sinkIn.value) : null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysIn.value.trim() || null,
|
||||
}) };
|
||||
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
|
||||
serverRows.push(entry);
|
||||
serversWrap.appendChild(row);
|
||||
|
||||
Reference in New Issue
Block a user