Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
32ce2f7ed7 | ||
|
|
9ec8861464 | ||
|
|
ad438a5dcb | ||
|
|
45def5c6b6 | ||
|
|
5d8455b4a0 | ||
|
|
b255c60b0b | ||
|
|
a75cacd0ec | ||
|
|
850a54acdf | ||
|
|
22110ac524 | ||
|
|
b3b37d27f4 | ||
|
|
a7c8ed72c7 | ||
|
|
f2e6047bff | ||
|
|
0b28aa3d59 | ||
|
|
1414ad1b52 | ||
|
|
94c8551206 | ||
|
|
6ecc709fd2 | ||
|
|
1661c19550 | ||
|
|
66ee32b233 | ||
|
|
f6ee045989 | ||
|
|
59e2eb3215 | ||
|
|
5b743a4691 | ||
|
|
c59ed9c949 | ||
|
|
b55e72789f | ||
|
|
0d1283c3fa | ||
|
|
0105a43ef5 | ||
|
|
b8d48c31f7 | ||
|
|
196ff52f0f | ||
|
|
e25814a54c | ||
|
|
0c20eb34f0 | ||
|
|
f8c1bb0914 | ||
|
|
0f299b77e6 | ||
|
|
7c49510c46 | ||
|
|
31da84cc45 | ||
|
|
749a47ce9b | ||
|
|
d2b1d50eb8 | ||
|
|
d48349a144 | ||
|
|
f05d26bfbb | ||
|
|
39f1fcb600 | ||
|
|
42377a75d4 | ||
|
|
6cf2608fc8 | ||
|
|
e6d1a630c9 | ||
|
|
8dd77ca7ff | ||
|
|
630149d141 | ||
|
|
85fc508f04 | ||
|
|
010171e05c | ||
|
|
39c56145ed | ||
|
|
17c8186383 | ||
|
|
e773e18efd | ||
|
|
6466006de6 | ||
|
|
6a25d8ac08 | ||
|
|
e486617e51 | ||
|
|
729a8bfbba | ||
|
|
5759de153b | ||
|
|
bf8534f13d | ||
|
|
70d8cde258 | ||
|
|
3d71b03788 | ||
|
|
492953d4c2 | ||
|
|
3b0231b32d | ||
|
|
fd2f2a165b | ||
|
|
cbc385f6c7 |
@@ -1,4 +1,6 @@
|
||||
target
|
||||
models
|
||||
*.cmf
|
||||
.git
|
||||
.github
|
||||
docs
|
||||
|
||||
@@ -21,9 +21,20 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
# Skip when this version is already on crates.io — a version can never be
|
||||
# re-uploaded, so publishing one that exists (released by hand, or a re-run
|
||||
# of this job) is a guaranteed failure rather than a real problem.
|
||||
- name: Publish to crates.io
|
||||
if: ${{ env.CARGO_REGISTRY_TOKEN != '' }}
|
||||
run: cargo publish
|
||||
run: |
|
||||
name=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].name')
|
||||
version=$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[0].version')
|
||||
if curl -sf -H 'User-Agent: cortiq-gateway-release' \
|
||||
"https://crates.io/api/v1/crates/$name/$version" >/dev/null; then
|
||||
echo "$name $version is already published — skipping"
|
||||
else
|
||||
cargo publish
|
||||
fi
|
||||
|
||||
# --- prebuilt binaries + GitHub Release -----------------------------------
|
||||
release:
|
||||
|
||||
+113
@@ -3,8 +3,121 @@
|
||||
All notable changes to this project are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
## [0.2.37] - 2026-08-06
|
||||
|
||||
### Added
|
||||
- **Zero-argument startup**: `cortiq-gateway` now runs with no flags. The config is created on
|
||||
first launch (repo `config/gateway.toml` when run from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml`, or `$CORTIQ_GATEWAY_HOME`), the browser opens by itself,
|
||||
and a busy port is swapped for the next free one. `--no-browser` / `--no-port-search` opt out;
|
||||
`--config` still takes an explicit path.
|
||||
- **No admin token required by default**: the console opens without auth. Set `--admin-token`,
|
||||
`[admin].token` or `[admin].token_env` to protect it.
|
||||
- **First-run wizard** (`/admin#/onboarding`): pick a language, download a ready-made `.cmf`
|
||||
model, and finish — Finish connects every completed download as a local model, starts its
|
||||
`cortiq serve` and points `routing.default` at it. Nothing to edit by hand.
|
||||
- **Ready-made models are read live** from the `infosave` HuggingFace account in both the wizard
|
||||
and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a
|
||||
delete button. Newly published models appear without a gateway update.
|
||||
- **Windows-friendly data dir**: `%APPDATA%` / `%LOCALAPPDATA%` / `%USERPROFILE%` are used before
|
||||
falling back, so running the exe from `Downloads` never writes into `Program Files`.
|
||||
- **Full localization**: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr).
|
||||
Import, Settings → Local models, the model form and the whole first-run wizard were English-only
|
||||
in de/fr/es/zh/tr before this release.
|
||||
|
||||
### Fixed
|
||||
- **Download progress jumped straight to 100% and then hung**: with `-L`, curl prints a complete
|
||||
`0→100%` progress bar for the redirect hop before the transfer starts, and that bar was being
|
||||
parsed as the download's progress. Progress is now derived from bytes on disk against the size
|
||||
reported by the HuggingFace tree (or a `HEAD` request), so the bar and the reported
|
||||
`done / total` bytes track the real transfer.
|
||||
- **Re-downloading over an existing file reported 100% instantly**: the previous copy is now
|
||||
removed before the transfer starts, so the size poller cannot read stale bytes.
|
||||
- **Downloaded models were not recognised**: the admin UI derived the output file name
|
||||
independently of the gateway, so a finished download did not mark its card as downloaded and the
|
||||
delete button never appeared. The gateway now returns the exact output stem (`local_name`) and
|
||||
the UI sends it back verbatim for download, progress and delete.
|
||||
- **Registering a local model produced an unusable pool entry**: with `manage_server = false` the
|
||||
model was added to the static pool pointing at `cortiq_server_url`, where nothing listens —
|
||||
every call failed with `upstream unavailable`. Registration now always creates a managed server
|
||||
the gateway runs itself, and repairs a stale static entry for the same file.
|
||||
- **Cancel did nothing during a sharded download**: cancellation was only checked between part
|
||||
files, so a multi-GB part had to finish first. The running transfer is now aborted immediately
|
||||
and the partial output removed.
|
||||
- **Deleting a model left the config broken**: removing a `.cmf` now also unregisters the model
|
||||
and stops its managed server instead of leaving a pool entry pointing at a missing file.
|
||||
- **Two variants of one repo collapsed into a single job row**; jobs are now keyed by repo + variant.
|
||||
- Curl's stderr is captured into the job log, so a failed download explains itself instead of
|
||||
showing a bare exit code.
|
||||
|
||||
## [0.2.36] - 2026-07-25
|
||||
|
||||
### Fixed
|
||||
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
|
||||
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
|
||||
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
|
||||
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.
|
||||
|
||||
## [0.2.35] - 2026-07-23
|
||||
|
||||
### Fixed
|
||||
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
|
||||
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.
|
||||
|
||||
## [0.2.34] - 2026-07-23
|
||||
|
||||
### Added
|
||||
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
|
||||
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.
|
||||
|
||||
### Fixed
|
||||
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
|
||||
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.
|
||||
|
||||
## [0.2.31] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
|
||||
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.
|
||||
|
||||
### Fixed
|
||||
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
|
||||
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.
|
||||
|
||||
## [0.2.30] - 2026-07-22
|
||||
|
||||
### Added
|
||||
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
|
||||
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.
|
||||
|
||||
|
||||
### Fixed
|
||||
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.
|
||||
|
||||
|
||||
### Added
|
||||
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
|
||||
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
### Added
|
||||
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
|
||||
gateway + a local Ollama model in one command. GHCR badge in the READMEs.
|
||||
|
||||
Generated
+31
-1
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.16"
|
||||
version = "0.2.37"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
@@ -297,6 +297,7 @@ dependencies = [
|
||||
"futures",
|
||||
"getrandom 0.2.17",
|
||||
"mime_guess",
|
||||
"open",
|
||||
"reqwest",
|
||||
"rust-embed",
|
||||
"serde",
|
||||
@@ -743,6 +744,25 @@ version = "2.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
|
||||
|
||||
[[package]]
|
||||
name = "is-docker"
|
||||
version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "928bae27f42bc99b60d9ac7334e3a21d10ad8f1835a4e12ec3ec0464765ed1b3"
|
||||
dependencies = [
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is-wsl"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "173609498df190136aa7dea1a91db051746d339e18476eed5ca40521f02d7aa5"
|
||||
dependencies = [
|
||||
"is-docker",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "is_terminal_polyfill"
|
||||
version = "1.70.2"
|
||||
@@ -874,6 +894,16 @@ version = "1.70.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
||||
|
||||
[[package]]
|
||||
name = "open"
|
||||
version = "5.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f9cfef937e9c486488c7e3d949ae31c0f1d06bdacd75b99c086cb35356e30408"
|
||||
dependencies = [
|
||||
"is-wsl",
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parking_lot"
|
||||
version = "0.12.5"
|
||||
|
||||
+15
-5
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "cortiq-gateway"
|
||||
version = "0.2.16"
|
||||
version = "0.2.37"
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
license = "Apache-2.0"
|
||||
@@ -10,12 +10,16 @@ homepage = "https://api.allaigate.com"
|
||||
readme = "README.md"
|
||||
keywords = ["llm", "gateway", "proxy", "openai", "routing"]
|
||||
categories = ["web-programming::http-server", "command-line-utilities"]
|
||||
# Files shipped in the published crate. The embedded admin SPA (web/) MUST be
|
||||
# included because rust-embed compiles it in; docs/screenshots are excluded to
|
||||
# keep the crate lean.
|
||||
# Files shipped in the published crate. The embedded admin SPA MUST be included
|
||||
# because rust-embed compiles it in; docs/screenshots are excluded to keep the
|
||||
# crate lean. The web/ subfolders are listed one by one rather than as web/**/*
|
||||
# so stray scratch files dropped next to the SPA never end up published.
|
||||
include = [
|
||||
"src/**/*",
|
||||
"web/**/*",
|
||||
"web/index.html",
|
||||
"web/favicon.svg",
|
||||
"web/css/**/*",
|
||||
"web/js/**/*",
|
||||
"config/gateway.example.toml",
|
||||
"README.md",
|
||||
"CHANGELOG.md",
|
||||
@@ -54,6 +58,8 @@ tokio-stream = "0.1"
|
||||
|
||||
# CLI / config
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
# open browser on first run (optional)
|
||||
open = { version = "5", optional = true }
|
||||
|
||||
# errors / logging / metrics
|
||||
thiserror = "1"
|
||||
@@ -61,6 +67,10 @@ anyhow = "1"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
|
||||
[features]
|
||||
default = ["open-browser"]
|
||||
open-browser = ["open"]
|
||||
|
||||
[profile.release]
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
|
||||
+13
-5
@@ -1,5 +1,7 @@
|
||||
# ---- build stage ------------------------------------------------------------
|
||||
FROM rust:1-slim AS builder
|
||||
# pin builder and runtime to the same Debian release — a floating `rust:1-slim`
|
||||
# drifts to a newer glibc than the runtime image and the binary refuses to start
|
||||
FROM rust:1-slim-trixie AS builder
|
||||
WORKDIR /app
|
||||
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
@@ -8,18 +10,24 @@ COPY web ./web
|
||||
RUN cargo build --release
|
||||
|
||||
# ---- runtime stage ----------------------------------------------------------
|
||||
FROM debian:bookworm-slim
|
||||
FROM debian:trixie-slim
|
||||
# curl is required at runtime — model imports download ready .cmf files with it
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates curl \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& useradd -r -u 10001 cortiq \
|
||||
&& mkdir -p /app/config
|
||||
&& mkdir -p /app/config /app/data/models
|
||||
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
|
||||
# ship the example as the default config so `docker run` works out of the box
|
||||
COPY config/gateway.example.toml /app/config/gateway.toml
|
||||
RUN chown -R 10001:10001 /app
|
||||
WORKDIR /app
|
||||
USER cortiq
|
||||
# all gateway state (downloaded models, stats, secrets) lives under /app/data —
|
||||
# mount a volume there to keep it across container recreates
|
||||
ENV CORTIQ_GATEWAY_HOME=/app/data
|
||||
# runs as root so writes to bind-mounted host folders (Docker Desktop on
|
||||
# Windows/macOS) don't fail with permission errors (curl exit code 23);
|
||||
# add `--user 10001` to docker run to harden when not using host bind mounts
|
||||
EXPOSE 9000
|
||||
ENTRYPOINT ["cortiq-gateway"]
|
||||
CMD ["--config", "/app/config/gateway.toml"]
|
||||
|
||||
@@ -36,6 +36,7 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Zero-config start.** `cortiq-gateway` with no flags: the config is created for you, a busy port is swapped for a free one, the browser opens on a setup wizard that downloads a ready-made local model and wires it into routing — no TOML, no token, no separate model server.
|
||||
- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
|
||||
- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
|
||||
@@ -54,11 +55,13 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
|
||||
```bash
|
||||
# Docker (GitHub Container Registry)
|
||||
docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
-v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
-v "$PWD/config:/app/config" -v cortiq-data:/app/data \
|
||||
ghcr.io/infosave2007/cortiq-gateway:latest
|
||||
# → admin console at http://localhost:9000/admin
|
||||
|
||||
# from crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # no flags, no config file, no token — see Quick start
|
||||
|
||||
# or build from source
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -67,7 +70,10 @@ cargo build --release # ./target/release/cortiq-gateway
|
||||
```
|
||||
|
||||
The image ships a default config and runs out of the box; mount `/app/config` (with a
|
||||
`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
|
||||
`gateway.toml`) to use your own. Downloaded models and stats live in `/app/data` — mount
|
||||
a volume there so they survive container recreates (a host folder works too, e.g.
|
||||
`-v "$PWD/models:/app/data/models"`). Tags: `:latest`, `:X.Y.Z`, `:edge`
|
||||
(linux/amd64 + arm64).
|
||||
|
||||
**Docker Compose** — gateway + a local Ollama model in one command:
|
||||
|
||||
@@ -98,32 +104,54 @@ The gateway ships with an embedded web console at **`/admin`**:
|
||||
|  |
|
||||
| test the live pipeline and see the routing decision |
|
||||
|
||||
The console is **open by default** — no token needed. To protect it, set any one of
|
||||
`--admin-token <TOKEN>`, `[admin].token`, or `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <YOUR_TOKEN>
|
||||
cortiq-gateway --admin-token <YOUR_TOKEN>
|
||||
# open http://localhost:9000/admin?token=<YOUR_TOKEN>
|
||||
```
|
||||
|
||||
If `--admin-token` / `[admin].token_env` are not set, a token is generated at startup
|
||||
and printed to the log. All `/admin/api/*` endpoints require a Bearer admin token;
|
||||
secret values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
Once a token is set, every `/admin/api/*` endpoint requires it as a Bearer token. Secret
|
||||
values are never returned by the API (only their presence: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick start
|
||||
|
||||
```bash
|
||||
# 1. start a router (the hosted allaigate router, or a local cortiq-router)
|
||||
# it listens e.g. on http://localhost:8080 (or https://api.allaigate.com)
|
||||
|
||||
# 2. describe your model pool
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. run the gateway
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
# Gateway listens on 0.0.0.0:9000 and serves the admin console at /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
That is the whole command. On first run the gateway:
|
||||
|
||||
1. **creates its config** — `config/gateway.toml` when started from a checkout, otherwise
|
||||
`~/.cortiq-gateway/gateway.toml` (override with `$CORTIQ_GATEWAY_HOME` or `--config`);
|
||||
2. **picks a free port** if 9000 is taken, and writes the choice back to the config;
|
||||
3. **opens your browser** on the setup wizard at `/admin#/onboarding`.
|
||||
|
||||
The wizard takes you from nothing to a working local model:
|
||||
|
||||
| Step | What happens |
|
||||
|---|---|
|
||||
| **Language** | 7 languages; changeable later from the header |
|
||||
| **Ready-made models** | the [`infosave`](https://huggingface.co/infosave) `.cmf` models, read live from HuggingFace — pick a quantization and watch the real download progress |
|
||||
| **Any HF model** | search HuggingFace and auto-convert to `.cmf` |
|
||||
| **Finish** | every completed download is connected as a local model, its `cortiq serve` is started, and `routing.default` points at it |
|
||||
|
||||
No TOML editing, no separate "register" step, no manually started model server. Prefer to skip
|
||||
the wizard? Add models from the console (**Models**, **Import model**) or hand-write the config —
|
||||
both still work, and changes hot-reload without a restart.
|
||||
|
||||
```bash
|
||||
# explicit config / port / no browser — all optional
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
For smart routing across a pool, point `[router]` at the hosted allaigate router or a local
|
||||
`cortiq-router` (see [docs/ROUTER.md](docs/ROUTER.md)). Without a router the gateway degrades
|
||||
gracefully to the default model, so a local-only setup works out of the box.
|
||||
|
||||
Now any OpenAI client works through the gateway:
|
||||
|
||||
```python
|
||||
|
||||
+38
-14
@@ -36,6 +36,7 @@
|
||||
|
||||
## ✨ Ключевое
|
||||
|
||||
- **Запуск без конфигурации.** `cortiq-gateway` без флагов: конфиг создаётся сам, занятый порт меняется на свободный, браузер открывается на мастере, который скачивает готовую локальную модель и заводит её в роутинг — без TOML, без токена, без отдельно запущенного сервера модели.
|
||||
- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
|
||||
- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
|
||||
- **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
|
||||
@@ -57,6 +58,7 @@ docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
|
||||
|
||||
# с crates.io
|
||||
cargo install cortiq-gateway
|
||||
cortiq-gateway # без флагов, без конфига, без токена — см. Быстрый старт
|
||||
|
||||
# или сборка из исходников
|
||||
git clone https://github.com/infosave2007/cortiq-gateway
|
||||
@@ -94,32 +96,54 @@ docker compose exec ollama ollama pull qwen2.5:0.5b # один раз скач
|
||||
|  |  |
|
||||
| **Playground** — тест живого пайплайна с решением роутинга | **Dashboard** — трафик, стоимость, латентность |
|
||||
|
||||
Панель **по умолчанию открыта** — токен не нужен. Чтобы закрыть её, задайте любое из
|
||||
`--admin-token <ТОКЕН>`, `[admin].token` или `[admin].token_env`:
|
||||
|
||||
```bash
|
||||
cargo run --release -- --config config/gateway.toml --admin-token <ВАШ_ТОКЕН>
|
||||
cortiq-gateway --admin-token <ВАШ_ТОКЕН>
|
||||
# откройте http://localhost:9000/admin?token=<ВАШ_ТОКЕН>
|
||||
```
|
||||
|
||||
Если `--admin-token` / `[admin].token_env` не заданы — токен генерируется при старте и
|
||||
печатается в лог. Все эндпоинты `/admin/api/*` требуют Bearer admin-токен; значения
|
||||
секретов API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
Когда токен задан, все эндпоинты `/admin/api/*` требуют его как Bearer. Значения секретов
|
||||
API наружу не отдаёт (только статус наличия: `store` / `env` / `missing`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Быстрый старт
|
||||
|
||||
```bash
|
||||
# 1. поднимите роутер (хостовый allaigate или локальный cortiq-router)
|
||||
# он слушает, например, http://localhost:8080 (или https://api.allaigate.com)
|
||||
|
||||
# 2. опишите свой пул моделей
|
||||
cp config/gateway.example.toml config/gateway.toml
|
||||
$EDITOR config/gateway.toml
|
||||
|
||||
# 3. запустите шлюз
|
||||
cargo run --release -- --config config/gateway.toml
|
||||
# Шлюз слушает 0.0.0.0:9000 и отдаёт панель на /admin
|
||||
cortiq-gateway
|
||||
```
|
||||
|
||||
Это вся команда. При первом запуске шлюз:
|
||||
|
||||
1. **создаёт конфиг** — `config/gateway.toml`, если запущен из репозитория, иначе
|
||||
`~/.cortiq-gateway/gateway.toml` (переопределяется через `$CORTIQ_GATEWAY_HOME` или `--config`);
|
||||
2. **подбирает свободный порт**, если 9000 занят, и записывает выбор в конфиг;
|
||||
3. **открывает браузер** на мастере настройки `/admin#/onboarding`.
|
||||
|
||||
Мастер доводит от нуля до работающей локальной модели:
|
||||
|
||||
| Шаг | Что происходит |
|
||||
|---|---|
|
||||
| **Язык** | 7 языков; потом меняется в шапке |
|
||||
| **Готовые модели** | `.cmf` модели [`infosave`](https://huggingface.co/infosave), список читается с HuggingFace вживую — выбираете квантизацию и видите реальный прогресс загрузки |
|
||||
| **Любая модель с HF** | поиск по HuggingFace и автоконвертация в `.cmf` |
|
||||
| **Завершить** | каждая скачанная модель подключается как локальная, поднимается её `cortiq serve`, `routing.default` указывает на неё |
|
||||
|
||||
Никаких правок TOML, отдельного шага «зарегистрировать» и вручную запущенного сервера модели.
|
||||
Мастер не нужен? Добавляйте модели из панели (**Модели**, **Импорт модели**) или пишите конфиг
|
||||
руками — оба пути работают, изменения применяются на лету без перезапуска.
|
||||
|
||||
```bash
|
||||
# явный конфиг / порт / без браузера — всё опционально
|
||||
cortiq-gateway --config config/gateway.toml --no-browser --no-port-search
|
||||
```
|
||||
|
||||
Для умного роутинга по пулу направьте `[router]` на хостовый allaigate или локальный
|
||||
`cortiq-router` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)). Без роутера шлюз мягко
|
||||
деградирует до модели по умолчанию, поэтому чисто локальная установка работает сразу.
|
||||
|
||||
Теперь любой OpenAI-клиент работает через шлюз:
|
||||
|
||||
```python
|
||||
|
||||
@@ -17,6 +17,10 @@ services:
|
||||
CORTIQ_ROUTER_KEY: ${CORTIQ_ROUTER_KEY:-}
|
||||
volumes:
|
||||
- ./config/gateway.compose.toml:/app/config/gateway.toml:ro
|
||||
# downloaded .cmf models + stats persist here across recreates; swap the
|
||||
# named volume for a host folder (e.g. ./models:/app/data/models) if you
|
||||
# want the files visible next to the compose file
|
||||
- cortiq-data:/app/data
|
||||
depends_on:
|
||||
- ollama
|
||||
restart: unless-stopped
|
||||
@@ -30,4 +34,5 @@ services:
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
cortiq-data:
|
||||
ollama:
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ timeout_ms = 15000 # allow for oracle escalation on hard
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <path> only if you keep the config elsewhere
|
||||
```
|
||||
|
||||
## API contract
|
||||
|
||||
+1
-1
@@ -49,7 +49,7 @@ timeout_ms = 15000 # запас под oracle-эскал
|
||||
|
||||
```bash
|
||||
export CORTIQ_ROUTER_KEY=cortiq_your_key
|
||||
cortiq-gateway --config config/gateway.toml
|
||||
cortiq-gateway # --config <путь> нужен, только если конфиг лежит в другом месте
|
||||
```
|
||||
|
||||
## Контракт API
|
||||
|
||||
+695
-95
@@ -20,7 +20,7 @@ use axum::{
|
||||
http::{header, StatusCode},
|
||||
middleware::{self, Next},
|
||||
response::{IntoResponse, Response},
|
||||
routing::{get, post},
|
||||
routing::{delete, get, post},
|
||||
Json, Router,
|
||||
};
|
||||
use serde::Deserialize;
|
||||
@@ -139,10 +139,11 @@ fn parse_routing(model: &str) -> RoutingDirective {
|
||||
|
||||
// ───────────────────────────── router ───────────────────────────────────────
|
||||
|
||||
/// Admin API routes (token-protected). Mounted into the main router.
|
||||
/// Admin API routes. If `admin_token` is empty, the panel is open (no auth);
|
||||
/// otherwise every `/admin/api/*` request needs `Authorization: Bearer <token>`.
|
||||
/// The token is captured in a middleware closure so it does not depend on State.
|
||||
pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
Router::new()
|
||||
let router = Router::new()
|
||||
.route("/admin/api/health", get(health))
|
||||
.route("/admin/api/router/probe", post(router_probe))
|
||||
.route("/admin/api/meta", get(meta))
|
||||
@@ -183,10 +184,18 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
|
||||
.route("/admin/api/requests", get(get_requests))
|
||||
.route("/admin/api/test", post(run_test))
|
||||
.route("/admin/api/test/stream", post(run_test_stream))
|
||||
.route_layer(middleware::from_fn(move |req: Request, next: Next| {
|
||||
.route("/admin/api/setup/status", get(setup_status))
|
||||
.route("/admin/api/setup/complete", post(setup_complete))
|
||||
.route("/admin/api/setup/featured", get(setup_featured))
|
||||
.route("/admin/api/setup/file", delete(delete_setup_file));
|
||||
if admin_token.is_empty() {
|
||||
router
|
||||
} else {
|
||||
router.route_layer(middleware::from_fn(move |req: Request, next: Next| {
|
||||
let expected = admin_token.clone();
|
||||
async move { auth(expected, req, next).await }
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract Bearer admin token from the `Authorization` header.
|
||||
@@ -467,16 +476,111 @@ async fn provider_models(
|
||||
}
|
||||
// OpenAI + Anthropic both return { "data": [ { "id": ... } ] }.
|
||||
let v: Value = serde_json::from_str(&body).unwrap_or(json!({}));
|
||||
let mut models: Vec<String> = v["data"]
|
||||
let mut models: Vec<Value> = v["data"]
|
||||
.as_array()
|
||||
.map(|arr| {
|
||||
arr.iter()
|
||||
.filter_map(|m| m["id"].as_str().map(|s| s.to_string()))
|
||||
.filter_map(|m| {
|
||||
let id = m["id"].as_str()?.to_string();
|
||||
let mut obj = json!({ "id": id });
|
||||
let mut caps = vec![];
|
||||
if let Some(pricing) = m.get("pricing").filter(|v| !v.is_null()) {
|
||||
if let Some(prompt) = pricing
|
||||
.get("prompt")
|
||||
.and_then(|s| s.as_str())
|
||||
.and_then(|s| s.parse::<f64>().ok())
|
||||
{
|
||||
obj["price_in"] = json!(prompt * 1_000_000.0);
|
||||
}
|
||||
if let Some(comp) = pricing
|
||||
.get("completion")
|
||||
.and_then(|s| s.as_str())
|
||||
.and_then(|s| s.parse::<f64>().ok())
|
||||
{
|
||||
obj["price_out"] = json!(comp * 1_000_000.0);
|
||||
}
|
||||
if pricing
|
||||
.get("input_cache_read")
|
||||
.filter(|v| !v.is_null())
|
||||
.is_some()
|
||||
|| pricing
|
||||
.get("input_cache_write")
|
||||
.filter(|v| !v.is_null())
|
||||
.is_some()
|
||||
{
|
||||
caps.push("caching");
|
||||
}
|
||||
}
|
||||
if let Some(params) = m.get("supported_parameters").and_then(|p| p.as_array()) {
|
||||
let has_tools = params.iter().any(|p| {
|
||||
p.as_str() == Some("tools") || p.as_str() == Some("tool_choice")
|
||||
});
|
||||
if has_tools {
|
||||
caps.push("tools");
|
||||
}
|
||||
let has_json = params.iter().any(|p| p.as_str() == Some("response_format"));
|
||||
if has_json {
|
||||
caps.push("json");
|
||||
}
|
||||
let has_reasoning = params.iter().any(|p| {
|
||||
p.as_str() == Some("reasoning")
|
||||
|| p.as_str() == Some("include_reasoning")
|
||||
});
|
||||
if has_reasoning || m.get("reasoning").filter(|v| !v.is_null()).is_some() {
|
||||
caps.push("reasoning");
|
||||
}
|
||||
}
|
||||
if let Some(arch) = m.get("architecture").filter(|v| !v.is_null()) {
|
||||
if let Some(inputs) =
|
||||
arch.get("input_modalities").and_then(|i| i.as_array())
|
||||
{
|
||||
if inputs.iter().any(|i| i.as_str() == Some("image")) {
|
||||
caps.push("vision");
|
||||
}
|
||||
if inputs.iter().any(|i| i.as_str() == Some("audio")) {
|
||||
caps.push("audio");
|
||||
}
|
||||
}
|
||||
}
|
||||
if !caps.is_empty() {
|
||||
obj["caps"] = json!(caps);
|
||||
}
|
||||
Some(obj)
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
models.sort();
|
||||
ok(json!({ "ok": true, "count": models.len(), "models": models }))
|
||||
models.sort_by(|a, b| a["id"].as_str().cmp(&b["id"].as_str()));
|
||||
|
||||
let mut resp = json!({ "ok": true, "count": models.len(), "models": models });
|
||||
|
||||
if b.provider == "openrouter" {
|
||||
if let Some(k) = key {
|
||||
if let Ok(cred_resp) = client
|
||||
.get("https://openrouter.ai/api/v1/credits")
|
||||
.header("authorization", format!("Bearer {k}"))
|
||||
.send()
|
||||
.await
|
||||
{
|
||||
if let Ok(cred_body) = cred_resp.text().await {
|
||||
let cred_v: Value = serde_json::from_str(&cred_body).unwrap_or(json!({}));
|
||||
if let Some(data) = cred_v.get("data") {
|
||||
let total = data
|
||||
.get("total_credits")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
let usage = data
|
||||
.get("total_usage")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
resp["balance"] = json!(total - usage);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ok(resp)
|
||||
}
|
||||
|
||||
async fn meta() -> ApiResult {
|
||||
@@ -905,75 +1009,12 @@ async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBo
|
||||
{
|
||||
needs_restart = true;
|
||||
}
|
||||
let old_servers = cfg.cmf.effective_servers();
|
||||
cfg.cmf = v;
|
||||
// Replacing or renaming a managed model must not brick the save: routing
|
||||
// may still reference the old id. Map a vanished id to its replacement
|
||||
// (the new server on the same port), strip ids that are truly gone, and
|
||||
// repair the default so validate() passes.
|
||||
heal_routing_after_cmf_change(&mut cfg, &old_servers);
|
||||
}
|
||||
state.reload(cfg)?;
|
||||
ok(json!({ "ok": true, "needs_restart": needs_restart }))
|
||||
}
|
||||
|
||||
/// After the managed-servers list changed, rewrite [routing] so it only
|
||||
/// references ids that still exist. A removed server whose port is now held by
|
||||
/// a NEW server is treated as a replacement (old id → new id); other vanished
|
||||
/// ids are stripped. An emptied default falls back to the first managed server,
|
||||
/// then the first pool model.
|
||||
fn heal_routing_after_cmf_change(cfg: &mut Config, old_servers: &[crate::config::CmfServer]) {
|
||||
let servers = cfg.cmf.effective_servers();
|
||||
let known: std::collections::HashSet<String> = cfg
|
||||
.models
|
||||
.iter()
|
||||
.map(|m| m.id.clone())
|
||||
.chain(servers.iter().map(|s| s.id.clone()))
|
||||
.collect();
|
||||
let new_by_port: HashMap<u16, String> =
|
||||
servers.iter().map(|s| (s.port, s.id.clone())).collect();
|
||||
let mut rename: HashMap<String, String> = HashMap::new();
|
||||
for old in old_servers {
|
||||
if !known.contains(&old.id) {
|
||||
if let Some(new_id) = new_by_port.get(&old.port) {
|
||||
rename.insert(old.id.clone(), new_id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
let fix = |id: &str| -> Option<String> {
|
||||
if known.contains(id) {
|
||||
Some(id.to_string())
|
||||
} else {
|
||||
rename.get(id).cloned()
|
||||
}
|
||||
};
|
||||
if !cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = fix(&cfg.routing.default).unwrap_or_default();
|
||||
}
|
||||
if cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = servers
|
||||
.first()
|
||||
.map(|s| s.id.clone())
|
||||
.or_else(|| cfg.models.first().map(|m| m.id.clone()))
|
||||
.unwrap_or_default();
|
||||
}
|
||||
for tt in cfg.routing.tiers.values_mut() {
|
||||
let list = match tt {
|
||||
TierTargets::List(v) => v.clone(),
|
||||
TierTargets::One(s) => vec![s.clone()],
|
||||
};
|
||||
let mut healed: Vec<String> = Vec::new();
|
||||
for id in &list {
|
||||
if let Some(new_id) = fix(id) {
|
||||
if !healed.contains(&new_id) {
|
||||
healed.push(new_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
*tt = TierTargets::List(healed);
|
||||
}
|
||||
}
|
||||
|
||||
// ───────────────────────────── cmf runtime ─────────────────────────────────
|
||||
/// Status of the `cortiq` CLI (the CMF format runtime): is it installed, which
|
||||
/// version, and whether a managed server is running. Drives the "Install cortiq"
|
||||
@@ -1091,9 +1132,6 @@ async fn cmf_port_check(State(state): State<SharedState>, Query(q): Query<PortQu
|
||||
ok(json!({
|
||||
"port": port,
|
||||
"available": available,
|
||||
// true = held by this gateway's own managed server (a replaced/edited
|
||||
// row frees it on restart) — the UI shows this as info, not an error
|
||||
"managed": running_here,
|
||||
"detail": detail,
|
||||
"suggested": suggested,
|
||||
}))
|
||||
@@ -1315,6 +1353,507 @@ async fn hf_search(State(state): State<SharedState>, Query(q): Query<HfQuery>) -
|
||||
ok(json!({ "models": models }))
|
||||
}
|
||||
|
||||
// ── first-run onboarding ──
|
||||
|
||||
fn setup_marker(state: &SharedState) -> std::path::PathBuf {
|
||||
std::path::Path::new(&state.config_path)
|
||||
.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.join(".setup_done")
|
||||
}
|
||||
|
||||
async fn setup_status(State(state): State<SharedState>) -> ApiResult {
|
||||
let live = state.live();
|
||||
let marker = setup_marker(&state);
|
||||
let done = marker.exists();
|
||||
let has_models = !live.cfg.models.is_empty() || !live.cfg.cmf.effective_servers().is_empty();
|
||||
let has_imports = !state.imports.list().is_empty();
|
||||
ok(json!({
|
||||
"setup_done": done,
|
||||
"has_models": has_models,
|
||||
"has_imports": has_imports,
|
||||
"needs_setup": !done && !has_models,
|
||||
"listen": live.cfg.listen,
|
||||
"version": env!("CARGO_PKG_VERSION"),
|
||||
}))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SetupCompleteBody {
|
||||
#[serde(default)]
|
||||
lang: Option<String>,
|
||||
/// Connect finished downloads as local models (default: yes).
|
||||
#[serde(default)]
|
||||
register: Option<bool>,
|
||||
}
|
||||
|
||||
/// Finish the wizard. Everything that finished downloading is connected as a
|
||||
/// local model here — the wizard promises exactly that, and leaving it to a
|
||||
/// separate button is how a first run ends with an empty model pool.
|
||||
async fn setup_complete(
|
||||
State(state): State<SharedState>,
|
||||
Json(b): Json<SetupCompleteBody>,
|
||||
) -> ApiResult {
|
||||
let marker = setup_marker(&state);
|
||||
if let Some(dir) = marker.parent() {
|
||||
let _ = std::fs::create_dir_all(dir);
|
||||
}
|
||||
let lang = b.lang.unwrap_or_else(|| "en".into());
|
||||
let _ = std::fs::write(
|
||||
&marker,
|
||||
format!(
|
||||
"done lang={lang} ts={}",
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs()
|
||||
),
|
||||
);
|
||||
|
||||
let mut registered: Vec<String> = Vec::new();
|
||||
if b.register.unwrap_or(true) {
|
||||
let finished: Vec<String> = state
|
||||
.imports
|
||||
.list()
|
||||
.into_iter()
|
||||
.filter(|j| j.state == "done")
|
||||
.map(|j| j.output)
|
||||
.collect();
|
||||
if !finished.is_empty() {
|
||||
let mut cfg = current_cfg(&state);
|
||||
for output in &finished {
|
||||
if let Some(id) = register_cmf_output(&mut cfg, output) {
|
||||
registered.push(id);
|
||||
}
|
||||
}
|
||||
if cfg.routing.default.is_empty() {
|
||||
if let Some(first) = registered.first() {
|
||||
cfg.routing.default = first.clone();
|
||||
}
|
||||
}
|
||||
if !registered.is_empty() {
|
||||
state.reload(cfg)?;
|
||||
}
|
||||
}
|
||||
}
|
||||
let runtime = crate::cmf_runtime::installed_version(&state.live().cfg.cmf.cortiq_bin);
|
||||
ok(json!({
|
||||
"ok": true,
|
||||
"registered": registered,
|
||||
"runtime_installed": runtime.is_some(),
|
||||
"runtime_version": runtime,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Curated ready-to-run .cmf models from infosave (HF author) — proxied server-side.
|
||||
async fn setup_featured(State(state): State<SharedState>) -> ApiResult {
|
||||
let live = state.live();
|
||||
let token = if live.cfg.cmf.hf_token_env.is_empty() {
|
||||
None
|
||||
} else {
|
||||
state.secrets.resolve(&live.cfg.cmf.hf_token_env)
|
||||
};
|
||||
// Try HF author search; on failure return curated fallback
|
||||
let hf_result = crate::import::hf_search_author("infosave", 30, token.as_deref()).await;
|
||||
let models = match hf_result {
|
||||
Ok(v) => v,
|
||||
Err(_) => serde_json::json!([]),
|
||||
};
|
||||
// curated fallback with real sizes (usedStorage from HF)
|
||||
let fallback = json!([
|
||||
{"id": "infosave/Bonsai-1.7Bcmf", "label": "Bonsai 1.7B", "badge": "1.7B · Q1", "desc": "Small, fast local model — fits 4GB RAM", "size": "334 MB", "size_bytes": 334185278},
|
||||
{"id": "infosave/Bonsai-8B_2bit_cmf", "label": "Bonsai 8B 2-bit", "badge": "8B · Q1T", "desc": "Balanced quality/size — 2-bit ternary", "size": "2.3 GB", "size_bytes": 2322861074_u64},
|
||||
{"id": "infosave/Bonsai-27Bcmf", "label": "Bonsai 27B", "badge": "27B · Q1", "desc": "High quality — needs 16GB RAM", "size": "5.1 GB", "size_bytes": 5103462190_u64},
|
||||
{"id": "infosave/Qwen3.6-27Bcmf", "label": "Qwen 3.6 27B CMF", "badge": "27B · Q4 · MoE", "desc": "MoE model, strong reasoning", "size": "14.3 GB", "size_bytes": 14264207150_u64},
|
||||
{"id": "infosave/Nanbeige4.2-3Bcmf", "label": "Nanbeige 4.2 3B", "badge": "3B · looped", "desc": "Looped transformer, compact", "size": "2.4 GB", "size_bytes": 2364287523_u64},
|
||||
{"id": "infosave/KAT-Coder-V2.5-CMF", "label": "KAT Coder 2.5", "badge": "code · MoE", "desc": "Code specialist", "size": "12.7 GB", "size_bytes": 12653975221_u64},
|
||||
]);
|
||||
let arr = if models.as_array().map(|a| a.is_empty()).unwrap_or(true) {
|
||||
fallback
|
||||
} else {
|
||||
// enrich HF data with label/badge/size for UI — fetch usedStorage concurrently
|
||||
let mut ids: Vec<String> = Vec::new();
|
||||
if let Some(arr) = models.as_array() {
|
||||
for m in arr {
|
||||
if let Some(id) = m["id"].as_str() {
|
||||
if id.to_ascii_lowercase().contains("cmf") {
|
||||
ids.push(id.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if ids.is_empty() {
|
||||
return ok(json!({ "models": fallback }));
|
||||
}
|
||||
// fetch per-variant sizes in parallel (tree API, 10s each)
|
||||
let token_owned = token.map(|s| s.to_string());
|
||||
let variant_lists: Vec<(String, Vec<(String, u64)>)> =
|
||||
futures::future::join_all(ids.iter().map(|id| {
|
||||
let tid = token_owned.clone();
|
||||
let iid = id.clone();
|
||||
async move {
|
||||
let vars = crate::import::hf_repo_variants(&iid, tid.as_deref()).await;
|
||||
(iid, vars)
|
||||
}
|
||||
}))
|
||||
.await;
|
||||
let mut variant_map: std::collections::HashMap<String, Vec<(String, u64)>> =
|
||||
std::collections::HashMap::new();
|
||||
let mut fallback_map: std::collections::HashMap<String, (String, u64)> =
|
||||
std::collections::HashMap::new();
|
||||
let mut fallback_badge: std::collections::HashMap<String, String> =
|
||||
std::collections::HashMap::new();
|
||||
// also keep fallback sizes/badges for when tree is unreachable
|
||||
for v in fallback.as_array().unwrap_or(&vec![]) {
|
||||
if let (Some(id), Some(sz)) = (v["id"].as_str(), v["size_bytes"].as_u64()) {
|
||||
if let Some(s) = v["size"].as_str() {
|
||||
fallback_map.insert(id.to_string(), (s.to_string(), sz));
|
||||
}
|
||||
}
|
||||
if let (Some(id), Some(b)) = (v["id"].as_str(), v["badge"].as_str()) {
|
||||
fallback_badge.insert(id.to_string(), b.to_string());
|
||||
}
|
||||
}
|
||||
for (id, vars) in variant_lists {
|
||||
if !vars.is_empty() {
|
||||
variant_map.insert(id, vars);
|
||||
}
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
if let Some(arr) = models.as_array() {
|
||||
for m in arr {
|
||||
let id = m["id"].as_str().unwrap_or("").to_string();
|
||||
if !id.to_ascii_lowercase().contains("cmf") {
|
||||
continue;
|
||||
}
|
||||
let likes = m["likes"].as_u64().unwrap_or(0);
|
||||
let vars = variant_map.get(&id);
|
||||
// extract size token from id
|
||||
let lower = id.to_ascii_lowercase();
|
||||
let size_token = ["1.7b", "48b", "35b", "27b", "12b", "8b", "7b", "3b", "4b"]
|
||||
.iter()
|
||||
.find(|c| lower.contains(*c))
|
||||
.map(|c| match *c {
|
||||
"1.7b" => "1.7B",
|
||||
"48b" => "48B",
|
||||
"35b" => "35B",
|
||||
"27b" => "27B",
|
||||
"12b" => "12B",
|
||||
"8b" => "8B",
|
||||
"7b" => "7B",
|
||||
"3b" => "3B",
|
||||
"4b" => "4B",
|
||||
_ => "CMF",
|
||||
})
|
||||
.unwrap_or("CMF");
|
||||
// quant from variant name if available (more accurate for Bonsai q1), else id
|
||||
let quant_src = vars
|
||||
.and_then(|vs| vs.first().map(|(n, _)| n.as_str()))
|
||||
.unwrap_or(&id)
|
||||
.to_ascii_lowercase();
|
||||
let quant = if quant_src.contains("q1t")
|
||||
|| quant_src.contains("2bit")
|
||||
|| lower.contains("q1t")
|
||||
|| lower.contains("2bit")
|
||||
{
|
||||
"Q1T"
|
||||
} else if quant_src.contains("q2tp") || lower.contains("q2tp") {
|
||||
"Q2TP"
|
||||
} else if quant_src.contains("q4tp") || lower.contains("q4tp") {
|
||||
"Q4TP"
|
||||
} else if quant_src.contains("q1") || lower.contains("q1") {
|
||||
"Q1"
|
||||
} else if quant_src.contains("q4") || lower.contains("q4") {
|
||||
"Q4"
|
||||
} else if quant_src.contains("q8") || lower.contains("q8") {
|
||||
"Q8"
|
||||
} else {
|
||||
""
|
||||
};
|
||||
let mut badge = if quant.is_empty() || size_token == "CMF" {
|
||||
if size_token == "CMF" {
|
||||
"CMF · ready".to_string()
|
||||
} else {
|
||||
size_token.to_string()
|
||||
}
|
||||
} else {
|
||||
format!("{size_token} · {quant}")
|
||||
};
|
||||
// fallback has more accurate quant (e.g. Bonsai 1.7B Q1) when tree fetch failed
|
||||
if let Some(fb) = fallback_badge.get(&id) {
|
||||
if badge == size_token && fb.contains('·') {
|
||||
badge = fb.clone();
|
||||
}
|
||||
}
|
||||
let (size, size_bytes, variants_json) = if let Some(vs) = vars {
|
||||
if vs.len() == 1 {
|
||||
let (name, sz) = &vs[0];
|
||||
let s = crate::import::format_bytes(*sz);
|
||||
(
|
||||
Some(s.clone()),
|
||||
Some(*sz),
|
||||
json!([{"name": name, "size": s, "size_bytes": sz}]),
|
||||
)
|
||||
} else {
|
||||
// dedupe same-size variants (e.g. q2tp v2/v3 both 112GB → keep latest v3)
|
||||
let mut seen_sz = std::collections::HashSet::new();
|
||||
let mut uniq_vs: Vec<(String, u64)> = Vec::new();
|
||||
// iterate reversed to keep latest (v3) over v2 when same size
|
||||
for (n, sz) in vs.iter().rev() {
|
||||
let s = crate::import::format_bytes(*sz);
|
||||
if seen_sz.insert(s) {
|
||||
uniq_vs.push((n.clone(), *sz));
|
||||
}
|
||||
}
|
||||
uniq_vs.reverse();
|
||||
let parts: Vec<String> = uniq_vs
|
||||
.iter()
|
||||
.map(|(_, sz)| crate::import::format_bytes(*sz))
|
||||
.collect();
|
||||
let max_sz = uniq_vs.iter().map(|(_, sz)| *sz).max();
|
||||
let vars_json: Vec<Value> = uniq_vs.iter().map(|(n, sz)| json!({"name": n, "size": crate::import::format_bytes(*sz), "size_bytes": sz})).collect();
|
||||
(Some(parts.join(" / ")), max_sz, json!(vars_json))
|
||||
}
|
||||
} else if let Some((s, sz)) = fallback_map.get(&id) {
|
||||
(
|
||||
Some(s.clone()),
|
||||
Some(*sz),
|
||||
json!([{"name": id.split('/').next_back().unwrap_or(&id), "size": s, "size_bytes": sz}]),
|
||||
)
|
||||
} else {
|
||||
(None::<String>, None::<u64>, Value::Null)
|
||||
};
|
||||
// check if already downloaded locally (case-insensitive, handles sharded dirs)
|
||||
let models_dir = live.cfg.cmf.models_dir.clone();
|
||||
let base = crate::import::sanitize_for_id(id.split('/').next_back().unwrap_or(&id));
|
||||
let mut downloaded = false;
|
||||
let mut downloaded_variants: Vec<Value> = Vec::new();
|
||||
if let Some(vs) = variants_json.as_array() {
|
||||
for v in vs {
|
||||
let vname = v["name"].as_str().unwrap_or("");
|
||||
let vbase = if vs.len() == 1 {
|
||||
base.clone()
|
||||
} else {
|
||||
format!("{}-{}", base, crate::import::sanitize_for_id(vname))
|
||||
};
|
||||
let expected = format!("{vbase}.cmf").to_lowercase();
|
||||
// case-insensitive search via read_dir (handles /tmp symlink and case)
|
||||
let mut exists = false;
|
||||
let mut local_sz: u64 = 0;
|
||||
if let Ok(rd) = std::fs::read_dir(&models_dir) {
|
||||
for entry in rd.filter_map(|e| e.ok()) {
|
||||
let name = entry.file_name().to_string_lossy().to_lowercase();
|
||||
if name == expected {
|
||||
exists = true;
|
||||
local_sz = entry.metadata().map(|m| m.len()).unwrap_or(0);
|
||||
if local_sz == 0 && entry.path().is_dir() {
|
||||
local_sz = std::fs::read_dir(entry.path())
|
||||
.ok()
|
||||
.map(|r| {
|
||||
r.filter_map(|e| e.ok())
|
||||
.filter_map(|e| e.metadata().ok())
|
||||
.map(|m| m.len())
|
||||
.sum()
|
||||
})
|
||||
.unwrap_or(0);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// fallback direct path check (for non-listed or still being written)
|
||||
if !exists {
|
||||
let p = std::path::Path::new(&models_dir).join(format!("{vbase}.cmf"));
|
||||
if p.exists() || p.is_dir() {
|
||||
exists = true;
|
||||
local_sz =
|
||||
std::fs::metadata(&p).map(|m| m.len()).unwrap_or_else(|_| {
|
||||
std::fs::read_dir(&p)
|
||||
.ok()
|
||||
.map(|r| {
|
||||
r.filter_map(|e| e.ok())
|
||||
.filter_map(|e| e.metadata().ok())
|
||||
.map(|m| m.len())
|
||||
.sum()
|
||||
})
|
||||
.unwrap_or(0)
|
||||
});
|
||||
}
|
||||
}
|
||||
let remote_sz = v["size_bytes"].as_u64().unwrap_or(0);
|
||||
let is_complete =
|
||||
exists && local_sz != 0 && remote_sz != 0 && local_sz == remote_sz;
|
||||
// downloaded = only when complete (not partial)
|
||||
downloaded = downloaded || is_complete;
|
||||
let mut vv = v.clone();
|
||||
vv["downloaded"] = json!(is_complete);
|
||||
vv["local_size"] = json!(local_sz);
|
||||
// the exact output stem the import will produce, and the id of
|
||||
// the job that produces it — the UI passes both back verbatim so
|
||||
// download / progress / delete can never disagree on the file.
|
||||
vv["local_name"] = json!(vbase);
|
||||
vv["job"] = json!(crate::import::gen_id(&format!("{id}::{vname}")));
|
||||
if exists && !is_complete && local_sz != 0 {
|
||||
// partial or outdated — show as update available if not currently downloading
|
||||
// we don't have job state here, so just mark update if sizes differ
|
||||
vv["update_available"] = json!(true);
|
||||
} else if is_complete && remote_sz != 0 && local_sz != remote_sz {
|
||||
vv["update_available"] = json!(true);
|
||||
}
|
||||
downloaded_variants.push(vv);
|
||||
}
|
||||
}
|
||||
let final_variants = if downloaded_variants.is_empty() {
|
||||
variants_json
|
||||
} else {
|
||||
json!(downloaded_variants)
|
||||
};
|
||||
let any_update = downloaded_variants
|
||||
.iter()
|
||||
.any(|v| v["update_available"].as_bool().unwrap_or(false));
|
||||
out.push(json!({
|
||||
"id": id,
|
||||
"label": id.split('/').next_back().unwrap_or(&id),
|
||||
"local_name": base,
|
||||
"badge": badge,
|
||||
"desc": m["pipeline_tag"].as_str().unwrap_or("text-generation"),
|
||||
"likes": likes,
|
||||
"size": size,
|
||||
"size_bytes": size_bytes,
|
||||
"variants": final_variants,
|
||||
"downloaded": downloaded,
|
||||
"update_available": any_update,
|
||||
"raw": m,
|
||||
}));
|
||||
}
|
||||
}
|
||||
if out.is_empty() {
|
||||
fallback
|
||||
} else {
|
||||
json!(out)
|
||||
}
|
||||
};
|
||||
ok(json!({ "models": arr }))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SetupFileQuery {
|
||||
repo: String,
|
||||
#[serde(default)]
|
||||
variant: Option<String>,
|
||||
/// Exact output stem (`local_name` from `setup/featured`). Preferred over the
|
||||
/// repo/variant guess, which cannot know how many variants a repo has.
|
||||
#[serde(default)]
|
||||
name: Option<String>,
|
||||
}
|
||||
|
||||
/// Drop a downloaded `.cmf` from the config: any managed server or pool model
|
||||
/// backed by that stem. Returns the ids that were unregistered.
|
||||
fn unregister_cmf_stem(state: &SharedState, stem: &str) -> Result<Vec<String>, ApiError> {
|
||||
let mut cfg = current_cfg(state);
|
||||
let stem_of = |path: &str| {
|
||||
std::path::Path::new(path)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_lowercase())
|
||||
};
|
||||
let mut ids: Vec<String> = cfg
|
||||
.cmf
|
||||
.servers
|
||||
.iter()
|
||||
.filter(|s| stem_of(&s.model).as_deref() == Some(stem))
|
||||
.map(|s| s.id.clone())
|
||||
.collect();
|
||||
for m in &cfg.models {
|
||||
if cmf_file_for_model(&cfg.cmf, m)
|
||||
.and_then(|p| stem_of(&p.to_string_lossy()))
|
||||
.as_deref()
|
||||
== Some(stem)
|
||||
{
|
||||
ids.push(m.id.clone());
|
||||
}
|
||||
}
|
||||
if ids.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
cfg.cmf
|
||||
.servers
|
||||
.retain(|s| stem_of(&s.model).as_deref() != Some(stem));
|
||||
if stem_of(&cfg.cmf.local_model).as_deref() == Some(stem) {
|
||||
cfg.cmf.local_model = String::new();
|
||||
cfg.cmf.manage_server = false;
|
||||
}
|
||||
cfg.models.retain(|m| !ids.contains(&m.id));
|
||||
for id in &ids {
|
||||
strip_id_from_routing(&mut cfg.routing, id);
|
||||
}
|
||||
state.reload(cfg)?;
|
||||
Ok(ids)
|
||||
}
|
||||
|
||||
async fn delete_setup_file(
|
||||
State(state): State<SharedState>,
|
||||
Query(q): Query<SetupFileQuery>,
|
||||
) -> ApiResult {
|
||||
let repo = q.repo.trim();
|
||||
if repo.is_empty() {
|
||||
return Err(ApiError::bad("repo is required"));
|
||||
}
|
||||
let live = state.live();
|
||||
let models_dir = live.cfg.cmf.models_dir.clone();
|
||||
let base = crate::import::sanitize_for_id(repo.split('/').next_back().unwrap_or(repo));
|
||||
let variant = q
|
||||
.variant
|
||||
.as_deref()
|
||||
.map(|s| s.trim())
|
||||
.filter(|s| !s.is_empty());
|
||||
// Exact stem wins; sanitizing it also keeps the delete inside models_dir.
|
||||
let vbase = match q.name.as_deref().map(str::trim).filter(|s| !s.is_empty()) {
|
||||
Some(n) => crate::import::sanitize_for_id(n),
|
||||
// for single-file the file is {base}.cmf, for sharded {base}-{variant}.cmf
|
||||
None => match variant {
|
||||
// only sharded variants need suffix; single file ignores
|
||||
Some(v) if v.starts_with("parts-") => {
|
||||
format!("{}-{}", base, crate::import::sanitize_for_id(v))
|
||||
}
|
||||
_ => base.clone(),
|
||||
},
|
||||
};
|
||||
if vbase.is_empty() {
|
||||
return Err(ApiError::bad("cannot derive a file name to delete"));
|
||||
}
|
||||
// Stop serving it before the file goes away, so a managed server doesn't keep
|
||||
// a deleted .cmf open (and the pool doesn't advertise a model that is gone).
|
||||
let unregistered = unregister_cmf_stem(&state, &vbase)?;
|
||||
for id in &unregistered {
|
||||
state.cmf.stop_one(id).await;
|
||||
}
|
||||
let p = std::path::Path::new(&models_dir).join(format!("{vbase}.cmf"));
|
||||
let mut removed = false;
|
||||
if p.exists() || p.is_dir() {
|
||||
removed = remove_cmf_path(&p);
|
||||
} else {
|
||||
// try case-insensitive search (macOS /tmp symlink, etc.)
|
||||
if let Ok(rd) = std::fs::read_dir(&models_dir) {
|
||||
let expected = format!("{vbase}.cmf").to_lowercase();
|
||||
for entry in rd.filter_map(|e| e.ok()) {
|
||||
if entry.file_name().to_string_lossy().to_lowercase() == expected {
|
||||
removed = remove_cmf_path(&entry.path());
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// also remove any job with same repo+variant id (so list doesn't keep stale "done")
|
||||
let jid = crate::import::gen_id(&format!("{}::{}", repo, variant.unwrap_or("")));
|
||||
let _ = state.imports.delete(&jid);
|
||||
// also try to delete by repo alone (old non-variant ids)
|
||||
if variant.is_none() {
|
||||
let jid2 = crate::import::gen_id(repo);
|
||||
let _ = state.imports.delete(&jid2);
|
||||
}
|
||||
ok(json!({ "ok": true, "removed": removed, "unregistered": unregistered }))
|
||||
}
|
||||
|
||||
async fn list_imports(State(state): State<SharedState>) -> ApiResult {
|
||||
ok(json!({ "jobs": state.imports.list() }))
|
||||
}
|
||||
@@ -1348,11 +1887,80 @@ async fn cancel_import(State(state): State<SharedState>, Path(job): Path<String>
|
||||
|
||||
/// Delete a finished conversion and its converted `.cmf` file(s) from disk.
|
||||
async fn delete_import(State(state): State<SharedState>, Path(job): Path<String>) -> ApiResult {
|
||||
if state.imports.get(&job).is_none() {
|
||||
let Some(j) = state.imports.get(&job) else {
|
||||
return Err(ApiError::not_found(format!("job '{job}' not found")));
|
||||
};
|
||||
// If it was already registered, drop it from the config first — otherwise the
|
||||
// pool keeps serving a model whose .cmf is about to disappear.
|
||||
let stem = std::path::Path::new(&j.output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_lowercase())
|
||||
.unwrap_or_default();
|
||||
let unregistered = if stem.is_empty() {
|
||||
Vec::new()
|
||||
} else {
|
||||
unregister_cmf_stem(&state, &stem)?
|
||||
};
|
||||
for id in &unregistered {
|
||||
state.cmf.stop_one(id).await;
|
||||
}
|
||||
let removed = state.imports.delete(&job).map_err(ApiError::bad)?;
|
||||
ok(json!({ "ok": removed }))
|
||||
ok(json!({ "ok": removed, "unregistered": unregistered }))
|
||||
}
|
||||
|
||||
/// Add a finished `.cmf` to the config as a *managed* local model: the gateway
|
||||
/// spawns and supervises its own `cortiq serve` for it. Registering it into the
|
||||
/// static pool instead would point the model at `cortiq_server_url` with nothing
|
||||
/// listening there, which is an upstream error the moment anyone calls it.
|
||||
///
|
||||
/// A stale static-pool entry for the same file is migrated rather than rejected,
|
||||
/// so pressing Register again repairs a model left dead by an earlier version.
|
||||
/// `None` = already registered and served.
|
||||
fn register_cmf_output(cfg: &mut crate::config::Config, output: &str) -> Option<String> {
|
||||
let base = std::path::Path::new(output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "cmf-local".into());
|
||||
let id = format!("cmf-{base}");
|
||||
let served = cfg.cmf.manage_server
|
||||
&& cfg
|
||||
.cmf
|
||||
.servers
|
||||
.iter()
|
||||
.any(|s| s.id == id || s.model == output);
|
||||
if served {
|
||||
return None;
|
||||
}
|
||||
// Same id sitting in the static pool → it is the dead form of this model.
|
||||
// Routing keeps referencing the id, so nothing needs re-pointing.
|
||||
cfg.models.retain(|m| m.id != id);
|
||||
cfg.cmf.servers.retain(|s| s.id != id && s.model != output);
|
||||
cfg.cmf.manage_server = true;
|
||||
let host = if cfg.cmf.local_host.trim().is_empty() {
|
||||
"127.0.0.1"
|
||||
} else {
|
||||
&cfg.cmf.local_host
|
||||
};
|
||||
let base_port = cfg.cmf.servers.iter().map(|s| s.port).max().unwrap_or(8089);
|
||||
let port = next_free_port(host, base_port).unwrap_or_else(|| base_port + 1);
|
||||
cfg.cmf.servers.push(crate::config::CmfServer {
|
||||
id: id.clone(),
|
||||
model: output.to_string(),
|
||||
port,
|
||||
threads: cfg.cmf.threads,
|
||||
gpu: cfg.cmf.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
});
|
||||
Some(id)
|
||||
}
|
||||
|
||||
/// Register a finished `.cmf` as a local model (OpenAI provider → cortiq-server).
|
||||
@@ -1364,27 +1972,19 @@ async fn register_import(State(state): State<SharedState>, Path(job): Path<Strin
|
||||
if j.state != "done" {
|
||||
return Err(ApiError::bad("conversion not finished"));
|
||||
}
|
||||
let base = std::path::Path::new(&j.output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "cmf-local".into());
|
||||
let id = format!("cmf-{base}");
|
||||
let mut cfg = current_cfg(&state);
|
||||
if cfg.models.iter().any(|m| m.id == id) {
|
||||
return Err(ApiError::bad(format!("model '{id}' already registered")));
|
||||
let id = register_cmf_output(&mut cfg, &j.output).ok_or_else(|| {
|
||||
ApiError::bad(format!(
|
||||
"model 'cmf-{}' is already registered and running",
|
||||
std::path::Path::new(&j.output)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_default()
|
||||
))
|
||||
})?;
|
||||
if cfg.routing.default.is_empty() {
|
||||
cfg.routing.default = id.clone();
|
||||
}
|
||||
cfg.models.push(crate::config::ModelCfg {
|
||||
id: id.clone(),
|
||||
provider: "openai".into(),
|
||||
base_url: cfg.cmf.cortiq_server_url.clone(),
|
||||
model: base,
|
||||
cost_tier: "local".into(),
|
||||
price_in: 0.0,
|
||||
price_out: 0.0,
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
});
|
||||
state.reload(cfg)?;
|
||||
ok(json!({ "ok": true, "model_id": id }))
|
||||
}
|
||||
|
||||
+22
-2
@@ -290,11 +290,31 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
|
||||
if server.threads > 1 {
|
||||
cmd.env("CMF_THREADS", server.threads.to_string());
|
||||
}
|
||||
// Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
|
||||
// on unified memory, so mostly a prefill win; opt-in.
|
||||
if server.gpu {
|
||||
cmd.env("CMF_GPU", "1");
|
||||
}
|
||||
// O(1) Nyström attention: layers spec + sub-parameters.
|
||||
if let Some(o1) = &server.o1 {
|
||||
let s = o1.trim();
|
||||
if !s.is_empty() {
|
||||
// Pass even "off" so it explicitly overrides a file converter hint.
|
||||
cmd.arg("--o1").arg(s);
|
||||
}
|
||||
}
|
||||
if let Some(m) = server.o1_m {
|
||||
cmd.arg("--o1-m").arg(m.to_string());
|
||||
}
|
||||
if let Some(w) = server.o1_window {
|
||||
cmd.arg("--o1-window").arg(w.to_string());
|
||||
}
|
||||
if let Some(sink) = server.o1_sink {
|
||||
cmd.arg("--o1-sink").arg(sink.to_string());
|
||||
}
|
||||
// MTP (Multi-Token Prediction) speculative decoding: the engine reads
|
||||
// CMF_MTP env; "0" disables. There is no CLI flag for this.
|
||||
if server.skip_mtp {
|
||||
cmd.env("CMF_MTP", "0");
|
||||
}
|
||||
cmd.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.kill_on_drop(true)
|
||||
|
||||
+160
-2
@@ -9,6 +9,9 @@ use std::path::PathBuf;
|
||||
/// `$CORTIQ_GATEWAY_HOME`, then `$XDG_DATA_HOME/cortiq-gateway`, then
|
||||
/// `~/.cortiq-gateway` — so defaults work regardless of the process CWD
|
||||
/// (important once the gateway is installed rather than run from its repo).
|
||||
/// On Windows also checks `%APPDATA%`, `%LOCALAPPDATA%` and `%USERPROFILE%`
|
||||
/// so double-clicking the exe from `Downloads` does not try to write into
|
||||
/// `Program Files`.
|
||||
pub fn data_dir() -> PathBuf {
|
||||
for (var, sub) in [
|
||||
("CORTIQ_GATEWAY_HOME", ""),
|
||||
@@ -25,6 +28,20 @@ pub fn data_dir() -> PathBuf {
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(windows)]
|
||||
{
|
||||
for (var, sub) in [
|
||||
("APPDATA", "cortiq-gateway"),
|
||||
("LOCALAPPDATA", "cortiq-gateway"),
|
||||
("USERPROFILE", ".cortiq-gateway"),
|
||||
] {
|
||||
if let Ok(v) = std::env::var(var) {
|
||||
if !v.is_empty() {
|
||||
return PathBuf::from(v).join(sub);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
PathBuf::from("data")
|
||||
}
|
||||
|
||||
@@ -83,6 +100,36 @@ pub struct CmfServer {
|
||||
/// Offload matvecs to the GPU (CMF_GPU=1).
|
||||
#[serde(default)]
|
||||
pub gpu: bool,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) Nyström attention layer spec (all | deepN | i,j,k | off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// O(1) landmark budget (default 32; ≥4).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_m: Option<u32>,
|
||||
/// O(1) exact-window width (default 128).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_window: Option<u32>,
|
||||
/// O(1) permanent exact sink keys (default 4, StreamingLLM discipline).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1_sink: Option<u32>,
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -172,6 +219,16 @@ impl CmfCfg {
|
||||
port: self.local_port,
|
||||
threads: self.threads,
|
||||
gpu: self.gpu,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
think_budget: None,
|
||||
system_prompt: None,
|
||||
o1: None,
|
||||
o1_m: None,
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
@@ -306,6 +363,27 @@ pub struct ModelCfg {
|
||||
pub api_key_env: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub caps: Vec<String>,
|
||||
/// Default generation temperature (0.0 .. 2.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
/// Default nucleus sampling threshold (0.0 .. 1.0).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
/// Default maximum completion tokens.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning / thinking token budget.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub think_budget: Option<u32>,
|
||||
/// Default system prompt prepended if request lacks one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub system_prompt: Option<String>,
|
||||
/// O(1) constant-memory streaming attention hint (all, deep, custom, off).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub o1: Option<String>,
|
||||
/// Skip Multi-Token Prediction (MTP) heads.
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
}
|
||||
fn default_kind() -> String {
|
||||
"chat".into()
|
||||
@@ -461,10 +539,12 @@ pub struct AdminCfg {
|
||||
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
|
||||
#[serde(default = "default_admin_enabled")]
|
||||
pub enabled: bool,
|
||||
/// Name of the env variable holding the admin token. If unset (or empty),
|
||||
/// a token is generated at startup and printed to the log.
|
||||
/// Name of the env variable holding the admin token.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token_env: Option<String>,
|
||||
/// Persistent admin token configured directly in toml.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub token: Option<String>,
|
||||
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
|
||||
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
@@ -475,6 +555,7 @@ impl Default for AdminCfg {
|
||||
Self {
|
||||
enabled: default_admin_enabled(),
|
||||
token_env: None,
|
||||
token: None,
|
||||
listen: None,
|
||||
}
|
||||
}
|
||||
@@ -638,6 +719,67 @@ fn default_semcache_max() -> usize {
|
||||
1000
|
||||
}
|
||||
|
||||
/// Sensible defaults for a fresh install (empty pool — user fills via wizard/UI).
|
||||
impl Default for Config {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
listen: "0.0.0.0:9000".to_string(),
|
||||
router: RouterCfg {
|
||||
enabled: true,
|
||||
url: "http://localhost:8080".to_string(),
|
||||
timeout_ms: default_router_timeout(),
|
||||
verify_tls: true,
|
||||
api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
|
||||
taxonomy_id: Some("general.task-type".to_string()),
|
||||
},
|
||||
route: RouteCfg::default(),
|
||||
models: Vec::new(),
|
||||
routing: RoutingCfg {
|
||||
tiers: HashMap::new(),
|
||||
default: String::new(),
|
||||
policy: RoutingPolicy::default(),
|
||||
},
|
||||
breaker: BreakerCfg::default(),
|
||||
protocols: ProtocolsCfg {
|
||||
openai_chat: true,
|
||||
openai_completions: true,
|
||||
openai_embeddings: true,
|
||||
openai_models: true,
|
||||
anthropic_messages: false,
|
||||
mcp: false,
|
||||
native_passthrough: true,
|
||||
},
|
||||
cortiq: CortiqCfg::default(),
|
||||
api_keys: Vec::new(),
|
||||
idempotency: IdempotencyCfg::default(),
|
||||
log: LogCfg::default(),
|
||||
telemetry: TelemetryCfg::default(),
|
||||
admin: AdminCfg::default(),
|
||||
stats: StatsCfg::default(),
|
||||
cache: CacheCfg::default(),
|
||||
shadow: ShadowCfg::default(),
|
||||
cmf: CmfCfg::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Where the config lives when `--config` is not given.
|
||||
/// Repo checkout (`config/gateway.toml` exists or `config/` dir exists) → keep it there;
|
||||
/// installed binary (no repo) → `$data_dir/gateway.toml`.
|
||||
pub fn default_config_path() -> String {
|
||||
let repo_path = "config/gateway.toml";
|
||||
if std::path::Path::new(repo_path).exists() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
if std::path::Path::new("Cargo.toml").exists() && std::path::Path::new("config").exists() {
|
||||
return repo_path.to_string();
|
||||
}
|
||||
data_dir()
|
||||
.join("gateway.toml")
|
||||
.to_string_lossy()
|
||||
.into_owned()
|
||||
}
|
||||
|
||||
impl Config {
|
||||
/// Load and validate the config from a file.
|
||||
pub fn load(path: &str) -> anyhow::Result<Self> {
|
||||
@@ -648,6 +790,22 @@ impl Config {
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Load if exists, otherwise create a fresh default config at `path` and return it.
|
||||
/// Never requires the user to create or copy a file by hand.
|
||||
pub fn load_or_create(path: &str) -> anyhow::Result<Self> {
|
||||
if std::path::Path::new(path).exists() {
|
||||
return Self::load(path);
|
||||
}
|
||||
let cfg = Self::default();
|
||||
// ensure parent dir exists
|
||||
if let Some(dir) = std::path::Path::new(path).parent() {
|
||||
let _ = std::fs::create_dir_all(dir);
|
||||
}
|
||||
cfg.save(path)?;
|
||||
eprintln!("created default config at {path} — open /admin to finish setup");
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
/// Integrity validation: non-empty model pool, valid routing targets.
|
||||
pub fn validate(&self) -> anyhow::Result<()> {
|
||||
// A managed local CMF model counts as an available model even though it
|
||||
|
||||
+752
-17
@@ -30,6 +30,47 @@ fn cleanup_output(path: &str) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Bytes an output occupies: a file's length, or the recursive sum for a sharded
|
||||
/// `.cmf` directory. 0 when nothing has been written yet.
|
||||
pub fn path_size(p: &std::path::Path) -> u64 {
|
||||
match std::fs::metadata(p) {
|
||||
Ok(m) if m.is_file() => m.len(),
|
||||
Ok(m) if m.is_dir() => std::fs::read_dir(p)
|
||||
.map(|rd| {
|
||||
rd.filter_map(|e| e.ok())
|
||||
.map(|e| path_size(&e.path()))
|
||||
.sum()
|
||||
})
|
||||
.unwrap_or(0),
|
||||
_ => 0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Aborts the wrapped task when it goes out of scope — used for the progress
|
||||
/// poller so every early return in a download path stops it.
|
||||
struct AbortOnDrop(tokio::task::JoinHandle<()>);
|
||||
impl Drop for AbortOnDrop {
|
||||
fn drop(&mut self) {
|
||||
self.0.abort();
|
||||
}
|
||||
}
|
||||
|
||||
/// Drive progress from bytes on disk. curl's own meter is unreliable here (its
|
||||
/// bar is redrawn with `\r` into a pipe, and a sharded download runs one curl
|
||||
/// per part), but the output size never lies.
|
||||
fn spawn_size_poller(store: Arc<JobStore>, id: String, path: String) -> AbortOnDrop {
|
||||
AbortOnDrop(tokio::spawn(async move {
|
||||
let p = std::path::PathBuf::from(&path);
|
||||
loop {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(600)).await;
|
||||
let done = path_size(&p);
|
||||
if done > 0 {
|
||||
store.set_bytes(&id, done);
|
||||
}
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
fn now() -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
@@ -37,16 +78,12 @@ fn now() -> u64 {
|
||||
.as_secs()
|
||||
}
|
||||
|
||||
/// Job id from a hash of (repo, nanos) — unique without a uuid dependency.
|
||||
fn gen_id(repo: &str) -> String {
|
||||
/// Job id from repo (deterministic) — same repo+variant reuses same id, so
|
||||
/// the job list doesn't fill with duplicates and "already downloaded" can be detected.
|
||||
pub fn gen_id(repo: &str) -> String {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos();
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
repo.hash(&mut h);
|
||||
nanos.hash(&mut h);
|
||||
format!("{:016x}", h.finish())
|
||||
}
|
||||
|
||||
@@ -58,6 +95,10 @@ pub struct ImportParams {
|
||||
pub quant: String,
|
||||
#[serde(default)]
|
||||
pub name: String, // output basename; empty → derived from repo
|
||||
/// For sharded CMF repos (e.g. DeepSeek with `parts-q2tp-v2`/`parts-q4tp`), which
|
||||
/// variant directory to download. Empty = auto (first .cmf or first variant).
|
||||
#[serde(default)]
|
||||
pub variant: Option<String>,
|
||||
// ── advanced ──
|
||||
#[serde(default)]
|
||||
pub linear_core: Option<String>, // gated_delta_net | vmf_phase
|
||||
@@ -97,6 +138,11 @@ pub struct Job {
|
||||
pub repo: String,
|
||||
pub quant: String,
|
||||
pub output: String,
|
||||
/// Output basename without `.cmf` — the id the UI uses to match a job to the
|
||||
/// featured card that started it, and to the file on disk.
|
||||
pub name: String,
|
||||
/// Sharded-variant directory / `.cmf` filename this job downloads, if any.
|
||||
pub variant: Option<String>,
|
||||
pub state: String, // running | done | error | cancelled
|
||||
pub log: Vec<String>,
|
||||
pub started: u64,
|
||||
@@ -104,6 +150,10 @@ pub struct Job {
|
||||
pub size_bytes: Option<u64>,
|
||||
pub progress: Option<f32>, // 0..1, parsed from converter @PROGRESS
|
||||
pub phase: Option<String>, // current phase label
|
||||
/// Expected download size (from the HF tree) and bytes already on disk —
|
||||
/// what makes the percentage in the UI real rather than a guess.
|
||||
pub total_bytes: Option<u64>,
|
||||
pub done_bytes: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
@@ -154,12 +204,60 @@ impl JobStore {
|
||||
}
|
||||
fn set_progress(&self, id: &str, frac: f32, phase: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.progress = Some(frac);
|
||||
let f = frac.clamp(0.0, 1.0);
|
||||
let cur = j.progress.unwrap_or(0.0);
|
||||
// monotonic only — never go backwards (curl reprints, re-download),
|
||||
// but an unset bar takes whatever the first reading is
|
||||
if j.progress.is_none() || f > cur + 0.001 {
|
||||
j.progress = Some(f);
|
||||
}
|
||||
if !phase.is_empty() {
|
||||
j.phase = Some(phase);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// Expected total download size, once the HF tree is known.
|
||||
fn set_total(&self, id: &str, total: u64) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
if total > 0 {
|
||||
j.total_bytes = Some(total);
|
||||
}
|
||||
}
|
||||
}
|
||||
/// Report bytes on disk; derives the percentage whenever the total is known.
|
||||
fn set_bytes(&self, id: &str, done: u64) {
|
||||
let total = {
|
||||
let mut g = self.jobs.lock().unwrap();
|
||||
let Some(j) = g.get_mut(id) else { return };
|
||||
if j.state != "running" {
|
||||
return;
|
||||
}
|
||||
if done > j.done_bytes.unwrap_or(0) {
|
||||
j.done_bytes = Some(done);
|
||||
}
|
||||
j.total_bytes
|
||||
};
|
||||
if let Some(t) = total.filter(|t| *t > 0) {
|
||||
self.set_progress(id, done as f32 / t as f32, "downloading".into());
|
||||
}
|
||||
}
|
||||
fn set_phase(&self, id: &str, phase: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.phase = Some(phase);
|
||||
// keep progress as is; if it was 0.0, make it indeterminate (None) so bar animates
|
||||
if j.progress == Some(0.0) {
|
||||
j.progress = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
fn update_quant(&self, id: &str, quant: String) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.quant = quant.clone();
|
||||
if !j.log.is_empty() && j.log[0].starts_with("→ downloading ready .cmf model") {
|
||||
j.log[0] = format!("→ downloading ready .cmf model {} ({})", j.repo, quant);
|
||||
}
|
||||
}
|
||||
}
|
||||
fn set_state(&self, id: &str, state: &str) {
|
||||
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
|
||||
j.state = state.into();
|
||||
@@ -172,11 +270,14 @@ impl JobStore {
|
||||
return; // a concurrent cancel already settled this job
|
||||
}
|
||||
j.finished = Some(now());
|
||||
j.size_bytes = std::fs::metadata(&j.output).ok().map(|m| m.len());
|
||||
let done = ok && j.size_bytes.unwrap_or(0) > 0;
|
||||
let on_disk = path_size(std::path::Path::new(&j.output));
|
||||
j.size_bytes = if on_disk > 0 { Some(on_disk) } else { None };
|
||||
let done = ok && on_disk > 0;
|
||||
j.state = if done { "done" } else { "error" }.into();
|
||||
if done {
|
||||
j.progress = Some(1.0);
|
||||
j.done_bytes = Some(on_disk);
|
||||
j.total_bytes = Some(j.total_bytes.unwrap_or(on_disk).max(on_disk));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -197,7 +298,19 @@ impl JobStore {
|
||||
Ok(true)
|
||||
}
|
||||
pub fn list(&self) -> Vec<Job> {
|
||||
let mut v: Vec<Job> = self.jobs.lock().unwrap().values().cloned().collect();
|
||||
let mut map: HashMap<String, Job> = HashMap::new();
|
||||
for j in self.jobs.lock().unwrap().values().cloned() {
|
||||
// dedupe by repo+variant — keep the latest (so re-download doesn't create
|
||||
// duplicate rows) while two variants of one repo stay separate rows
|
||||
let key = format!("{}::{}", j.repo, j.variant.clone().unwrap_or_default());
|
||||
match map.get(&key) {
|
||||
Some(prev) if prev.started >= j.started => {}
|
||||
_ => {
|
||||
map.insert(key, j);
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut v: Vec<Job> = map.into_values().collect();
|
||||
v.sort_by_key(|x| std::cmp::Reverse(x.started));
|
||||
v.truncate(20);
|
||||
v
|
||||
@@ -234,6 +347,153 @@ pub async fn hf_search(
|
||||
resp.json().await.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
pub async fn hf_search_author(
|
||||
author: &str,
|
||||
limit: usize,
|
||||
token: Option<&str>,
|
||||
) -> Result<serde_json::Value, String> {
|
||||
let url = format!(
|
||||
"https://huggingface.co/api/models?author={}&limit={}&sort=trendingScore&direction=-1",
|
||||
url_escape(author),
|
||||
limit
|
||||
);
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(15))
|
||||
.build()
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = req.send().await.map_err(|e| e.to_string())?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(format!("HuggingFace API {}", resp.status()));
|
||||
}
|
||||
resp.json().await.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
pub async fn hf_model_size(id: &str, token: Option<&str>) -> Option<u64> {
|
||||
let url = format!("https://huggingface.co/api/models/{}", url_escape(id));
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(8))
|
||||
.build()
|
||||
.ok()?;
|
||||
let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = req.send().await.ok()?;
|
||||
if !resp.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
let v: serde_json::Value = resp.json().await.ok()?;
|
||||
v.get("usedStorage").and_then(|x| x.as_u64())
|
||||
}
|
||||
|
||||
/// Per-variant sizes for a repo (for CMF repos with sharded directories like
|
||||
/// `parts-q2tp-v2`, `parts-q4tp`). Returns `[(variant_name, bytes)]`. For a
|
||||
/// single-file repo returns one entry. Empty = tree unreachable.
|
||||
pub async fn hf_repo_variants(id: &str, token: Option<&str>) -> Vec<(String, u64)> {
|
||||
let url = format!(
|
||||
"https://huggingface.co/api/models/{}/tree/main?recursive=true",
|
||||
url_escape(id)
|
||||
);
|
||||
let client = match reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
{
|
||||
Ok(c) => c,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = match req.send().await {
|
||||
Ok(r) if r.status().is_success() => r,
|
||||
_ => return Vec::new(),
|
||||
};
|
||||
let arr: serde_json::Value = match resp.json().await {
|
||||
Ok(v) => v,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
let list = match arr.as_array() {
|
||||
Some(a) => a,
|
||||
None => return Vec::new(),
|
||||
};
|
||||
use std::collections::HashMap;
|
||||
let mut dir_sizes: HashMap<String, u64> = HashMap::new();
|
||||
let mut single_cmf: Vec<(String, u64)> = Vec::new();
|
||||
for item in list {
|
||||
if item.get("type").and_then(|t| t.as_str()) != Some("file") {
|
||||
continue;
|
||||
}
|
||||
let path = match item.get("path").and_then(|p| p.as_str()) {
|
||||
Some(p) => p,
|
||||
None => continue,
|
||||
};
|
||||
let sz = item
|
||||
.get("size")
|
||||
.and_then(|v| v.as_u64())
|
||||
.or_else(|| {
|
||||
item.get("lfs")
|
||||
.and_then(|l| l.get("size"))
|
||||
.and_then(|v| v.as_u64())
|
||||
})
|
||||
.unwrap_or(0);
|
||||
if sz == 0 {
|
||||
continue;
|
||||
}
|
||||
if path.to_ascii_lowercase().ends_with(".cmf") {
|
||||
single_cmf.push((path.to_string(), sz));
|
||||
} else if path.contains('/') && path.starts_with("parts-") {
|
||||
let top = path.split('/').next().unwrap_or(path).to_string();
|
||||
*dir_sizes.entry(top).or_default() += sz;
|
||||
}
|
||||
}
|
||||
if !single_cmf.is_empty() {
|
||||
return single_cmf;
|
||||
}
|
||||
let mut out: Vec<(String, u64)> = dir_sizes.into_iter().collect();
|
||||
out.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
out
|
||||
}
|
||||
|
||||
/// `Content-Length` of a download URL (redirects followed) — the fallback total
|
||||
/// when the repo tree didn't tell us how big the file is.
|
||||
async fn head_content_length(url: &str, token: Option<&str>) -> Option<u64> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.ok()?;
|
||||
let mut req = client.head(url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
let resp = req.send().await.ok()?;
|
||||
if !resp.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
resp.headers()
|
||||
.get(reqwest::header::CONTENT_LENGTH)?
|
||||
.to_str()
|
||||
.ok()?
|
||||
.parse()
|
||||
.ok()
|
||||
}
|
||||
|
||||
pub fn format_bytes(n: u64) -> String {
|
||||
if n >= 1_000_000_000 {
|
||||
format!("{:.1} GB", n as f64 / 1_000_000_000.0)
|
||||
} else if n >= 1_000_000 {
|
||||
format!("{} MB", n / 1_000_000)
|
||||
} else if n >= 1_000 {
|
||||
format!("{} KB", n / 1_000)
|
||||
} else {
|
||||
format!("{n} B")
|
||||
}
|
||||
}
|
||||
|
||||
fn url_escape(s: &str) -> String {
|
||||
s.chars()
|
||||
.map(|c| match c {
|
||||
@@ -255,19 +515,389 @@ fn sanitize(s: &str) -> String {
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
pub fn sanitize_for_id(s: &str) -> String {
|
||||
sanitize(&s.to_lowercase())
|
||||
}
|
||||
|
||||
/// Kick off a conversion. Returns the job id immediately; progress streams
|
||||
/// Map the UI quant label to the native `cortiq` quant flag.
|
||||
fn map_quant(q: &str) -> &'static str {
|
||||
match q.to_ascii_uppercase().as_str() {
|
||||
s if s.starts_with("Q4T") || s == "Q4_TILED" => "q4t",
|
||||
s if s.starts_with("Q4") => "q4",
|
||||
"Q8_2F" => "q8_2f",
|
||||
"Q8_2F" | "Q82F" => "q8_2f",
|
||||
"F16" | "FP16" => "f16",
|
||||
"VBIT" => "vbit",
|
||||
"Q1" => "q1",
|
||||
"Q1P" | "Q1_PTQ" => "q1p",
|
||||
"Q1S" | "Q1_MASK" => "q1s",
|
||||
"Q1T" | "Q1_TERNARY" => "q1t",
|
||||
_ => "q8",
|
||||
}
|
||||
}
|
||||
|
||||
fn is_cmf_repo(repo: &str) -> bool {
|
||||
let lower = repo.to_ascii_lowercase();
|
||||
lower.contains("cmf") || lower.ends_with(".cmf")
|
||||
}
|
||||
|
||||
async fn download_cmf_repo(
|
||||
repo: String,
|
||||
output_abs: String,
|
||||
hf_token: Option<String>,
|
||||
store: Arc<JobStore>,
|
||||
id: String,
|
||||
cancel: Arc<Notify>,
|
||||
variant: Option<String>,
|
||||
) {
|
||||
// don't show 0% "listing files" — it looks stuck; show indeterminate "preparing"
|
||||
store.set_phase(&id, "preparing".into());
|
||||
|
||||
let tree_url = format!("https://huggingface.co/api/models/{repo}/tree/main?recursive=true");
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()
|
||||
.unwrap_or_else(|_| reqwest::Client::new());
|
||||
let mut req = client.get(&tree_url).header("User-Agent", "cortiq-gateway");
|
||||
if let Some(t) = &hf_token {
|
||||
req = req.bearer_auth(t);
|
||||
}
|
||||
|
||||
let mut cmf_files: Vec<(String, u64)> = Vec::new();
|
||||
let mut sharded_dirs: std::collections::HashMap<String, Vec<String>> =
|
||||
std::collections::HashMap::new();
|
||||
let mut dir_sizes: std::collections::HashMap<String, u64> = std::collections::HashMap::new();
|
||||
if let Ok(resp) = req.send().await {
|
||||
if resp.status().is_success() {
|
||||
if let Ok(json) = resp.json::<serde_json::Value>().await {
|
||||
if let Some(arr) = json.as_array() {
|
||||
for item in arr {
|
||||
if item.get("type").and_then(|t| t.as_str()) != Some("file") {
|
||||
continue;
|
||||
}
|
||||
if let Some(path) = item.get("path").and_then(|p| p.as_str()) {
|
||||
let sz = item
|
||||
.get("size")
|
||||
.and_then(|v| v.as_u64())
|
||||
.or_else(|| {
|
||||
item.get("lfs")
|
||||
.and_then(|l| l.get("size"))
|
||||
.and_then(|v| v.as_u64())
|
||||
})
|
||||
.unwrap_or(0);
|
||||
if path.to_ascii_lowercase().ends_with(".cmf") {
|
||||
cmf_files.push((path.to_string(), sz));
|
||||
} else if path.contains('/') && path.starts_with("parts-") {
|
||||
let top = path.split('/').next().unwrap_or(path).to_string();
|
||||
sharded_dirs
|
||||
.entry(top.clone())
|
||||
.or_default()
|
||||
.push(path.to_string());
|
||||
*dir_sizes.entry(top).or_default() += sz;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Decide what to download: single .cmf file vs sharded variant directory
|
||||
let is_sharded = !sharded_dirs.is_empty();
|
||||
let chosen_variant = variant
|
||||
.as_deref()
|
||||
.map(|v| v.trim())
|
||||
.filter(|v| !v.is_empty());
|
||||
|
||||
if is_sharded {
|
||||
// pick variant: explicit > first alphabetically
|
||||
let target_dir = if let Some(v) = chosen_variant {
|
||||
if sharded_dirs.contains_key(v) {
|
||||
v.to_string()
|
||||
} else {
|
||||
sharded_dirs.keys().next().cloned().unwrap_or_default()
|
||||
}
|
||||
} else {
|
||||
// prefer smallest? use first sorted
|
||||
let mut keys: Vec<String> = sharded_dirs.keys().cloned().collect();
|
||||
keys.sort();
|
||||
keys.into_iter().next().unwrap_or_default()
|
||||
};
|
||||
let files = sharded_dirs.get(&target_dir).cloned().unwrap_or_default();
|
||||
if files.is_empty() {
|
||||
store.push_line(&id, format!("✗ variant '{target_dir}' not found"));
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
let total_sz = dir_sizes.get(&target_dir).copied().unwrap_or(0);
|
||||
let q = if target_dir.contains("q2tp") {
|
||||
"Q2TP"
|
||||
} else if target_dir.contains("q4tp") {
|
||||
"Q4TP"
|
||||
} else {
|
||||
"READY_CMF"
|
||||
};
|
||||
store.update_quant(&id, q.to_string());
|
||||
store.push_line(
|
||||
&id,
|
||||
format!(
|
||||
"→ downloading {} ({} files, {}) from HuggingFace...",
|
||||
target_dir,
|
||||
files.len(),
|
||||
format_bytes(total_sz)
|
||||
),
|
||||
);
|
||||
store.set_total(&id, total_sz);
|
||||
store.set_progress(&id, 0.001, "downloading".into());
|
||||
|
||||
// output is a directory: ensure {base}.cmf is a directory
|
||||
let out_dir = std::path::Path::new(&output_abs);
|
||||
// Drop a previous copy first (re-download / update). There is no resume,
|
||||
// so it would be overwritten anyway — and leaving it there would make the
|
||||
// size poller read the old bytes and report 100% before we transfer any.
|
||||
cleanup_output(&output_abs);
|
||||
// if output ends with .cmf and variant is sharded, keep it as dir (e.g. model.cmf/parts-*)
|
||||
// create base dir
|
||||
let _ = std::fs::create_dir_all(out_dir);
|
||||
// live percentage from bytes landing on disk (curl runs quiet, per part)
|
||||
let _poller = spawn_size_poller(store.clone(), id.clone(), output_abs.clone());
|
||||
// download each part file
|
||||
for (idx, rel) in files.iter().enumerate() {
|
||||
let filename = rel.rsplit('/').next().unwrap_or(rel);
|
||||
let dest = out_dir.join(filename);
|
||||
let url = format!("https://huggingface.co/{repo}/resolve/main/{rel}");
|
||||
store.push_line(&id, format!(" [{}/{}] {filename}", idx + 1, files.len()));
|
||||
let mut cmd = tokio::process::Command::new("curl");
|
||||
cmd.arg("-f")
|
||||
.arg("-L")
|
||||
.arg("-s")
|
||||
.arg("-S")
|
||||
.arg("-o")
|
||||
.arg(&dest)
|
||||
.arg(&url);
|
||||
if let Some(t) = &hf_token {
|
||||
cmd.arg("-H").arg(format!("Authorization: Bearer {t}"));
|
||||
}
|
||||
cmd.stdout(Stdio::null());
|
||||
cmd.stderr(Stdio::piped());
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed {filename}: {e}"));
|
||||
cleanup_output(&output_abs);
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let t_err = child.stderr.take().map(|s| {
|
||||
let st = store.clone();
|
||||
let jid = id.clone();
|
||||
tokio::spawn(async move {
|
||||
let mut lines = BufReader::new(s).lines();
|
||||
while let Ok(Some(l)) = lines.next_line().await {
|
||||
let l = l.trim().to_string();
|
||||
if !l.is_empty() {
|
||||
st.push_line(&jid, l);
|
||||
}
|
||||
}
|
||||
})
|
||||
});
|
||||
// Race the part against a cancel — a single part can be many GB, so
|
||||
// waiting for it to finish first would make Cancel look dead.
|
||||
let status = tokio::select! {
|
||||
r = child.wait() => {
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
r.map_err(|e| e.to_string())
|
||||
}
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, "✗ cancelled by user — partial output removed".into());
|
||||
store.set_state(&id, "cancelled");
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
match status {
|
||||
Ok(s) if s.success() => {
|
||||
// exact byte count after each part (the poller covers the gaps)
|
||||
let on_disk = path_size(out_dir);
|
||||
if total_sz > 0 {
|
||||
store.set_bytes(&id, on_disk);
|
||||
} else {
|
||||
let frac = (idx + 1) as f32 / files.len() as f32;
|
||||
store.set_progress(&id, frac, "downloading".into());
|
||||
}
|
||||
}
|
||||
Ok(s) => {
|
||||
store.push_line(&id, format!("✗ failed {filename}: exit {:?}", s.code()));
|
||||
cleanup_output(&output_abs);
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed {filename}: {e}"));
|
||||
cleanup_output(&output_abs);
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
store.push_line(&id, "✓ done".into());
|
||||
store.finish(&id, true);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
|
||||
// single-file path (original logic)
|
||||
let (cmf_path, cmf_size) = if !cmf_files.is_empty() {
|
||||
// if variant specified and matches a file prefix, prefer it
|
||||
if let Some(v) = chosen_variant {
|
||||
cmf_files
|
||||
.iter()
|
||||
.find(|(p, _)| p.contains(v))
|
||||
.cloned()
|
||||
.unwrap_or_else(|| cmf_files[0].clone())
|
||||
} else {
|
||||
cmf_files[0].clone()
|
||||
}
|
||||
} else {
|
||||
let repo_name = repo.rsplit('/').next().unwrap_or(&repo);
|
||||
(format!("{repo_name}.cmf"), 0)
|
||||
};
|
||||
|
||||
let path_lower = cmf_path.to_ascii_lowercase();
|
||||
let detected_quant =
|
||||
if path_lower.contains("-q1t.") || repo.to_ascii_lowercase().contains("2bit") {
|
||||
"Q1T".to_string()
|
||||
} else if path_lower.contains("-q1.") || repo.to_ascii_lowercase().contains("cmf") {
|
||||
"Q1".to_string()
|
||||
} else if path_lower.contains("-q4.") {
|
||||
"Q4".to_string()
|
||||
} else if path_lower.contains("-q8.") {
|
||||
"Q8".to_string()
|
||||
} else {
|
||||
"READY_CMF".to_string()
|
||||
};
|
||||
store.update_quant(&id, detected_quant);
|
||||
|
||||
let download_url = format!("https://huggingface.co/{repo}/resolve/main/{cmf_path}");
|
||||
// The tree listing is the usual source of the size; ask the CDN directly when
|
||||
// it wasn't (private repo, tree fetch failed) so the bar still gets a total.
|
||||
let total = if cmf_size > 0 {
|
||||
cmf_size
|
||||
} else {
|
||||
head_content_length(&download_url, hf_token.as_deref())
|
||||
.await
|
||||
.unwrap_or(0)
|
||||
};
|
||||
store.push_line(&id, {
|
||||
let sz = if total > 0 {
|
||||
format!(" ({})", format_bytes(total))
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
format!("→ downloading {cmf_path}{sz} from HuggingFace...")
|
||||
});
|
||||
store.set_total(&id, total);
|
||||
store.set_progress(&id, 0.001, "downloading".into());
|
||||
// Drop a previous copy first (re-download / update). There is no resume, so
|
||||
// it would be overwritten anyway — and leaving it there would make the size
|
||||
// poller read the old bytes and report 100% before we transfer any.
|
||||
cleanup_output(&output_abs);
|
||||
// Progress comes from bytes on disk only. curl's own meter is worse than
|
||||
// useless here: with -L it prints a complete 0→100% bar for the redirect hop
|
||||
// before the real transfer starts, which used to pin the job at 100%.
|
||||
let _poller = spawn_size_poller(store.clone(), id.clone(), output_abs.clone());
|
||||
|
||||
let mut cmd = tokio::process::Command::new("curl");
|
||||
cmd.arg("-f")
|
||||
.arg("-L")
|
||||
.arg("-s")
|
||||
.arg("-S")
|
||||
.arg("-o")
|
||||
.arg(&output_abs);
|
||||
|
||||
if let Some(t) = &hf_token {
|
||||
cmd.arg("-H").arg(format!("Authorization: Bearer {t}"));
|
||||
}
|
||||
|
||||
cmd.arg(&download_url);
|
||||
cmd.stderr(std::process::Stdio::piped());
|
||||
cmd.stdout(std::process::Stdio::null());
|
||||
|
||||
let mut child = match cmd.spawn() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
store.push_line(&id, format!("✗ failed to start curl download: {e}"));
|
||||
store.finish(&id, false);
|
||||
store.drop_cancel(&id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// -sS leaves stderr for real errors only — drain it into the log so a failure
|
||||
// says what went wrong instead of just an exit code.
|
||||
let stderr = child.stderr.take();
|
||||
let mut t_err = None;
|
||||
if let Some(err_stream) = stderr {
|
||||
let store_clone = store.clone();
|
||||
let id_clone = id.clone();
|
||||
t_err = Some(tokio::spawn(async move {
|
||||
let mut lines = BufReader::new(err_stream).lines();
|
||||
while let Ok(Some(l)) = lines.next_line().await {
|
||||
let l = l.trim().to_string();
|
||||
if !l.is_empty() {
|
||||
store_clone.push_line(&id_clone, l);
|
||||
}
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
tokio::select! {
|
||||
status_res = child.wait() => {
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
match status_res {
|
||||
Ok(status) if status.success() => {
|
||||
store.push_line(&id, "✓ done".into());
|
||||
store.finish(&id, true);
|
||||
}
|
||||
Ok(status) => {
|
||||
cleanup_output(&output_abs);
|
||||
// curl 23 = write error: almost always a non-writable models dir
|
||||
let hint = match status.code() {
|
||||
Some(23) => " (write error — is the models dir writable?)",
|
||||
Some(22) => " (HTTP error — repo/file not found or auth required)",
|
||||
_ => "",
|
||||
};
|
||||
store.push_line(&id, format!("✗ download failed: exit code {:?}{hint}", status.code()));
|
||||
store.finish(&id, false);
|
||||
}
|
||||
Err(e) => {
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, format!("✗ download process error: {e}"));
|
||||
store.finish(&id, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
_ = cancel.notified() => {
|
||||
let _ = child.start_kill();
|
||||
let _ = child.wait().await;
|
||||
if let Some(t) = t_err { t.abort(); }
|
||||
cleanup_output(&output_abs);
|
||||
store.push_line(&id, "✗ cancelled by user — partial output removed".into());
|
||||
store.set_state(&id, "cancelled");
|
||||
}
|
||||
}
|
||||
store.drop_cancel(&id);
|
||||
}
|
||||
|
||||
/// into the job log. Fails fast if the converter script is missing.
|
||||
pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Result<String, String> {
|
||||
if p.repo.trim().is_empty() {
|
||||
@@ -311,12 +941,32 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
}
|
||||
}
|
||||
}
|
||||
let _ = std::fs::create_dir_all(&cfg.models_dir);
|
||||
let base = if p.name.trim().is_empty() {
|
||||
std::fs::create_dir_all(&cfg.models_dir).map_err(|e| {
|
||||
format!(
|
||||
"cannot create models dir {}: {e} — check that the gateway has write \
|
||||
access (in Docker, mount a writable volume at the models path)",
|
||||
cfg.models_dir
|
||||
)
|
||||
})?;
|
||||
let mut base = if p.name.trim().is_empty() {
|
||||
sanitize(p.repo.rsplit('/').next().unwrap_or(&p.repo))
|
||||
} else {
|
||||
sanitize(&p.name)
|
||||
};
|
||||
// for sharded variants make output distinct (e.g. DeepSeek parts-q2tp vs q4tp); single-file CMF ignores variant
|
||||
if let Some(v) = p
|
||||
.variant
|
||||
.as_deref()
|
||||
.map(|s| s.trim())
|
||||
.filter(|s| !s.is_empty())
|
||||
{
|
||||
if v.starts_with("parts-") {
|
||||
let vs = sanitize(v);
|
||||
if !base.ends_with(&vs) {
|
||||
base = format!("{base}-{vs}");
|
||||
}
|
||||
}
|
||||
}
|
||||
let output = std::path::Path::new(&cfg.models_dir)
|
||||
.join(format!("{base}.cmf"))
|
||||
.to_string_lossy()
|
||||
@@ -325,19 +975,66 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
.map(|d| d.join(format!("{base}.cmf")).to_string_lossy().to_string())
|
||||
.unwrap_or_else(|_| output.clone());
|
||||
|
||||
let id = gen_id(&p.repo);
|
||||
let id = gen_id(&format!(
|
||||
"{}::{}",
|
||||
p.repo,
|
||||
p.variant.as_deref().unwrap_or("")
|
||||
));
|
||||
// if same repo/variant is already running, don't reset progress — just return it
|
||||
if let Some(existing) = store.get(&id) {
|
||||
if existing.state == "running" {
|
||||
return Ok(id);
|
||||
}
|
||||
}
|
||||
let is_cmf = is_cmf_repo(&p.repo);
|
||||
let quant_display = if is_cmf {
|
||||
let r = p.repo.to_lowercase();
|
||||
if r.contains("2bit") || r.contains("q1t") {
|
||||
"Q1T".to_string()
|
||||
} else if r.contains("1.7bcmf")
|
||||
|| r.contains("27bcmf")
|
||||
|| r.contains("cmf")
|
||||
|| p.quant.is_empty()
|
||||
|| p.quant == "auto"
|
||||
|| p.quant == "Q8_2F"
|
||||
{
|
||||
"Q1".to_string()
|
||||
} else {
|
||||
p.quant.clone()
|
||||
}
|
||||
} else if p.quant.is_empty() || p.quant == "auto" {
|
||||
"q8".to_string()
|
||||
} else {
|
||||
p.quant.clone()
|
||||
};
|
||||
store.insert(Job {
|
||||
id: id.clone(),
|
||||
repo: p.repo.clone(),
|
||||
quant: p.quant.clone(),
|
||||
quant: quant_display.clone(),
|
||||
output: output_abs.clone(),
|
||||
name: base.clone(),
|
||||
variant: p
|
||||
.variant
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(str::to_string),
|
||||
state: "running".into(),
|
||||
log: vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)],
|
||||
log: if is_cmf {
|
||||
vec![format!(
|
||||
"→ downloading ready .cmf model {} ({})",
|
||||
p.repo, quant_display
|
||||
)]
|
||||
} else {
|
||||
vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)]
|
||||
},
|
||||
started: now(),
|
||||
finished: None,
|
||||
size_bytes: None,
|
||||
progress: Some(0.0),
|
||||
phase: Some("starting".into()),
|
||||
total_bytes: None,
|
||||
done_bytes: None,
|
||||
});
|
||||
let cancel = Arc::new(Notify::new());
|
||||
store.set_cancel(&id, cancel.clone());
|
||||
@@ -348,6 +1045,15 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
std::env::var(&cfg.hf_token_env).ok()
|
||||
};
|
||||
|
||||
if is_cmf && !use_python {
|
||||
let ret_id = id.clone();
|
||||
let variant = p.variant.clone();
|
||||
tokio::spawn(async move {
|
||||
download_cmf_repo(p.repo, output_abs, hf_token, store, id, cancel, variant).await;
|
||||
});
|
||||
return Ok(ret_id);
|
||||
}
|
||||
|
||||
// Program + args: `cortiq convert` by default, or the Python converter for
|
||||
// advanced options. Both stream `@PROGRESS <frac>` markers into the log.
|
||||
let (program, args, workdir): (String, Vec<String>, std::path::PathBuf) = if use_python {
|
||||
@@ -530,3 +1236,32 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
|
||||
|
||||
Ok(ret_id)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_map_quant_variants() {
|
||||
assert_eq!(map_quant("Q8_2F"), "q8_2f");
|
||||
assert_eq!(map_quant("q82f"), "q8_2f");
|
||||
assert_eq!(map_quant("Q8_ROW"), "q8");
|
||||
assert_eq!(map_quant("Q4_BLOCK"), "q4");
|
||||
assert_eq!(map_quant("Q4_TILED"), "q4t");
|
||||
assert_eq!(map_quant("q4t"), "q4t");
|
||||
assert_eq!(map_quant("vbit"), "vbit");
|
||||
assert_eq!(map_quant("q1"), "q1");
|
||||
assert_eq!(map_quant("Q1P"), "q1p");
|
||||
assert_eq!(map_quant("q1s"), "q1s");
|
||||
assert_eq!(map_quant("Q1T"), "q1t");
|
||||
assert_eq!(map_quant("F16"), "f16");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_cmf_repo_check() {
|
||||
assert!(is_cmf_repo("infosave/Bonsai-8B_2bit_cmf"));
|
||||
assert!(is_cmf_repo("infosave/Bonsai-1.7Bcmf"));
|
||||
assert!(is_cmf_repo("user/model.cmf"));
|
||||
assert!(!is_cmf_repo("Qwen/Qwen2.5-0.5B-Instruct"));
|
||||
}
|
||||
}
|
||||
|
||||
+197
-18
@@ -38,37 +38,171 @@ use state::SharedState;
|
||||
about = "Universal LLM gateway with intelligent routing"
|
||||
)]
|
||||
struct Args {
|
||||
/// Path to the TOML config file.
|
||||
#[arg(long, default_value = "config/gateway.toml")]
|
||||
config: String,
|
||||
/// Path to the TOML config file. If omitted, uses `config/gateway.toml` when running
|
||||
/// inside the repo, otherwise `~/.cortiq-gateway/gateway.toml` (or `$CORTIQ_GATEWAY_HOME`).
|
||||
/// Missing file is created automatically — no manual copying needed.
|
||||
#[arg(long)]
|
||||
config: Option<String>,
|
||||
/// Admin token for the web panel (overrides `[admin].token_env`).
|
||||
#[arg(long)]
|
||||
admin_token: Option<String>,
|
||||
/// Do not automatically open the browser on first run.
|
||||
#[arg(long)]
|
||||
no_browser: bool,
|
||||
/// Do not auto-select a free port if the configured one is busy.
|
||||
#[arg(long)]
|
||||
no_port_search: bool,
|
||||
}
|
||||
|
||||
/// Resolve the admin token: env from `[admin].token_env` → `--admin-token` →
|
||||
/// generate a random one. Returns `(token, generated)`.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
|
||||
///
|
||||
/// By default no token is required (open mode). User sets it explicitly via
|
||||
/// `--admin-token`, `[admin].token` or `[admin].token_env` if they want auth.
|
||||
fn resolve_admin_token(cfg: &Config, args: &Args) -> String {
|
||||
if let Some(env) = &cfg.admin.token_env {
|
||||
if let Ok(v) = std::env::var(env) {
|
||||
if !v.is_empty() {
|
||||
return (v, false);
|
||||
return v;
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(t) = &args.admin_token {
|
||||
if !t.is_empty() {
|
||||
return (t.clone(), false);
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
(admin::random_token(24), true)
|
||||
if let Some(t) = &cfg.admin.token {
|
||||
if !t.is_empty() {
|
||||
return t.clone();
|
||||
}
|
||||
}
|
||||
String::new()
|
||||
}
|
||||
|
||||
fn is_docker() -> bool {
|
||||
std::path::Path::new("/.dockerenv").exists()
|
||||
|| std::fs::read_to_string("/proc/1/cgroup")
|
||||
.map(|s| s.contains("docker") || s.contains("kubepods"))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn should_open_browser(args: &Args) -> bool {
|
||||
if args.no_browser {
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CORTIQ_NO_BROWSER")
|
||||
.map(|v| !v.is_empty() && v != "0")
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if std::env::var("CI").is_ok() || std::env::var("NO_BROWSER").is_ok() {
|
||||
return false;
|
||||
}
|
||||
if is_docker() {
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn parse_listen(listen: &str) -> (String, u16) {
|
||||
if let Some((host, port_s)) = listen.rsplit_once(':') {
|
||||
if let Ok(p) = port_s.parse::<u16>() {
|
||||
return (host.to_string(), p);
|
||||
}
|
||||
}
|
||||
(listen.to_string(), 9000)
|
||||
}
|
||||
|
||||
fn is_port_free(host: &str, port: u16) -> bool {
|
||||
let addr = format!("{host}:{port}");
|
||||
// bind test + connect test (docker port forwarding can make bind succeed while port is still forwarded)
|
||||
if std::net::TcpListener::bind(&addr).is_err() {
|
||||
return false;
|
||||
}
|
||||
use std::net::ToSocketAddrs;
|
||||
let probe_host = if host == "0.0.0.0" { "127.0.0.1" } else { host };
|
||||
if let Ok(mut addrs) = (probe_host, port).to_socket_addrs() {
|
||||
if let Some(a) = addrs.next() {
|
||||
if std::net::TcpStream::connect_timeout(&a, std::time::Duration::from_millis(200))
|
||||
.is_ok()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
fn resolve_listen(cfg: &mut Config, no_search: bool) -> String {
|
||||
if no_search {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
let (host, port) = parse_listen(&cfg.listen);
|
||||
let clean_host = if host.is_empty() {
|
||||
"0.0.0.0".to_string()
|
||||
} else {
|
||||
host
|
||||
};
|
||||
if is_port_free(&clean_host, port) {
|
||||
return cfg.listen.clone();
|
||||
}
|
||||
for p in (port + 1)..=port.saturating_add(50) {
|
||||
if is_port_free(&clean_host, p) {
|
||||
let new_listen = format!("{clean_host}:{p}");
|
||||
eprintln!("port {port} busy — switching to free port {p} ({new_listen})");
|
||||
cfg.listen = new_listen.clone();
|
||||
return new_listen;
|
||||
}
|
||||
}
|
||||
if !is_port_free(&clean_host, port) {
|
||||
eprintln!(
|
||||
"warning: port {port} busy and no free port found in next 50, will try to bind anyway"
|
||||
);
|
||||
}
|
||||
cfg.listen.clone()
|
||||
}
|
||||
|
||||
fn setup_marker_path(config_path: &str) -> std::path::PathBuf {
|
||||
// marker lives next to config file: gateway.toml -> .setup_done
|
||||
std::path::Path::new(config_path)
|
||||
.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.join(".setup_done")
|
||||
}
|
||||
|
||||
fn needs_first_run_wizard(cfg: &Config, config_path: &str) -> bool {
|
||||
let marker = setup_marker_path(config_path);
|
||||
if marker.exists() {
|
||||
return false;
|
||||
}
|
||||
// also consider "needs setup" if model pool is empty and no managed cmf
|
||||
if !cfg.models.is_empty() || !cfg.cmf.effective_servers().is_empty() {
|
||||
// still show wizard once if marker missing, but not if user already has models
|
||||
// we treat missing marker + existing models as "already configured" to avoid nagging
|
||||
// persist marker lazily
|
||||
let _ = std::fs::write(&marker, "done");
|
||||
return false;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let args = Args::parse();
|
||||
let config_path = args
|
||||
.config
|
||||
.clone()
|
||||
.unwrap_or_else(config::default_config_path);
|
||||
|
||||
let cfg = Config::load(&args.config)?;
|
||||
let mut cfg = Config::load_or_create(&config_path)?;
|
||||
// auto-pick a free port if the configured one is busy (unless disabled)
|
||||
let original_listen = cfg.listen.clone();
|
||||
let listen = resolve_listen(&mut cfg, args.no_port_search);
|
||||
// persist the resolved port if it changed
|
||||
if listen != original_listen {
|
||||
let _ = cfg.save(&config_path);
|
||||
}
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
@@ -76,11 +210,11 @@ async fn main() -> anyhow::Result<()> {
|
||||
)
|
||||
.init();
|
||||
|
||||
let listen = cfg.listen.clone();
|
||||
let admin_enabled = cfg.admin.enabled;
|
||||
let (admin_token, generated) = resolve_admin_token(&cfg, &args);
|
||||
let admin_token = resolve_admin_token(&cfg, &args);
|
||||
let first_run = needs_first_run_wizard(&cfg, &config_path);
|
||||
|
||||
let state = SharedState::build(cfg, args.config.clone())?;
|
||||
let state = SharedState::build(cfg, config_path.clone())?;
|
||||
|
||||
// Close the loop with the CMF format (github.com/infosave2007/cmf): install
|
||||
// / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
|
||||
@@ -94,6 +228,7 @@ async fn main() -> anyhow::Result<()> {
|
||||
|
||||
let mut app = Router::new()
|
||||
.route("/healthz", get(|| async { "ok" }))
|
||||
.route("/health", get(|| async { "ok" }))
|
||||
.route("/readyz", get(|| async { "ready" }))
|
||||
.route("/metrics", get(admin::metrics))
|
||||
.merge(protocols::build_router());
|
||||
@@ -104,16 +239,60 @@ async fn main() -> anyhow::Result<()> {
|
||||
.fallback(admin::assets::fallback);
|
||||
}
|
||||
|
||||
let app = app.with_state(state);
|
||||
let cors = tower_http::cors::CorsLayer::new()
|
||||
.allow_origin(tower_http::cors::Any)
|
||||
.allow_methods(tower_http::cors::Any)
|
||||
.allow_headers(tower_http::cors::Any);
|
||||
|
||||
let app = app.with_state(state).layer(cors);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(&listen).await?;
|
||||
tracing::info!("cortiq-gateway listening on {listen}");
|
||||
if admin_enabled {
|
||||
tracing::info!("admin console: http://{listen}/admin");
|
||||
if generated {
|
||||
tracing::warn!(
|
||||
"admin token (auto-generated, set [admin].token_env to persist): {admin_token}"
|
||||
);
|
||||
let host_for_url = if listen.starts_with("0.0.0.0:") {
|
||||
listen.replacen("0.0.0.0", "127.0.0.1", 1)
|
||||
} else {
|
||||
listen.clone()
|
||||
};
|
||||
// On first run open the onboarding wizard, otherwise the normal console.
|
||||
let admin_path = if first_run {
|
||||
"/admin#/onboarding"
|
||||
} else {
|
||||
"/admin"
|
||||
};
|
||||
let url = if admin_token.is_empty() {
|
||||
format!("http://{host_for_url}{admin_path}")
|
||||
} else {
|
||||
format!("http://{host_for_url}{admin_path}?token={admin_token}")
|
||||
};
|
||||
if admin_token.is_empty() {
|
||||
tracing::info!("admin console (open, no token): http://{host_for_url}/admin");
|
||||
} else {
|
||||
tracing::info!("admin console: http://{host_for_url}/admin");
|
||||
}
|
||||
if first_run {
|
||||
tracing::info!("first run — opening onboarding wizard at {url}");
|
||||
}
|
||||
if admin_enabled && should_open_browser(&args) {
|
||||
let browser_url = url.clone();
|
||||
tokio::spawn(async move {
|
||||
// give server a moment to be ready
|
||||
tokio::time::sleep(std::time::Duration::from_millis(600)).await;
|
||||
#[cfg(feature = "open-browser")]
|
||||
{
|
||||
if let Err(e) = open::that(&browser_url) {
|
||||
tracing::warn!("failed to open browser: {e}");
|
||||
} else {
|
||||
tracing::info!("opened browser at {browser_url}");
|
||||
}
|
||||
}
|
||||
#[cfg(not(feature = "open-browser"))]
|
||||
{
|
||||
tracing::info!("open browser: {browser_url} (feature disabled)");
|
||||
}
|
||||
});
|
||||
} else if first_run {
|
||||
tracing::info!("open this URL to finish setup: {url}");
|
||||
}
|
||||
}
|
||||
axum::serve(listener, app).await?;
|
||||
|
||||
@@ -29,6 +29,7 @@ pub struct GenParams {
|
||||
pub temperature: Option<f32>,
|
||||
pub max_tokens: Option<u32>,
|
||||
pub top_p: Option<f32>,
|
||||
pub think_budget: Option<u32>,
|
||||
pub stop: Vec<String>,
|
||||
/// Fields we do not interpret but must proxy through to the provider.
|
||||
pub passthrough: serde_json::Map<String, serde_json::Value>,
|
||||
|
||||
+117
-2
@@ -341,7 +341,64 @@ impl Pipeline {
|
||||
route_source
|
||||
);
|
||||
|
||||
match provider.chat(req.clone()).await {
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat(req_call).await {
|
||||
Ok(mut resp) => {
|
||||
let cost =
|
||||
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
|
||||
@@ -415,7 +472,65 @@ impl Pipeline {
|
||||
model_id,
|
||||
route_source
|
||||
);
|
||||
match provider.chat_stream(req.clone()).await {
|
||||
// Merge per-model/server config (same as run_inner).
|
||||
let mut req_call = req.clone();
|
||||
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = m_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = m_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = m_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = m_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &m_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
|
||||
if req_call.params.temperature.is_none() {
|
||||
req_call.params.temperature = s_cfg.temperature;
|
||||
}
|
||||
if req_call.params.top_p.is_none() {
|
||||
req_call.params.top_p = s_cfg.top_p;
|
||||
}
|
||||
if req_call.params.max_tokens.is_none() {
|
||||
req_call.params.max_tokens = s_cfg.max_tokens;
|
||||
}
|
||||
if req_call.params.think_budget.is_none() {
|
||||
req_call.params.think_budget = s_cfg.think_budget;
|
||||
}
|
||||
if let Some(sys) = &s_cfg.system_prompt {
|
||||
if !sys.trim().is_empty()
|
||||
&& !req_call.messages.iter().any(|m| m.role == "system")
|
||||
{
|
||||
req_call.messages.insert(
|
||||
0,
|
||||
Message {
|
||||
role: "system".into(),
|
||||
content: sys.clone(),
|
||||
tool_calls: Vec::new(),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
match provider.chat_stream(req_call).await {
|
||||
Ok(stream) => {
|
||||
let info = RouteInfo {
|
||||
task_label: decision.task_label.clone(),
|
||||
|
||||
@@ -101,6 +101,7 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
|
||||
@@ -49,6 +49,10 @@ struct OpenAiChatRequest {
|
||||
#[serde(default)]
|
||||
top_p: Option<f32>,
|
||||
#[serde(default)]
|
||||
think_budget: Option<u32>,
|
||||
#[serde(default)]
|
||||
reasoning_effort: Option<u32>,
|
||||
#[serde(default)]
|
||||
stream: bool,
|
||||
#[serde(default)]
|
||||
tools: Vec<serde_json::Value>,
|
||||
@@ -93,6 +97,7 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: req.think_budget.or(req.reasoning_effort),
|
||||
stop: Vec::new(),
|
||||
passthrough: req.rest,
|
||||
},
|
||||
|
||||
@@ -82,6 +82,7 @@ async fn handler(
|
||||
temperature: req.temperature,
|
||||
max_tokens: req.max_tokens,
|
||||
top_p: req.top_p,
|
||||
think_budget: None,
|
||||
stop: Vec::new(),
|
||||
passthrough: Default::default(),
|
||||
},
|
||||
|
||||
@@ -42,9 +42,20 @@ impl OpenAiProvider {
|
||||
if let Some(t) = req.params.temperature {
|
||||
body["temperature"] = t.into();
|
||||
}
|
||||
if let Some(p) = req.params.top_p {
|
||||
body["top_p"] = p.into();
|
||||
}
|
||||
if let Some(m) = req.params.max_tokens {
|
||||
body["max_tokens"] = m.into();
|
||||
}
|
||||
eprintln!("[to_wire] think_budget={:?}", req.params.think_budget);
|
||||
if let Some(tb) = req.params.think_budget {
|
||||
body["think_budget"] = tb.into();
|
||||
body["reasoning_effort"] = tb.into();
|
||||
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
|
||||
body["enable_thinking"] = (tb > 0).into();
|
||||
eprintln!("[to_wire] SET enable_thinking={}", tb > 0);
|
||||
}
|
||||
if !req.tools.is_empty() {
|
||||
body["tools"] = serde_json::Value::Array(req.tools.clone());
|
||||
}
|
||||
@@ -52,6 +63,10 @@ impl OpenAiProvider {
|
||||
for (k, v) in &req.params.passthrough {
|
||||
body[k] = v.clone();
|
||||
}
|
||||
eprintln!(
|
||||
"[to_wire] enable_thinking={:?}",
|
||||
body.get("enable_thinking")
|
||||
);
|
||||
body
|
||||
}
|
||||
}
|
||||
|
||||
@@ -49,6 +49,13 @@ impl Registry {
|
||||
kind: "chat".into(),
|
||||
api_key_env: None,
|
||||
caps: Vec::new(),
|
||||
temperature: s.temperature,
|
||||
top_p: s.top_p,
|
||||
max_tokens: s.max_tokens,
|
||||
think_budget: s.think_budget,
|
||||
system_prompt: s.system_prompt.clone(),
|
||||
o1: s.o1.clone(),
|
||||
skip_mtp: s.skip_mtp,
|
||||
};
|
||||
by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
|
||||
}
|
||||
|
||||
@@ -145,6 +145,14 @@ impl AppState {
|
||||
let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
|
||||
new_cfg.save(&self.config_path)?;
|
||||
self.live.store(Arc::new(live));
|
||||
|
||||
let cmf_rt = self.cmf.clone();
|
||||
let cmf_cfg = new_cfg.cmf.clone();
|
||||
tokio::spawn(async move {
|
||||
cmf_rt.stop().await;
|
||||
crate::cmf_runtime::manage(cmf_rt, cmf_cfg).await;
|
||||
});
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
Vendored
+197
@@ -397,3 +397,200 @@ tbody tr:hover td { background: var(--hover); }
|
||||
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
|
||||
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
|
||||
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
|
||||
|
||||
/* ---- responsive shell + touch layout ----------------------------------- */
|
||||
.menu-toggle, .nav-close, .mobile-nav-backdrop { display: none; }
|
||||
.hamburger { width: 18px; display: flex; flex-direction: column; gap: 4px; }
|
||||
.hamburger i { display: block; height: 2px; border-radius: 2px; background: currentColor; }
|
||||
body.modal-open { overflow: hidden; }
|
||||
|
||||
@media (max-width: 980px) {
|
||||
.app { display: block; min-height: 100dvh; }
|
||||
.main { min-height: 100dvh; }
|
||||
.menu-toggle { display: inline-flex; flex: 0 0 auto; }
|
||||
.nav-close { display: inline-flex; margin-left: auto; flex: 0 0 auto; font-size: 24px; line-height: 1; }
|
||||
.brand-copy { min-width: 0; }
|
||||
|
||||
.sidebar {
|
||||
display: flex;
|
||||
position: fixed;
|
||||
inset: 0 auto 0 0;
|
||||
width: min(84vw, 310px);
|
||||
height: 100dvh;
|
||||
padding: max(18px, env(safe-area-inset-top)) 14px max(14px, env(safe-area-inset-bottom));
|
||||
z-index: 70;
|
||||
box-shadow: 18px 0 45px rgba(0, 0, 0, .42);
|
||||
transform: translateX(-104%);
|
||||
visibility: hidden;
|
||||
transition: transform .22s ease, visibility 0s linear .22s;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.app.nav-open .sidebar {
|
||||
transform: translateX(0);
|
||||
visibility: visible;
|
||||
transition: transform .22s ease;
|
||||
}
|
||||
.mobile-nav-backdrop {
|
||||
display: block;
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 65;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border: 0;
|
||||
background: rgba(3, 7, 18, .62);
|
||||
opacity: 0;
|
||||
visibility: hidden;
|
||||
pointer-events: none;
|
||||
transition: opacity .22s ease, visibility 0s linear .22s;
|
||||
}
|
||||
.app.nav-open .mobile-nav-backdrop {
|
||||
opacity: 1;
|
||||
visibility: visible;
|
||||
pointer-events: auto;
|
||||
transition: opacity .22s ease;
|
||||
}
|
||||
body.mobile-nav-open { overflow: hidden; }
|
||||
|
||||
.topbar { padding: 10px 18px; min-height: 58px; gap: 10px; }
|
||||
.topbar .title { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.topbar .right { gap: 8px; min-width: 0; }
|
||||
.content { max-width: none; padding: 22px 20px 52px; }
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
body { font-size: 14px; }
|
||||
.status-pills { display: none; }
|
||||
.topbar { padding: 9px 12px; }
|
||||
.content { padding: 18px 14px 42px; }
|
||||
|
||||
.page-head { margin-bottom: 16px; }
|
||||
.page-head > .flex { align-items: stretch; flex-direction: column; }
|
||||
.page-head > .flex > .btn { width: 100%; }
|
||||
.page-head p { line-height: 1.45; }
|
||||
|
||||
.card { padding: 15px; margin-bottom: 14px; border-radius: 12px; }
|
||||
.card .card-head { align-items: flex-start; flex-wrap: wrap; }
|
||||
.card .card-head .right { width: 100%; margin-left: 0; flex-wrap: wrap; }
|
||||
.grid { gap: 14px; }
|
||||
.kpis { gap: 10px; margin-bottom: 14px; }
|
||||
.kpi { padding: 12px; border-radius: 11px; }
|
||||
.kpi .big { font-size: 21px; }
|
||||
|
||||
.row { display: grid; grid-template-columns: minmax(0, 1fr); gap: 0; }
|
||||
.row > * { min-width: 0; width: 100%; }
|
||||
input, select, textarea { min-height: 44px; font-size: 16px; }
|
||||
input[type="checkbox"], input[type="radio"] { min-height: 0; }
|
||||
.check { min-height: 36px; margin-bottom: 8px; }
|
||||
.btn, .icon-btn { min-height: 44px; }
|
||||
.btn { padding: 10px 14px; }
|
||||
.btn.sm, .btn.small { min-height: 38px; padding: 8px 11px; }
|
||||
|
||||
.table-wrap {
|
||||
width: 100%;
|
||||
overflow-x: auto;
|
||||
overscroll-behavior-x: contain;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: thin;
|
||||
}
|
||||
.table-wrap table { min-width: max-content; }
|
||||
th, td { padding: 10px; }
|
||||
|
||||
.tierrow { align-items: flex-start; flex-direction: column; gap: 8px; }
|
||||
.tierrow .tname { width: auto; }
|
||||
.chips { width: 100%; }
|
||||
.togglerow { align-items: flex-start; }
|
||||
|
||||
.pg { gap: 14px; }
|
||||
.routecard .kv { flex-wrap: wrap; }
|
||||
.routecard .kv .v { max-width: 100%; overflow-wrap: anywhere; }
|
||||
.answer { padding: 12px; }
|
||||
.seg { max-width: 100%; overflow-x: auto; }
|
||||
.seg button { min-height: 40px; white-space: nowrap; }
|
||||
|
||||
.hf-grid { grid-template-columns: 1fr; max-height: 56dvh; }
|
||||
.job-head, .actions { align-items: stretch; flex-wrap: wrap; }
|
||||
.actions .btn { flex: 1 1 150px; }
|
||||
.key-row { flex-direction: column; }
|
||||
.key-row .btn { width: 100%; }
|
||||
.cmf-server-row { align-items: stretch !important; padding: 10px; border: 1px solid var(--border-soft); border-radius: 10px; }
|
||||
.cmf-server-row > input, .cmf-server-row > select { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
|
||||
.cmf-server-row > .btn { flex: 1 1 calc(50% - 6px); }
|
||||
.cmf-server-row > .small { flex: 1 1 100%; }
|
||||
|
||||
.modal-backdrop { align-items: flex-end; padding: 8px 0 0; }
|
||||
.modal {
|
||||
max-width: none;
|
||||
max-height: calc(100dvh - 8px);
|
||||
overflow-y: auto;
|
||||
padding: 18px 15px max(14px, env(safe-area-inset-bottom));
|
||||
border-radius: 18px 18px 0 0;
|
||||
}
|
||||
.modal .modal-foot {
|
||||
position: sticky;
|
||||
bottom: calc(-1 * max(14px, env(safe-area-inset-bottom)));
|
||||
margin: 12px -15px 0;
|
||||
padding: 12px 15px max(14px, env(safe-area-inset-bottom));
|
||||
background: color-mix(in srgb, var(--card) 94%, transparent);
|
||||
backdrop-filter: blur(10px);
|
||||
border-top: 1px solid var(--border-soft);
|
||||
}
|
||||
.modal .modal-foot .btn { flex: 1; }
|
||||
|
||||
.toasts { left: 12px; right: 12px; bottom: max(12px, env(safe-area-inset-bottom)); }
|
||||
.toast { min-width: 0; width: 100%; }
|
||||
.login-wrap { align-items: flex-start; padding: max(20px, env(safe-area-inset-top)) 14px; }
|
||||
.login-card { padding: 22px 18px; margin-top: 8dvh; }
|
||||
}
|
||||
|
||||
@media (max-width: 480px) {
|
||||
.topbar .title { font-size: 14px; }
|
||||
.topbar .right { gap: 6px; }
|
||||
.lang-select { width: 54px; min-height: 44px; padding: 6px; }
|
||||
.icon-btn { width: 42px; height: 42px; }
|
||||
.kpis { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.kpi .lbl { line-height: 1.25; }
|
||||
.barlist .b { grid-template-columns: minmax(0, 1fr) auto; gap: 6px 10px; }
|
||||
.barlist .b .track { grid-column: 1 / -1; grid-row: 2; }
|
||||
.barlist .b .val { min-width: 0; }
|
||||
.empty { padding: 28px 12px; }
|
||||
}
|
||||
|
||||
@media (max-width: 360px) {
|
||||
.topbar .title { display: none; }
|
||||
.kpis { grid-template-columns: 1fr; }
|
||||
}
|
||||
|
||||
/* ---- onboarding ------------------------------------------------------------ */
|
||||
.onb-shell { min-height: 100vh; display: flex; align-items: flex-start; justify-content: center; padding: 32px 20px; background: var(--bg-grad); }
|
||||
.onb-content { width: 100%; max-width: 860px; }
|
||||
.onb-wrap { max-width: 860px; margin: 0 auto; }
|
||||
.onb-header { display: flex; align-items: center; justify-content: space-between; margin-bottom: 18px; }
|
||||
.onb-dots { display: flex; gap: 8px; }
|
||||
.onb-dots .dot { width: 28px; height: 6px; border-radius: 999px; background: var(--border); }
|
||||
.onb-dots .dot.on { background: var(--accent); }
|
||||
.onb-dots .dot.done { background: color-mix(in srgb, var(--accent) 55%, transparent); }
|
||||
.onb-title { font-size: 12px; font-weight: 700; color: var(--muted); letter-spacing: 0.04em; text-transform: uppercase; }
|
||||
.onb-step h2 { font-size: 22px; margin-bottom: 6px; }
|
||||
.onb-langs { display: grid; grid-template-columns: repeat(auto-fill, minmax(132px, 1fr)); gap: 12px; margin: 18px 0; }
|
||||
.onb-lang {
|
||||
display: flex; flex-direction: column; align-items: center; gap: 10px;
|
||||
padding: 18px 10px; border-radius: 14px; border: 1.5px solid var(--border);
|
||||
background: var(--card); cursor: pointer; transition: .15s; font-family: inherit;
|
||||
}
|
||||
.onb-lang:hover { border-color: color-mix(in srgb, var(--accent) 45%, transparent); transform: translateY(-1px); }
|
||||
.onb-lang.sel { border-color: var(--accent); background: color-mix(in srgb, var(--accent) 10%, var(--card)); box-shadow: 0 6px 18px rgba(99,102,241,.18); }
|
||||
.onb-lang .flag { font-size: 32px; line-height: 1; }
|
||||
.onb-lang b { font-size: 14px; font-weight: 700; color: var(--text); }
|
||||
/* Featured gallery: the list grows as new models are published, so give it more
|
||||
room than the search grid before it starts scrolling. */
|
||||
.featured-section .hf-grid { max-height: min(62vh, 760px); }
|
||||
/* Live download progress inside a featured card (empty until a job runs) */
|
||||
.card-prog:empty { display: none; }
|
||||
.card-prog { margin-top: 6px; }
|
||||
.card-prog .pbar { margin: 4px 0; }
|
||||
.card-prog .btn.sm { padding: 0 6px; line-height: 18px; }
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sidebar, .mobile-nav-backdrop { transition: none; }
|
||||
}
|
||||
|
||||
Vendored
+1
-1
@@ -19,6 +19,6 @@
|
||||
<noscript>This console requires JavaScript.</noscript>
|
||||
<div id="app"><div class="boot">Loading…</div></div>
|
||||
<div id="toasts" class="toasts"></div>
|
||||
<script type="module" src="/admin/js/app.js"></script>
|
||||
<script type="module" src="/admin/js/app.js?v=3"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Vendored
+23
-2
@@ -19,7 +19,10 @@ export function setToken(tok) {
|
||||
}
|
||||
|
||||
async function j(method, path, body) {
|
||||
const opt = { method, headers: { Authorization: "Bearer " + getToken() } };
|
||||
const tok = getToken();
|
||||
const headers = {};
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const opt = { method, headers };
|
||||
if (body !== undefined) {
|
||||
opt.headers["content-type"] = "application/json";
|
||||
opt.body = JSON.stringify(body);
|
||||
@@ -114,14 +117,32 @@ export const api = {
|
||||
deleteImport: (job) => j("DELETE", "/import/" + encodeURIComponent(job)),
|
||||
registerImport: (job) =>
|
||||
j("POST", "/import/" + encodeURIComponent(job) + "/register"),
|
||||
|
||||
// first-run onboarding
|
||||
setupStatus: () => j("GET", "/setup/status"),
|
||||
// Connects every finished download as a local model unless register === false.
|
||||
setupComplete: (lang, register) => j("POST", "/setup/complete", { lang, register }),
|
||||
setupFeatured: () => j("GET", "/setup/featured"),
|
||||
// `name` is the exact output stem from setup/featured — it wins over the
|
||||
// repo/variant guess so the delete always hits the file the download wrote.
|
||||
deleteSetupFile: (repo, variant, name) =>
|
||||
j(
|
||||
"DELETE",
|
||||
`/setup/file?repo=${encodeURIComponent(repo)}` +
|
||||
(variant ? `&variant=${encodeURIComponent(variant)}` : "") +
|
||||
(name ? `&name=${encodeURIComponent(name)}` : "")
|
||||
),
|
||||
};
|
||||
|
||||
// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
|
||||
// returns the routing decision read from X-Cortiq-* response headers.
|
||||
export async function testStream(body, onDelta) {
|
||||
const tok = getToken();
|
||||
const headers = { "content-type": "application/json" };
|
||||
if (tok) headers["Authorization"] = "Bearer " + tok;
|
||||
const r = await fetch(BASE + "/test/stream", {
|
||||
method: "POST",
|
||||
headers: { Authorization: "Bearer " + getToken(), "content-type": "application/json" },
|
||||
headers,
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (r.status === 401) {
|
||||
|
||||
Vendored
+71
-11
@@ -4,13 +4,14 @@ import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
|
||||
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
|
||||
import { renderLogin } from "./views/login.js";
|
||||
import { renderDashboard } from "./views/dashboard.js";
|
||||
import { renderModels } from "./views/models.js";
|
||||
import { renderModels } from "./views/models.js?v=3";
|
||||
import { renderImport } from "./views/import.js";
|
||||
import { renderRouting } from "./views/routing.js";
|
||||
import { renderProtocols } from "./views/protocols.js";
|
||||
import { renderKeys } from "./views/keys.js";
|
||||
import { renderPlayground } from "./views/playground.js";
|
||||
import { renderSettings } from "./views/settings.js";
|
||||
import { renderOnboarding } from "./views/onboarding.js";
|
||||
|
||||
const ROUTES = [
|
||||
{ id: "dashboard", icon: "▤", view: renderDashboard },
|
||||
@@ -21,6 +22,7 @@ const ROUTES = [
|
||||
{ id: "keys", icon: "🔑", view: renderKeys },
|
||||
{ id: "playground", icon: "✦", view: renderPlayground },
|
||||
{ id: "settings", icon: "⚙", view: renderSettings },
|
||||
{ id: "onboarding", icon: "✦", view: renderOnboarding, hidden: true },
|
||||
];
|
||||
|
||||
export const appState = { meta: null, health: null, version: "" };
|
||||
@@ -28,6 +30,17 @@ export const appState = { meta: null, health: null, version: "" };
|
||||
// allaigate site — where router keys are issued and topped up
|
||||
export const SITE_URL = "https://api.allaigate.com/";
|
||||
|
||||
let mobileNavOpen = false;
|
||||
|
||||
function setMobileNav(open) {
|
||||
mobileNavOpen = !!open;
|
||||
const app = document.querySelector(".app");
|
||||
if (app) app.classList.toggle("nav-open", mobileNavOpen);
|
||||
document.body.classList.toggle("mobile-nav-open", mobileNavOpen);
|
||||
const toggle = document.querySelector(".menu-toggle");
|
||||
if (toggle) toggle.setAttribute("aria-expanded", String(mobileNavOpen));
|
||||
}
|
||||
|
||||
function currentRoute() {
|
||||
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
|
||||
return ROUTES.find((r) => r.id === id) || ROUTES[0];
|
||||
@@ -40,19 +53,21 @@ const FAVICON =
|
||||
);
|
||||
|
||||
function sidebar(activeId) {
|
||||
const visibleRoutes = ROUTES.filter((r) => !r.hidden);
|
||||
return h(
|
||||
"aside",
|
||||
{ class: "sidebar", id: "sidebar" },
|
||||
{ class: "sidebar", id: "sidebar", "aria-label": "Main navigation" },
|
||||
h(
|
||||
"div",
|
||||
{ class: "brand" },
|
||||
h("img", { src: FAVICON, alt: "" }),
|
||||
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
|
||||
h("div", { class: "brand-copy" }, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub"))),
|
||||
h("button", { class: "icon-btn nav-close", type: "button", title: "Close menu", "aria-label": "Close menu", onClick: () => setMobileNav(false) }, "×")
|
||||
),
|
||||
...ROUTES.map((r) =>
|
||||
...visibleRoutes.map((r) =>
|
||||
h(
|
||||
"a",
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
|
||||
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id, onClick: () => setMobileNav(false) },
|
||||
h("span", { class: "ico" }, r.icon),
|
||||
t("nav." + r.id)
|
||||
)
|
||||
@@ -85,7 +100,7 @@ function statusPills() {
|
||||
: t("status.checking");
|
||||
return h(
|
||||
"div",
|
||||
{ class: "flex" },
|
||||
{ class: "flex status-pills" },
|
||||
h(
|
||||
"span",
|
||||
{ class: "pill" },
|
||||
@@ -109,7 +124,7 @@ function langSelect() {
|
||||
title: "Language",
|
||||
onChange: (e) => setLang(e.target.value),
|
||||
},
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, l.label))
|
||||
...LANGS.map((l) => h("option", { value: l.code, selected: l.code === getLang() }, `${l.flag} ${l.label}`))
|
||||
);
|
||||
return sel;
|
||||
}
|
||||
@@ -136,6 +151,19 @@ function topbar(activeId) {
|
||||
return h(
|
||||
"header",
|
||||
{ class: "topbar" },
|
||||
h(
|
||||
"button",
|
||||
{
|
||||
class: "icon-btn menu-toggle",
|
||||
type: "button",
|
||||
title: "Menu",
|
||||
"aria-label": "Open menu",
|
||||
"aria-controls": "sidebar",
|
||||
"aria-expanded": String(mobileNavOpen),
|
||||
onClick: () => setMobileNav(!mobileNavOpen),
|
||||
},
|
||||
h("span", { class: "hamburger", "aria-hidden": "true" }, h("i"), h("i"), h("i"))
|
||||
),
|
||||
h("div", { class: "title" }, t("nav." + activeId)),
|
||||
h(
|
||||
"div",
|
||||
@@ -175,12 +203,24 @@ async function renderView() {
|
||||
|
||||
function renderShell() {
|
||||
const route = currentRoute();
|
||||
mobileNavOpen = false;
|
||||
document.body.classList.remove("mobile-nav-open");
|
||||
// onboarding — full screen without sidebar/topbar
|
||||
if (route.id === "onboarding") {
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h("div", { class: "onb-shell" }, h("div", { class: "onb-content", id: "view" }))
|
||||
);
|
||||
renderView();
|
||||
return;
|
||||
}
|
||||
mount(
|
||||
document.getElementById("app"),
|
||||
h(
|
||||
"div",
|
||||
{ class: "app" },
|
||||
sidebar(route.id),
|
||||
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
|
||||
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
|
||||
)
|
||||
);
|
||||
@@ -213,20 +253,34 @@ export async function boot() {
|
||||
setToken(urlTok);
|
||||
history.replaceState({}, "", location.pathname + location.hash);
|
||||
}
|
||||
if (!getToken()) {
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// try without token first — open mode (no CORTIQ_ADMIN_TOKEN) succeeds,
|
||||
// protected mode returns 401 and we show login
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch (e) {
|
||||
if (String(e.message) === "unauthorized") {
|
||||
// if we have no token stored, show login; with a stale token also show login
|
||||
showLogin();
|
||||
return;
|
||||
}
|
||||
// gateway reachable but error — still show shell
|
||||
// if meta failed for other reason but token was present, fall through
|
||||
if (!getToken()) {
|
||||
// open mode but gateway unreachable — still try shell; health will show
|
||||
try {
|
||||
appState.meta = await api.meta();
|
||||
appState.version = appState.meta.version || "";
|
||||
} catch {}
|
||||
}
|
||||
}
|
||||
// first-run check — redirect to onboarding if needed (unless already there)
|
||||
try {
|
||||
const st = await api.setupStatus();
|
||||
if (st.needs_setup && location.hash !== "#/onboarding") {
|
||||
location.hash = "#/onboarding";
|
||||
}
|
||||
} catch {}
|
||||
if (!location.hash) location.hash = "#/dashboard";
|
||||
renderShell();
|
||||
refreshHealth();
|
||||
@@ -243,6 +297,12 @@ onLangChange(() => {
|
||||
window.addEventListener("hashchange", () => {
|
||||
if (document.querySelector(".app")) renderShell();
|
||||
});
|
||||
window.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Escape" && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
window.addEventListener("resize", () => {
|
||||
if (window.innerWidth > 980 && mobileNavOpen) setMobileNav(false);
|
||||
});
|
||||
setInterval(() => {
|
||||
if (document.querySelector(".app")) refreshHealth();
|
||||
}, 20000);
|
||||
|
||||
Vendored
+7
-7
@@ -10,13 +10,13 @@ import tr from "./i18n/tr.js";
|
||||
|
||||
const DICTS = { en, ru, de, fr, es, zh, tr };
|
||||
export const LANGS = [
|
||||
{ code: "en", label: "English" },
|
||||
{ code: "ru", label: "Русский" },
|
||||
{ code: "de", label: "Deutsch" },
|
||||
{ code: "fr", label: "Français" },
|
||||
{ code: "es", label: "Español" },
|
||||
{ code: "zh", label: "中文" },
|
||||
{ code: "tr", label: "Türkçe" },
|
||||
{ code: "en", label: "English", flag: "🇬🇧" },
|
||||
{ code: "ru", label: "Русский", flag: "🇷🇺" },
|
||||
{ code: "de", label: "Deutsch", flag: "🇩🇪" },
|
||||
{ code: "fr", label: "Français", flag: "🇫🇷" },
|
||||
{ code: "es", label: "Español", flag: "🇪🇸" },
|
||||
{ code: "zh", label: "中文", flag: "🇨🇳" },
|
||||
{ code: "tr", label: "Türkçe", flag: "🇹🇷" },
|
||||
];
|
||||
|
||||
const listeners = new Set();
|
||||
|
||||
Vendored
+191
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Routing",
|
||||
"nav.protocols": "Protokolle",
|
||||
"nav.keys": "API-Schlüssel",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Testbereich",
|
||||
"nav.settings": "Einstellungen",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Router-Key & Abrechnung",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "nach Aufgabe",
|
||||
"dash.groupby.account": "nach Account",
|
||||
"dash.recent.title": "Letzte Anfragen",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Playground ausprobieren.",
|
||||
"dash.recent.empty": "Noch keine Anfragen aufgezeichnet. Testbereich ausprobieren.",
|
||||
"dash.col.time": "Zeit",
|
||||
"dash.col.model": "Modell",
|
||||
"dash.col.tier": "Tier",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "alle",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Testbereich",
|
||||
"pg.subtitle": "Prompt durch die Live-Pipeline senden und Routing-Entscheidung prüfen.",
|
||||
"pg.system": "System-Prompt",
|
||||
"pg.systemPlaceholder": "optionale Systemnachricht",
|
||||
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "Eintrags-TTL",
|
||||
"settings.cache.embed": "Embedding-Modell",
|
||||
"settings.cache.note": "Cache-Einstellungen werden nach einem Neustart wirksam.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Modell importieren",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Logs leeren",
|
||||
"dash.clearConfirm": "Alle erfassten Statistiken und Logs löschen? Das lässt sich nicht rückgängig machen.",
|
||||
"dash.cleared": "Logs geleert",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Gib eine ID oder einen Modellnamen ein",
|
||||
"models.form.apiKey": "API-Schlüssel",
|
||||
"models.form.apiKeyPlaceholder": "API-Schlüssel einfügen",
|
||||
"models.form.apiKeyNotNeeded": "nicht erforderlich (lokaler Server)",
|
||||
"models.form.apiKeyHint": "Wird sicher im Gateway gespeichert und nicht erneut angezeigt.",
|
||||
"models.form.keyStored": "Ein Schlüssel ist gespeichert — leer lassen, um ihn zu behalten, oder einen neuen einfügen.",
|
||||
"models.form.keyFromEnv": "Es wird ein Schlüssel aus einer Umgebungsvariable verwendet.",
|
||||
"models.form.paramsTitle": "Generierungs- und Denkparameter (O(1), MTP, Temperatur)",
|
||||
"models.form.temperature": "Temperatur",
|
||||
"models.form.temperatureHint": "Generierungstemperatur (0.0 = deterministisch, 1.0+ = kreativ).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling-Schwelle (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max. Tokens",
|
||||
"models.form.maxTokensHint": "Maximale Anzahl generierter Tokens.",
|
||||
"models.form.thinkBudget": "Denkmodus / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Token-Budget fürs Nachdenken (0 = aus, N > 0 = max. Reasoning-Tokens).",
|
||||
"models.form.systemPrompt": "Standard-System-Prompt",
|
||||
"models.form.systemPromptHint": "Wird automatisch vorangestellt, wenn die Anfrage keinen System-Prompt enthält.",
|
||||
"models.form.systemPromptPlaceholder": "Du bist ein hilfreicher Assistent…",
|
||||
"models.form.testKey": "Testen & Modelle laden",
|
||||
"models.form.testing": "wird geprüft…",
|
||||
"models.form.foundModels": "{n} Modelle geladen",
|
||||
"models.form.modelPlaceholder": "tippen zum Suchen…",
|
||||
"models.form.modelHint": "Klicke „Testen & Modelle laden“, um die Liste mit deinem Schlüssel abzurufen, und wähle dann eines aus.",
|
||||
"models.managed": "verwaltet",
|
||||
"models.managedHint": "Wird vom verwalteten CMF-Server bereitgestellt (Einstellungen → Lokale Modelle). Dort konfigurieren.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Router verwenden (aus = nur lokale Modelle)",
|
||||
"settings.cmf": "Lokale Modelle (CMF)",
|
||||
"settings.cmf.localOnly": "Nur lokale Modelle verwenden",
|
||||
"settings.cmf.manage": "Lokalen Modellserver starten",
|
||||
"settings.cmf.autoInstall": "cortiq automatisch installieren (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq automatisch aktualisieren",
|
||||
"settings.cmf.model": "Pfad zum lokalen .cmf-Modell",
|
||||
"settings.cmf.port": "Port des lokalen Servers",
|
||||
"settings.cmf.modelId": "Modell-ID im Pool",
|
||||
"settings.cmf.note": "Stellt ein lokales .cmf über cortiq bereit (github.com/infosave2007/cmf). Funktioniert auch, wenn der Router nicht erreichbar ist.",
|
||||
"settings.cmf.threads": "Worker-Threads",
|
||||
"settings.cmf.gpu": "GPU verwenden (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Threads parallelisieren die Decode-Matvecs (~3× schneller; 0 = Standard der Runtime). GPU-Offload hilft vor allem beim Prefill langer Prompts — das Decoding ist speicherbandbreitenbegrenzt und daher auf Apple Silicon ungefähr neutral.",
|
||||
"settings.cmf.pick": "Modell auswählen",
|
||||
"settings.cmf.models": "Lokale Modelle",
|
||||
"settings.cmf.modelsHelp": "Jede Zeile ist ein Modell, das das Gateway ausführt (eigenes cortiq serve auf eigenem Port) und im Pool registriert. Füge so viele hinzu, wie du .cmf-Dateien und freie Ports hast.",
|
||||
"settings.cmf.rowHint": "ID · .cmf-Datei · Port · Threads · GPU. Änderungen greifen nach einem Neustart; das Löschen einer Zeile stoppt diesen Server.",
|
||||
"settings.cmf.addServer": "+ Lokales Modell hinzufügen",
|
||||
"settings.cmf.checkPort": "Port prüfen",
|
||||
"settings.cmf.portFree": "Port ist frei",
|
||||
"settings.cmf.portBusy": "Port ist belegt",
|
||||
"settings.cmf.useSuggested": "{port} verwenden",
|
||||
"settings.cmf.serving": "{model} läuft auf :{port}",
|
||||
"settings.cmf.intro": "Modelle lokal mit der cortiq-Engine ausführen — funktioniert auch offline oder wenn der Router ausfällt.",
|
||||
"settings.cmf.runtime": "cortiq-Runtime",
|
||||
"settings.cmf.runtimeHelp": "Das Kommandozeilenwerkzeug `cortiq`, das .cmf-Modelle lädt und ausführt. Einmal installieren, bevor du einen lokalen Server startest.",
|
||||
"settings.cmf.install": "cortiq installieren",
|
||||
"settings.cmf.reinstall": "Neu installieren / aktualisieren",
|
||||
"settings.cmf.installing": "wird installiert… (kompiliert, einige Minuten)",
|
||||
"settings.cmf.installedOk": "cortiq installiert",
|
||||
"settings.cmf.notInstalled": "nicht installiert",
|
||||
"settings.cmf.addModel": "Modell hinzufügen",
|
||||
"settings.cmf.addModelHelp": "Modelle werden im Tab „Modell importieren“ heruntergeladen und konvertiert (HuggingFace → .cmf). Richte den lokalen Server danach unten auf die .cmf-Datei aus.",
|
||||
"settings.cmf.importNote": "Noch kein Modell? Importiere eines:",
|
||||
"settings.cmf.importLink": "Import öffnen",
|
||||
"settings.cmf.modelHelp": "Pfad zur .cmf-Datei, die bereitgestellt werden soll (z. B. models/qwen.cmf), erzeugt im Import-Tab. Mehrere Modelle für verschiedene Aufgaben? Importiere sie und wechsle diesen Pfad.",
|
||||
"settings.cmf.portHelp": "TCP-Port, auf dem der lokale Modellserver lauscht (Standard 8081). Nur ändern, wenn er mit einem anderen Dienst kollidiert.",
|
||||
"settings.cmf.threadsHelp": "Wie viele CPU-Kerne für die Generierung genutzt werden. Mehr = schneller (~3× gegenüber einem Kern); 0 überlässt die Wahl der Runtime.",
|
||||
"settings.cmf.gpuHelp": "Matrixberechnungen auf die GPU auslagern (Metal/wgpu). Auf Apple Silicon beschleunigt das vor allem lange Prompts — das Decoding ist speicherbandbreitenbegrenzt.",
|
||||
"settings.cmf.localOnlyHelp": "Jede Anfrage an das lokale Modell leiten und dabei Router sowie Cloud-Anbieter ignorieren.",
|
||||
"settings.cmf.manageHelp": "Das Gateway startet beim Hochfahren `cortiq serve` für die untenstehende .cmf-Datei und routet dorthin. Erfordert ein installiertes cortiq (Schritt 1). Greift nach einem Neustart.",
|
||||
"settings.cmf.autoInstallHelp": "Wenn cortiq beim Start fehlt, automatisch von crates.io installieren.",
|
||||
"settings.cmf.autoUpdateHelp": "Beim Start cortiq auf die neueste Version aktualisieren, falls eine neuere existiert.",
|
||||
"settings.router.enabledHelp": "Den cortiq-Router jede Anfrage klassifizieren lassen (Aufgabentyp + Komplexität) und ein Modell wählen. Aus = nur die lokale Routing-Tabelle verwenden.",
|
||||
"settings.router.verifyTlsHelp": "Das TLS-Zertifikat des Routers prüfen. Nur bei einem selbstsignierten oder IP-gebundenen Zertifikat abschalten.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus-Metriken unter GET /metrics bereitstellen.",
|
||||
"settings.cortiq.echoHelp": "Ein Feld `cortiq` (Aufgabe, Modell, Kosten) zu jedem Antwort-Body hinzufügen. Response-Header werden immer gesendet.",
|
||||
"settings.stats.enabledHelp": "Aktuelle Anfragen aufzeichnen, um Dashboard und Diagramme zu speisen.",
|
||||
"settings.cache.enabledHelp": "Semantischer Cache: eine gespeicherte Antwort zurückgeben, wenn ein neuer Prompt einem früheren sehr ähnlich ist, und den Modellaufruf überspringen. Erfordert ein Embedding-Modell im Pool.",
|
||||
"settings.log.bodiesHelp": "Vollständige Nachrichteninhalte von Anfrage und Antwort loggen. Aus Datenschutzgründen standardmäßig aus.",
|
||||
"settings.router.saveKey": "Schlüssel speichern",
|
||||
"settings.router.keySaved": "Router-Schlüssel gespeichert",
|
||||
"settings.router.keyEmpty": "Füge zuerst den Schlüssel ein",
|
||||
"settings.router.secretHelp": "Füge den Schlüssel aus deiner E-Mail von api.allaigate.com ein und klicke auf „Schlüssel speichern“. Das ist KEIN Wert aus „API-Schlüssel“ — die sind für Clients, die dein Gateway aufrufen.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — Schlüssel gespeichert (zum Ersetzen erneut einfügen)",
|
||||
"settings.route.strategyHelp": "Welcher Text zur Klassifizierung an den Router geht: last_user = nur die letzte Nutzernachricht; last_user_plus_system = letzte Nutzernachricht + System-Prompt; concat_all = die gesamte Unterhaltung.",
|
||||
"settings.route.maxCharsHelp": "Maximale Zeichenzahl dieses Textes, die an den Router gesendet wird (längerer Text wird gekürzt). Mehr = genauere Klassifizierung, aber langsamere und teurere Router-Aufrufe.",
|
||||
"settings.route.cacheTtlHelp": "Wie lange eine Routing-Entscheidung für eine identische Anfrage zwischengespeichert wird, bevor der Router erneut gefragt wird (z. B. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Standard-Routing-Profil, wenn eine Anfrage keines angibt: cost-saver (am günstigsten), balanced oder quality-first (bestes Modell). Beeinflusst, welches Modell der Router wählt.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Suche auf HuggingFace, konvertiere in ein lokales .cmf und stelle es bereit.",
|
||||
"import.searchPlaceholder": "HuggingFace-Modelle suchen (z. B. qwen3, llama)…",
|
||||
"import.searching": "Suche läuft…",
|
||||
"import.noResults": "Keine Modelle gefunden.",
|
||||
"import.configure": "Konvertierung einrichten",
|
||||
"import.outName": "Ausgabename",
|
||||
"import.outNameHint": "Name der .cmf-Datei (Buchstaben, Ziffern, - und _).",
|
||||
"import.quant": "Quantisierung",
|
||||
"import.convert": "Konvertieren & laden",
|
||||
"import.converting": "Konvertierung",
|
||||
"import.started": "Konvertierung gestartet",
|
||||
"import.failed": "Konvertierung fehlgeschlagen — siehe Log.",
|
||||
"import.register": "Als lokales Modell registrieren",
|
||||
"import.registered": "Als '{id}' registriert. Starte cortiq-server, um es bereitzustellen.",
|
||||
"import.jobs": "Downloads & Konvertierungen",
|
||||
"import.cancel": "Abbrechen",
|
||||
"import.cancelling": "Wird abgebrochen — unvollständige Ausgabe wird entfernt…",
|
||||
"import.delete": "Löschen",
|
||||
"import.deleteConfirm": "Dieses Modell und seine konvertierte .cmf-Datei von der Festplatte löschen?",
|
||||
"import.deleted": "Mitsamt Dateien gelöscht",
|
||||
"import.state.running": "läuft",
|
||||
"import.state.done": "fertig",
|
||||
"import.state.error": "Fehler",
|
||||
"import.state.cancelled": "abgebrochen",
|
||||
"import.featuredTitle": "Empfohlene fertige .cmf-Modelle",
|
||||
"import.featuredSub": "Vorkonvertierte .cmf-Modelle vom infosave-Konto auf HuggingFace — die Liste wird live gelesen, neue Modelle erscheinen von selbst.",
|
||||
"import.readyCmfBadge": "Fertiges .cmf",
|
||||
"import.directDownload": "Herunterladen",
|
||||
"import.q.q8_2f": "8-Bit Zwei-Feld (𝒲×θ) — empfohlen: bestes Verhältnis von Qualität und Größe.",
|
||||
"import.q.q8_row": "8-Bit pro Zeile — einfach und robust.",
|
||||
"import.q.q4": "4-Bit Block — am kleinsten, geringere Qualität.",
|
||||
"import.q.q4_tiled": "4-Bit gekachelt (Q4_TILED) — 1,66× schneller auf der Metal-GPU.",
|
||||
"import.q.vbit": "Variabel 3–8 Bit — Bit-Budget pro Experte.",
|
||||
"import.q.q1t": "1,58-Bit ternär (Q1T) — BitNet b1.58 mit spärlichen Ausreißern.",
|
||||
"import.q.q1s": "1-Bit spärlich (Q1S) — 1-Bit-Basis mit holografischen Ausreißern.",
|
||||
"import.q.q1p": "1-Bit PTQ (Q1P) — 1 Bit mit Fehlerdiffusion.",
|
||||
"import.q.q1": "1-Bit (Q1) — rein binäres Format.",
|
||||
"import.q.f16": "16-Bit — keine Quantisierung.",
|
||||
"import.q.f32": "32-Bit — volle Präzision (große Datei).",
|
||||
"import.adv.title": "Erweiterte Einstellungen",
|
||||
"import.adv.auto": "automatisch",
|
||||
"import.adv.linearCore": "Linearer Kern",
|
||||
"import.adv.linearHint": "Wie Layer mit linearer Attention übernommen bzw. gefaltet werden.",
|
||||
"import.adv.nphase": "vmf_phase Phasen/Kopf",
|
||||
"import.adv.vbitShape": "VBIT-Budgetform",
|
||||
"import.adv.meanBits": "VBIT mittlere Bits",
|
||||
"import.adv.shard": "Shard max. (GB)",
|
||||
"import.adv.noShard": "kein Sharding",
|
||||
"import.adv.skipMtp": "MTP-Kopf überspringen",
|
||||
"import.adv.o1": "O(1)-Attention",
|
||||
"import.adv.o1Hint": "Streaming-Attention mit konstantem Speicher: der KV-Cache wird nach dem Prefill freigegeben, lange Kontexte lassen den Speicher also nicht wachsen. Die Gewichte bleiben Byte-identisch — gespeichert wird nur ein Hinweis für die Ladezeit; du kannst jederzeit zurückwechseln (mit --o1 off starten). Auf Qwen3-0.6B validiert: alle Layer ≈ ×1,18 Perplexität.",
|
||||
"import.adv.o1Off": "aus — exakte Attention (Standard)",
|
||||
"import.adv.o1All": "alle — jeden Softmax-Layer konvertieren",
|
||||
"import.adv.o1Deep": "deep N — nur die N tiefsten Layer",
|
||||
"import.adv.o1Custom": "benutzerdefiniert — explizite Layer-Liste",
|
||||
"import.adv.o1DeepN": "N tiefe Layer",
|
||||
"import.adv.o1Layers": "Layer (kommagetrennt)",
|
||||
"import.adv.o1mHint": "Landmark-Budget (Standard 32)",
|
||||
"import.adv.o1windowHint": "Breite des exakten Fensters (Standard 128)",
|
||||
"import.adv.o1sinkHint": "permanente exakte Sink-Keys (Standard 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Ersteinrichtung",
|
||||
"onboarding.welcome": "Willkommen bei Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Wähle deine Sprache — du kannst sie jederzeit in der Kopfzeile ändern.",
|
||||
"onboarding.next": "Weiter",
|
||||
"onboarding.back": "Zurück",
|
||||
"onboarding.skip": "Vorerst überspringen",
|
||||
"onboarding.skipSetup": "Überspringen",
|
||||
"onboarding.finish": "Einrichtung abschließen",
|
||||
"onboarding.featuredTitle": "Fertige Modelle (ein Klick)",
|
||||
"onboarding.featuredSub": "Das sind bereits konvertierte .cmf-Modelle von infosave auf HuggingFace — einfach herunterladen, keine Konvertierung nötig. Wähle eines oder mehrere für den Sofortstart.",
|
||||
"onboarding.download": "Herunterladen",
|
||||
"onboarding.hfTitle": "Jedes HuggingFace-Modell — automatisch konvertiert",
|
||||
"onboarding.hfSub": "Suche ein beliebiges Modell, wähle eine Quantisierung, und wir konvertieren es automatisch nach .cmf. Keine manuellen Config-Änderungen.",
|
||||
"onboarding.noJobs": "Noch keine Downloads.",
|
||||
"onboarding.doneTitle": "Alles bereit",
|
||||
"onboarding.doneSub": "Heruntergeladene Modelle werden automatisch registriert. Weitere kannst du unter Import oder Modelle hinzufügen.",
|
||||
"onboarding.doneHint": "„Einrichtung abschließen“ verbindet jeden fertigen Download als lokales Modell, startet dessen .cmf-Server und richtet das Routing darauf aus. Alles lässt sich später unter Modelle ändern.",
|
||||
"onboarding.willRegister": "Werden als lokale Modelle verbunden",
|
||||
"onboarding.nothingToRegister": "Noch keine abgeschlossenen Downloads — nichts zu verbinden. Geh einen Schritt zurück und wähle ein Modell.",
|
||||
"onboarding.registeredN": "{n} Modell(e) verbunden: {ids}",
|
||||
"onboarding.runtimeMissing": "Die cortiq-Runtime ist nicht installiert — installiere sie unter Einstellungen → Lokale Modelle, um diese Modelle bereitzustellen.",
|
||||
"onboarding.downloaded": "heruntergeladen",
|
||||
"onboarding.update": "Aktualisieren",
|
||||
"onboarding.deleteFile": "Die .cmf-Datei von der Festplatte löschen",
|
||||
"onboarding.confirmDelete": "„{id}“ löschen? Die .cmf-Datei wird von der Festplatte entfernt.",
|
||||
"onboarding.unregistered": "Ebenfalls abgemeldet: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+51
-2
@@ -139,6 +139,18 @@ export default {
|
||||
"models.form.apiKeyHint": "Stored securely on the gateway; it is not shown again.",
|
||||
"models.form.keyStored": "A key is saved — leave blank to keep it, or paste a new one.",
|
||||
"models.form.keyFromEnv": "Using a key from an environment variable.",
|
||||
"models.form.paramsTitle": "Generation & Thinking Parameters (O(1), MTP, Temperature)",
|
||||
"models.form.temperature": "Temperature",
|
||||
"models.form.temperatureHint": "Generation temperature (0.0 = deterministic, 1.0+ = creative).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Sampling threshold (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Max Tokens",
|
||||
"models.form.maxTokensHint": "Maximum completion tokens to generate.",
|
||||
"models.form.thinkBudget": "Thinking Mode / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Reasoning token budget (0 = disabled, N > 0 = max reasoning tokens).",
|
||||
"models.form.systemPrompt": "Default System Prompt",
|
||||
"models.form.systemPromptHint": "Prepend automatically if request lacks a system prompt.",
|
||||
"models.form.systemPromptPlaceholder": "You are a helpful assistant...",
|
||||
"models.form.testKey": "Test & load models",
|
||||
"models.form.testing": "checking…",
|
||||
"models.form.foundModels": "{n} models loaded",
|
||||
@@ -267,7 +279,6 @@ export default {
|
||||
"settings.cmf.checkPort": "Check port",
|
||||
"settings.cmf.portFree": "port is free",
|
||||
"settings.cmf.portBusy": "port is in use",
|
||||
"settings.cmf.portOwn": "in use by this gateway's current server — freed on restart, replacing is fine",
|
||||
"settings.cmf.useSuggested": "use {port}",
|
||||
"settings.cmf.serving": "serving {model} on :{port}",
|
||||
"settings.cmf.intro": "Run models locally with the cortiq engine — works even offline or when the router is down.",
|
||||
@@ -375,7 +386,7 @@ export default {
|
||||
"import.failed": "Conversion failed — see the log.",
|
||||
"import.register": "Register as local model",
|
||||
"import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
|
||||
"import.jobs": "Conversions",
|
||||
"import.jobs": "Downloads & Conversions",
|
||||
"import.cancel": "Cancel",
|
||||
"import.cancelling": "Cancelling — cleaning up partial output…",
|
||||
"import.delete": "Delete",
|
||||
@@ -385,12 +396,22 @@ export default {
|
||||
"import.state.done": "done",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelled",
|
||||
"import.featuredTitle": "Featured Ready-to-Run .cmf Models",
|
||||
"import.featuredSub": "Pre-converted .cmf models from the infosave account on HuggingFace — the list is read live, so newly published models appear here on their own.",
|
||||
"import.readyCmfBadge": "Ready .cmf",
|
||||
"import.directDownload": "Download",
|
||||
"import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
|
||||
"import.q.q8_row": "8-bit per-row — simple, robust.",
|
||||
"import.q.q4": "4-bit block — smallest, lower quality.",
|
||||
"import.q.q4_tiled": "4-bit tiled (Q4_TILED) — 1.66x faster on Metal GPU.",
|
||||
"import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
|
||||
"import.q.q1t": "1.58-bit ternary (Q1T) — BitNet b1.58 with sparse outliers.",
|
||||
"import.q.q1s": "1-bit sparse (Q1S) — 1-bit base with holographic outliers.",
|
||||
"import.q.q1p": "1-bit PTQ (Q1P) — 1-bit with error diffusion.",
|
||||
"import.q.q1": "1-bit (Q1) — pure binary format.",
|
||||
"import.q.f16": "16-bit — no quantization.",
|
||||
"import.q.f32": "32-bit — full precision (large file).",
|
||||
|
||||
"import.adv.title": "Advanced settings",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Linear core",
|
||||
@@ -412,4 +433,32 @@ export default {
|
||||
"import.adv.o1mHint": "landmark budget (default 32)",
|
||||
"import.adv.o1windowHint": "exact-window width (default 128)",
|
||||
"import.adv.o1sinkHint": "permanent exact sink keys (default 4)",
|
||||
|
||||
// onboarding (first run)
|
||||
"onboarding.title": "Welcome setup",
|
||||
"onboarding.welcome": "Welcome to Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choose your language — you can change it anytime in the header.",
|
||||
"onboarding.next": "Next",
|
||||
"onboarding.back": "Back",
|
||||
"onboarding.skip": "Skip for now",
|
||||
"onboarding.skipSetup": "Skip",
|
||||
"onboarding.finish": "Finish setup",
|
||||
"onboarding.featuredTitle": "Ready-made models (one click)",
|
||||
"onboarding.featuredSub": "These are pre-converted .cmf models from infosave on HuggingFace — just Download, no conversion needed. Pick one or more to get started instantly.",
|
||||
"onboarding.download": "Download",
|
||||
"onboarding.hfTitle": "Any HuggingFace model — auto convert",
|
||||
"onboarding.hfSub": "Search any model, pick a quantization, and we convert it to .cmf automatically. No manual config edits.",
|
||||
"onboarding.noJobs": "No downloads yet.",
|
||||
"onboarding.doneTitle": "You're all set",
|
||||
"onboarding.doneSub": "Models you downloaded will be registered automatically. You can add more in Import or Models.",
|
||||
"onboarding.doneHint": "Finish connects every completed download as a local model, starts its .cmf server and points routing at it. You can change all of it later in Models.",
|
||||
"onboarding.willRegister": "Will be connected as local models",
|
||||
"onboarding.nothingToRegister": "No finished downloads yet — nothing to connect. Go back a step to pick a model.",
|
||||
"onboarding.registeredN": "Connected {n} model(s): {ids}",
|
||||
"onboarding.runtimeMissing": "The cortiq runtime isn't installed — install it in Settings → Local models to serve these.",
|
||||
"onboarding.downloaded": "downloaded",
|
||||
"onboarding.update": "Update",
|
||||
"onboarding.deleteFile": "Delete the .cmf file from disk",
|
||||
"onboarding.confirmDelete": "Delete '{id}'? The .cmf file will be removed from disk.",
|
||||
"onboarding.unregistered": "Also unregistered: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+191
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Enrutamiento",
|
||||
"nav.protocols": "Protocolos",
|
||||
"nav.keys": "Claves API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Zona de pruebas",
|
||||
"nav.settings": "Configuración",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clave del router y pagos",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "por tarea",
|
||||
"dash.groupby.account": "por cuenta",
|
||||
"dash.recent.title": "Solicitudes recientes",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba el Playground.",
|
||||
"dash.recent.empty": "Sin solicitudes registradas. Prueba la Zona de pruebas.",
|
||||
"dash.col.time": "Hora",
|
||||
"dash.col.model": "Modelo",
|
||||
"dash.col.tier": "Nivel",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "todos",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Zona de pruebas",
|
||||
"pg.subtitle": "Envía un prompt por el pipeline en vivo e inspecciona la decisión de enrutamiento.",
|
||||
"pg.system": "Prompt del sistema",
|
||||
"pg.systemPlaceholder": "mensaje de sistema opcional",
|
||||
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "TTL de entrada",
|
||||
"settings.cache.embed": "Modelo de embeddings",
|
||||
"settings.cache.note": "Los ajustes del caché se aplican tras reiniciar.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importar modelo",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Borrar registros",
|
||||
"dash.clearConfirm": "¿Borrar todas las estadísticas y registros guardados? Esta acción no se puede deshacer.",
|
||||
"dash.cleared": "Registros borrados",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Introduce un id o un nombre de modelo",
|
||||
"models.form.apiKey": "Clave API",
|
||||
"models.form.apiKeyPlaceholder": "pega tu clave API",
|
||||
"models.form.apiKeyNotNeeded": "no necesaria (servidor local)",
|
||||
"models.form.apiKeyHint": "Se guarda de forma segura en la pasarela; no se vuelve a mostrar.",
|
||||
"models.form.keyStored": "Hay una clave guardada — déjalo vacío para conservarla o pega una nueva.",
|
||||
"models.form.keyFromEnv": "Se está usando una clave de una variable de entorno.",
|
||||
"models.form.paramsTitle": "Parámetros de generación y razonamiento (O(1), MTP, temperatura)",
|
||||
"models.form.temperature": "Temperatura",
|
||||
"models.form.temperatureHint": "Temperatura de generación (0.0 = determinista, 1.0+ = creativa).",
|
||||
"models.form.topP": "Top-P (muestreo nucleus)",
|
||||
"models.form.topPHint": "Umbral de muestreo (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens máx.",
|
||||
"models.form.maxTokensHint": "Número máximo de tokens generados.",
|
||||
"models.form.thinkBudget": "Modo razonamiento / presupuesto de pensamiento",
|
||||
"models.form.thinkBudgetHint": "Presupuesto de tokens de razonamiento (0 = desactivado, N > 0 = máximo de tokens de razonamiento).",
|
||||
"models.form.systemPrompt": "Prompt de sistema por defecto",
|
||||
"models.form.systemPromptHint": "Se antepone automáticamente si la petición no trae prompt de sistema.",
|
||||
"models.form.systemPromptPlaceholder": "Eres un asistente útil…",
|
||||
"models.form.testKey": "Probar y cargar modelos",
|
||||
"models.form.testing": "comprobando…",
|
||||
"models.form.foundModels": "{n} modelos cargados",
|
||||
"models.form.modelPlaceholder": "escribe para buscar…",
|
||||
"models.form.modelHint": "Pulsa «Probar y cargar modelos» para traer la lista con tu clave y luego elige uno.",
|
||||
"models.managed": "gestionado",
|
||||
"models.managedHint": "Lo proporciona el servidor CMF gestionado (Ajustes → Modelos locales). Configúralo allí.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Usar el router (desactivado = solo modelos locales)",
|
||||
"settings.cmf": "Modelos locales (CMF)",
|
||||
"settings.cmf.localOnly": "Usar solo modelos locales",
|
||||
"settings.cmf.manage": "Ejecutar un servidor de modelo local",
|
||||
"settings.cmf.autoInstall": "Instalar cortiq automáticamente (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Actualizar cortiq automáticamente",
|
||||
"settings.cmf.model": "Ruta del modelo .cmf local",
|
||||
"settings.cmf.port": "Puerto del servidor local",
|
||||
"settings.cmf.modelId": "Id del modelo en el pool",
|
||||
"settings.cmf.note": "Sirve un .cmf local mediante cortiq (github.com/infosave2007/cmf). Funciona incluso cuando el router no está disponible.",
|
||||
"settings.cmf.threads": "Hilos de trabajo",
|
||||
"settings.cmf.gpu": "Usar GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Los hilos paralelizan los matvec de decodificación (~3× más rápido; 0 = valor por defecto del runtime). Descargar en GPU ayuda sobre todo al prefill de prompts largos — la decodificación está limitada por el ancho de banda de memoria, así que en Apple Silicon es prácticamente neutro.",
|
||||
"settings.cmf.pick": "selecciona un modelo",
|
||||
"settings.cmf.models": "Modelos locales",
|
||||
"settings.cmf.modelsHelp": "Cada fila es un modelo que la pasarela ejecuta (su propio cortiq serve en su propio puerto) y registra en el pool. Añade tantos como archivos .cmf y puertos libres tengas.",
|
||||
"settings.cmf.rowHint": "id · archivo .cmf · puerto · hilos · GPU. Los cambios se aplican al reiniciar; borrar una fila detiene ese servidor.",
|
||||
"settings.cmf.addServer": "+ Añadir modelo local",
|
||||
"settings.cmf.checkPort": "Comprobar puerto",
|
||||
"settings.cmf.portFree": "puerto libre",
|
||||
"settings.cmf.portBusy": "puerto ocupado",
|
||||
"settings.cmf.useSuggested": "usar {port}",
|
||||
"settings.cmf.serving": "sirviendo {model} en :{port}",
|
||||
"settings.cmf.intro": "Ejecuta modelos en local con el motor cortiq — funciona incluso sin conexión o cuando el router está caído.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "La herramienta de línea de comandos `cortiq` que carga y ejecuta modelos .cmf. Instálala una vez antes de arrancar un servidor local.",
|
||||
"settings.cmf.install": "Instalar cortiq",
|
||||
"settings.cmf.reinstall": "Reinstalar / actualizar",
|
||||
"settings.cmf.installing": "instalando… (compila, unos minutos)",
|
||||
"settings.cmf.installedOk": "cortiq instalado",
|
||||
"settings.cmf.notInstalled": "no instalado",
|
||||
"settings.cmf.addModel": "Añadir un modelo",
|
||||
"settings.cmf.addModelHelp": "Los modelos se descargan y convierten en la pestaña Importar (HuggingFace → .cmf). Después apunta el servidor local al archivo .cmf de abajo.",
|
||||
"settings.cmf.importNote": "¿Aún no tienes modelo? Importa uno:",
|
||||
"settings.cmf.importLink": "Abrir Importar",
|
||||
"settings.cmf.modelHelp": "Ruta del archivo .cmf que se va a servir (p. ej. models/qwen.cmf), generado en la pestaña Importar. ¿Varios modelos para distintas tareas? Impórtalos y cambia esta ruta.",
|
||||
"settings.cmf.portHelp": "Puerto TCP en el que escucha el servidor de modelo local (8081 por defecto). Cámbialo solo si choca con otro servicio.",
|
||||
"settings.cmf.threadsHelp": "Cuántos núcleos de CPU usar para la generación. Más = más rápido (~3× frente a un núcleo); 0 deja que decida el runtime.",
|
||||
"settings.cmf.gpuHelp": "Descargar el cálculo matricial en la GPU (Metal/wgpu). En Apple Silicon acelera sobre todo los prompts largos — la decodificación está limitada por el ancho de banda de memoria.",
|
||||
"settings.cmf.localOnlyHelp": "Enviar cada petición al modelo local, ignorando el router y cualquier proveedor en la nube.",
|
||||
"settings.cmf.manageHelp": "La pasarela arranca `cortiq serve` para el .cmf de abajo al iniciarse y enruta hacia él. Requiere cortiq instalado (paso 1). Se aplica al reiniciar.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq falta al arrancar, instalarlo automáticamente desde crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Al arrancar, actualizar cortiq a la versión más reciente si existe una más nueva.",
|
||||
"settings.router.enabledHelp": "Dejar que el router de cortiq clasifique cada petición (tipo de tarea + complejidad) y elija un modelo. Desactivado = usar solo la tabla de enrutado local.",
|
||||
"settings.router.verifyTlsHelp": "Verificar el certificado TLS del router. Desactívalo solo con un certificado autofirmado o ligado a una IP.",
|
||||
"settings.telemetry.metricsHelp": "Exponer métricas Prometheus en GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Añadir un campo `cortiq` (tarea, modelo, coste) al cuerpo de cada respuesta. Las cabeceras de respuesta se envían siempre.",
|
||||
"settings.stats.enabledHelp": "Registrar las peticiones recientes para alimentar el panel y las gráficas.",
|
||||
"settings.cache.enabledHelp": "Caché semántica: devolver una respuesta guardada cuando un prompt nuevo se parece mucho a uno anterior, evitando la llamada al modelo. Requiere un modelo de embeddings en el pool.",
|
||||
"settings.log.bodiesHelp": "Registrar el cuerpo completo de los mensajes de petición y respuesta. Desactivado por defecto por privacidad.",
|
||||
"settings.router.saveKey": "Guardar clave",
|
||||
"settings.router.keySaved": "Clave del router guardada",
|
||||
"settings.router.keyEmpty": "Pega antes la clave",
|
||||
"settings.router.secretHelp": "Pega la clave del correo de api.allaigate.com y pulsa Guardar clave. NO es un valor de «Claves API» — esas son para los clientes que llaman a tu pasarela.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clave guardada (pega otra para reemplazarla)",
|
||||
"settings.route.strategyHelp": "Qué texto se envía al router para clasificar: last_user = solo el último mensaje del usuario; last_user_plus_system = último mensaje del usuario + prompt de sistema; concat_all = toda la conversación.",
|
||||
"settings.route.maxCharsHelp": "Máximo de caracteres de ese texto enviados al router (lo que exceda se recorta). Más = clasificación más precisa, pero llamadas al router más lentas y caras.",
|
||||
"settings.route.cacheTtlHelp": "Cuánto tiempo se cachea una decisión de enrutado para una petición idéntica antes de volver a preguntar al router (p. ej. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Perfil de enrutado por defecto cuando la petición no indica ninguno: cost-saver (el más barato), balanced o quality-first (el mejor modelo). Influye en qué modelo elige el router.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Busca en HuggingFace, convierte a un .cmf local y sírvelo.",
|
||||
"import.searchPlaceholder": "Buscar modelos en HuggingFace (p. ej. qwen3, llama)…",
|
||||
"import.searching": "Buscando…",
|
||||
"import.noResults": "No se encontraron modelos.",
|
||||
"import.configure": "Configurar la conversión",
|
||||
"import.outName": "Nombre de salida",
|
||||
"import.outNameHint": "Nombre del archivo .cmf (letras, dígitos, - y _).",
|
||||
"import.quant": "Cuantización",
|
||||
"import.convert": "Convertir y cargar",
|
||||
"import.converting": "Convirtiendo",
|
||||
"import.started": "Conversión iniciada",
|
||||
"import.failed": "La conversión falló — revisa el registro.",
|
||||
"import.register": "Registrar como modelo local",
|
||||
"import.registered": "Registrado como «{id}». Ejecuta cortiq-server para servirlo.",
|
||||
"import.jobs": "Descargas y conversiones",
|
||||
"import.cancel": "Cancelar",
|
||||
"import.cancelling": "Cancelando — limpiando el archivo incompleto…",
|
||||
"import.delete": "Eliminar",
|
||||
"import.deleteConfirm": "¿Eliminar este modelo y su archivo .cmf convertido del disco?",
|
||||
"import.deleted": "Eliminado junto con sus archivos",
|
||||
"import.state.running": "en curso",
|
||||
"import.state.done": "listo",
|
||||
"import.state.error": "error",
|
||||
"import.state.cancelled": "cancelado",
|
||||
"import.featuredTitle": "Modelos .cmf listos para usar",
|
||||
"import.featuredSub": "Modelos .cmf ya convertidos de la cuenta infosave en HuggingFace — la lista se lee en vivo, los modelos nuevos aparecen solos.",
|
||||
"import.readyCmfBadge": ".cmf listo",
|
||||
"import.directDownload": "Descargar",
|
||||
"import.q.q8_2f": "8 bits de dos campos (𝒲×θ) — recomendado: mejor relación calidad/tamaño.",
|
||||
"import.q.q8_row": "8 bits por fila — simple y robusto.",
|
||||
"import.q.q4": "4 bits por bloque — el más pequeño, menor calidad.",
|
||||
"import.q.q4_tiled": "4 bits en mosaico (Q4_TILED) — 1,66× más rápido en GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — presupuesto de bits por experto.",
|
||||
"import.q.q1t": "Ternario de 1,58 bits (Q1T) — BitNet b1.58 con valores atípicos dispersos.",
|
||||
"import.q.q1s": "1 bit disperso (Q1S) — base de 1 bit con valores atípicos holográficos.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit con difusión de error.",
|
||||
"import.q.q1": "1 bit (Q1) — formato puramente binario.",
|
||||
"import.q.f16": "16 bits — sin cuantización.",
|
||||
"import.q.f32": "32 bits — precisión completa (archivo grande).",
|
||||
"import.adv.title": "Ajustes avanzados",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Núcleo lineal",
|
||||
"import.adv.linearHint": "Cómo se trasladan o pliegan las capas de atención lineal.",
|
||||
"import.adv.nphase": "vmf_phase fases/cabeza",
|
||||
"import.adv.vbitShape": "Forma del presupuesto VBIT",
|
||||
"import.adv.meanBits": "Bits medios VBIT",
|
||||
"import.adv.shard": "Tamaño máx. de shard (GB)",
|
||||
"import.adv.noShard": "sin fragmentar",
|
||||
"import.adv.skipMtp": "omitir la cabeza MTP",
|
||||
"import.adv.o1": "Atención O(1)",
|
||||
"import.adv.o1Hint": "Atención en streaming con memoria constante: la caché KV se libera tras el prefill, así que los contextos largos no hacen crecer la memoria. Los pesos quedan idénticos byte a byte — solo se guarda una indicación para el momento de carga; puedes volver atrás cuando quieras (servir con --o1 off). Validado en Qwen3-0.6B: todas las capas ≈ ×1,18 de perplejidad.",
|
||||
"import.adv.o1Off": "desactivada — atención exacta (por defecto)",
|
||||
"import.adv.o1All": "all — convertir todas las capas softmax",
|
||||
"import.adv.o1Deep": "deep N — solo las N capas más profundas",
|
||||
"import.adv.o1Custom": "personalizado — lista explícita de capas",
|
||||
"import.adv.o1DeepN": "N capas profundas",
|
||||
"import.adv.o1Layers": "Capas (separadas por comas)",
|
||||
"import.adv.o1mHint": "presupuesto de landmarks (32 por defecto)",
|
||||
"import.adv.o1windowHint": "ancho de la ventana exacta (128 por defecto)",
|
||||
"import.adv.o1sinkHint": "claves sink exactas permanentes (4 por defecto)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuración inicial",
|
||||
"onboarding.welcome": "Bienvenido a Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Elige tu idioma — puedes cambiarlo cuando quieras en la cabecera.",
|
||||
"onboarding.next": "Siguiente",
|
||||
"onboarding.back": "Atrás",
|
||||
"onboarding.skip": "Omitir por ahora",
|
||||
"onboarding.skipSetup": "Omitir",
|
||||
"onboarding.finish": "Finalizar configuración",
|
||||
"onboarding.featuredTitle": "Modelos listos para usar (un clic)",
|
||||
"onboarding.featuredSub": "Son modelos .cmf ya convertidos de infosave en HuggingFace — basta con descargarlos, sin conversión. Elige uno o varios para empezar al instante.",
|
||||
"onboarding.download": "Descargar",
|
||||
"onboarding.hfTitle": "Cualquier modelo de HuggingFace — conversión automática",
|
||||
"onboarding.hfSub": "Busca cualquier modelo, elige una cuantización y lo convertimos a .cmf automáticamente. Sin editar la configuración a mano.",
|
||||
"onboarding.noJobs": "Todavía no hay descargas.",
|
||||
"onboarding.doneTitle": "Todo listo",
|
||||
"onboarding.doneSub": "Los modelos descargados se registrarán automáticamente. Puedes añadir más en Importar o Modelos.",
|
||||
"onboarding.doneHint": "«Finalizar configuración» conecta cada descarga completada como modelo local, arranca su servidor .cmf y apunta el enrutado hacia él. Todo esto se puede cambiar luego en Modelos.",
|
||||
"onboarding.willRegister": "Se conectarán como modelos locales",
|
||||
"onboarding.nothingToRegister": "Aún no hay descargas terminadas — no hay nada que conectar. Vuelve un paso atrás y elige un modelo.",
|
||||
"onboarding.registeredN": "{n} modelo(s) conectado(s): {ids}",
|
||||
"onboarding.runtimeMissing": "El runtime cortiq no está instalado — instálalo en Ajustes → Modelos locales para poder servirlos.",
|
||||
"onboarding.downloaded": "descargado",
|
||||
"onboarding.update": "Actualizar",
|
||||
"onboarding.deleteFile": "Eliminar el archivo .cmf del disco",
|
||||
"onboarding.confirmDelete": "¿Eliminar «{id}»? El archivo .cmf se borrará del disco.",
|
||||
"onboarding.unregistered": "También se dio de baja: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+191
-3
@@ -30,7 +30,7 @@ export default {
|
||||
"nav.routing": "Routage",
|
||||
"nav.protocols": "Protocoles",
|
||||
"nav.keys": "Clés API",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Bac à sable",
|
||||
"nav.settings": "Paramètres",
|
||||
"nav.foot": "Gateway · v{v}",
|
||||
"nav.billing": "Clé routeur & facturation",
|
||||
@@ -76,7 +76,7 @@ export default {
|
||||
"dash.groupby.label": "par tâche",
|
||||
"dash.groupby.account": "par compte",
|
||||
"dash.recent.title": "Requêtes récentes",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Playground.",
|
||||
"dash.recent.empty": "Aucune requête enregistrée. Essayez le Bac à sable.",
|
||||
"dash.col.time": "Heure",
|
||||
"dash.col.model": "Modèle",
|
||||
"dash.col.tier": "Niveau",
|
||||
@@ -189,7 +189,7 @@ export default {
|
||||
"keys.allowAll": "tous",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Bac à sable",
|
||||
"pg.subtitle": "Envoyez une invite via le pipeline en direct et inspectez la décision de routage.",
|
||||
"pg.system": "Invite système",
|
||||
"pg.systemPlaceholder": "message système optionnel",
|
||||
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "TTL d'entrée",
|
||||
"settings.cache.embed": "Modèle d'embeddings",
|
||||
"settings.cache.note": "Les paramètres du cache prennent effet après un redémarrage.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Importer un modèle",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Effacer les journaux",
|
||||
"dash.clearConfirm": "Effacer toutes les statistiques et les journaux enregistrés ? Action irréversible.",
|
||||
"dash.cleared": "Journaux effacés",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Saisissez un identifiant ou un nom de modèle",
|
||||
"models.form.apiKey": "Clé API",
|
||||
"models.form.apiKeyPlaceholder": "collez votre clé API",
|
||||
"models.form.apiKeyNotNeeded": "non requise (serveur local)",
|
||||
"models.form.apiKeyHint": "Stockée de façon sécurisée sur la passerelle ; elle n'est plus affichée ensuite.",
|
||||
"models.form.keyStored": "Une clé est enregistrée — laissez vide pour la conserver, ou collez-en une nouvelle.",
|
||||
"models.form.keyFromEnv": "Une clé issue d'une variable d'environnement est utilisée.",
|
||||
"models.form.paramsTitle": "Paramètres de génération et de raisonnement (O(1), MTP, température)",
|
||||
"models.form.temperature": "Température",
|
||||
"models.form.temperatureHint": "Température de génération (0.0 = déterministe, 1.0+ = créatif).",
|
||||
"models.form.topP": "Top-P (échantillonnage nucleus)",
|
||||
"models.form.topPHint": "Seuil d'échantillonnage (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Tokens max.",
|
||||
"models.form.maxTokensHint": "Nombre maximal de tokens générés.",
|
||||
"models.form.thinkBudget": "Mode raisonnement / budget de réflexion",
|
||||
"models.form.thinkBudgetHint": "Budget de tokens de raisonnement (0 = désactivé, N > 0 = maximum de tokens de raisonnement).",
|
||||
"models.form.systemPrompt": "Prompt système par défaut",
|
||||
"models.form.systemPromptHint": "Ajouté automatiquement si la requête ne contient pas de prompt système.",
|
||||
"models.form.systemPromptPlaceholder": "Vous êtes un assistant utile…",
|
||||
"models.form.testKey": "Tester et charger les modèles",
|
||||
"models.form.testing": "vérification…",
|
||||
"models.form.foundModels": "{n} modèles chargés",
|
||||
"models.form.modelPlaceholder": "tapez pour rechercher…",
|
||||
"models.form.modelHint": "Cliquez sur « Tester et charger les modèles » pour récupérer la liste liée à votre clé, puis choisissez-en un.",
|
||||
"models.managed": "géré",
|
||||
"models.managedHint": "Fourni par le serveur CMF géré (Paramètres → Modèles locaux). Configurez-le là-bas.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Utiliser le routeur (désactivé = modèles locaux uniquement)",
|
||||
"settings.cmf": "Modèles locaux (CMF)",
|
||||
"settings.cmf.localOnly": "Utiliser uniquement les modèles locaux",
|
||||
"settings.cmf.manage": "Lancer un serveur de modèle local",
|
||||
"settings.cmf.autoInstall": "Installer cortiq automatiquement (crates.io)",
|
||||
"settings.cmf.autoUpdate": "Mettre cortiq à jour automatiquement",
|
||||
"settings.cmf.model": "Chemin du modèle .cmf local",
|
||||
"settings.cmf.port": "Port du serveur local",
|
||||
"settings.cmf.modelId": "Identifiant du modèle dans le pool",
|
||||
"settings.cmf.note": "Sert un .cmf local via cortiq (github.com/infosave2007/cmf). Fonctionne même quand le routeur est indisponible.",
|
||||
"settings.cmf.threads": "Threads de travail",
|
||||
"settings.cmf.gpu": "Utiliser le GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Les threads parallélisent les matvecs de décodage (~3× plus rapide ; 0 = valeur par défaut du runtime). Le déport GPU aide surtout au prefill des longs prompts — le décodage est limité par la bande passante mémoire, donc sur Apple Silicon l'effet est à peu près neutre.",
|
||||
"settings.cmf.pick": "choisir un modèle",
|
||||
"settings.cmf.models": "Modèles locaux",
|
||||
"settings.cmf.modelsHelp": "Chaque ligne est un modèle que la passerelle exécute (son propre cortiq serve sur son propre port) et enregistre dans le pool. Ajoutez-en autant que vous avez de fichiers .cmf et de ports libres.",
|
||||
"settings.cmf.rowHint": "id · fichier .cmf · port · threads · GPU. Les changements prennent effet au redémarrage ; supprimer une ligne arrête ce serveur.",
|
||||
"settings.cmf.addServer": "+ Ajouter un modèle local",
|
||||
"settings.cmf.checkPort": "Vérifier le port",
|
||||
"settings.cmf.portFree": "port libre",
|
||||
"settings.cmf.portBusy": "port occupé",
|
||||
"settings.cmf.useSuggested": "utiliser {port}",
|
||||
"settings.cmf.serving": "{model} servi sur :{port}",
|
||||
"settings.cmf.intro": "Exécutez des modèles en local avec le moteur cortiq — fonctionne même hors ligne ou quand le routeur est en panne.",
|
||||
"settings.cmf.runtime": "Runtime cortiq",
|
||||
"settings.cmf.runtimeHelp": "L'outil en ligne de commande `cortiq` qui charge et exécute les modèles .cmf. Installez-le une fois avant de démarrer un serveur local.",
|
||||
"settings.cmf.install": "Installer cortiq",
|
||||
"settings.cmf.reinstall": "Réinstaller / mettre à jour",
|
||||
"settings.cmf.installing": "installation… (compilation, quelques minutes)",
|
||||
"settings.cmf.installedOk": "cortiq installé",
|
||||
"settings.cmf.notInstalled": "non installé",
|
||||
"settings.cmf.addModel": "Ajouter un modèle",
|
||||
"settings.cmf.addModelHelp": "Les modèles se téléchargent et se convertissent dans l'onglet Importer (HuggingFace → .cmf). Faites ensuite pointer le serveur local vers le fichier .cmf ci-dessous.",
|
||||
"settings.cmf.importNote": "Pas encore de modèle ? Importez-en un :",
|
||||
"settings.cmf.importLink": "Ouvrir Importer",
|
||||
"settings.cmf.modelHelp": "Chemin du fichier .cmf à servir (par ex. models/qwen.cmf), produit par l'onglet Importer. Plusieurs modèles pour des tâches différentes ? Importez-les et changez ce chemin.",
|
||||
"settings.cmf.portHelp": "Port TCP sur lequel écoute le serveur de modèle local (8081 par défaut). À changer seulement en cas de conflit avec un autre service.",
|
||||
"settings.cmf.threadsHelp": "Nombre de cœurs CPU utilisés pour la génération. Plus = plus rapide (~3× par rapport à un seul cœur) ; 0 laisse le runtime décider.",
|
||||
"settings.cmf.gpuHelp": "Déporter les calculs matriciels sur le GPU (Metal/wgpu). Sur Apple Silicon, cela accélère surtout les longs prompts — le décodage est limité par la bande passante mémoire.",
|
||||
"settings.cmf.localOnlyHelp": "Router chaque requête vers le modèle local, en ignorant le routeur et tous les fournisseurs cloud.",
|
||||
"settings.cmf.manageHelp": "Au démarrage, la passerelle lance `cortiq serve` pour le .cmf ci-dessous et y route les requêtes. Nécessite cortiq installé (étape 1). Prend effet au redémarrage.",
|
||||
"settings.cmf.autoInstallHelp": "Si cortiq est absent au démarrage, l'installer automatiquement depuis crates.io.",
|
||||
"settings.cmf.autoUpdateHelp": "Au démarrage, mettre cortiq à niveau vers la dernière version si elle est plus récente.",
|
||||
"settings.router.enabledHelp": "Laisser le routeur cortiq classer chaque requête (type de tâche + complexité) et choisir un modèle. Désactivé = utiliser uniquement la table de routage locale.",
|
||||
"settings.router.verifyTlsHelp": "Vérifier le certificat TLS du routeur. À désactiver uniquement pour un certificat auto-signé ou lié à une IP.",
|
||||
"settings.telemetry.metricsHelp": "Exposer les métriques Prometheus sur GET /metrics.",
|
||||
"settings.cortiq.echoHelp": "Ajouter un champ `cortiq` (tâche, modèle, coût) au corps de chaque réponse. Les en-têtes de réponse sont toujours envoyés.",
|
||||
"settings.stats.enabledHelp": "Enregistrer les requêtes récentes pour alimenter le tableau de bord et les graphiques.",
|
||||
"settings.cache.enabledHelp": "Cache sémantique : renvoyer une réponse enregistrée quand un nouveau prompt ressemble beaucoup à un précédent, en évitant l'appel au modèle. Nécessite un modèle d'embedding dans le pool.",
|
||||
"settings.log.bodiesHelp": "Journaliser le corps complet des messages de requête et de réponse. Désactivé par défaut pour la confidentialité.",
|
||||
"settings.router.saveKey": "Enregistrer la clé",
|
||||
"settings.router.keySaved": "Clé du routeur enregistrée",
|
||||
"settings.router.keyEmpty": "Collez d'abord la clé",
|
||||
"settings.router.secretHelp": "Collez la clé reçue par e-mail depuis api.allaigate.com, puis cliquez sur Enregistrer la clé. Ce n'est PAS une valeur de « Clés API » — celles-ci servent aux clients qui appellent votre passerelle.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — clé enregistrée (recollez-en une pour la remplacer)",
|
||||
"settings.route.strategyHelp": "Texte envoyé au routeur pour la classification : last_user = uniquement le dernier message utilisateur ; last_user_plus_system = dernier message utilisateur + prompt système ; concat_all = toute la conversation.",
|
||||
"settings.route.maxCharsHelp": "Nombre maximal de caractères de ce texte envoyés au routeur (au-delà, il est tronqué). Plus = classification plus précise, mais appels au routeur plus lents et plus coûteux.",
|
||||
"settings.route.cacheTtlHelp": "Durée de mise en cache d'une décision de routage pour une requête identique avant de réinterroger le routeur (par ex. 60s, 5m).",
|
||||
"settings.route.profileHelp": "Profil de routage par défaut quand la requête n'en précise aucun : cost-saver (le moins cher), balanced ou quality-first (meilleur modèle). Influence le modèle choisi par le routeur.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "Cherchez sur HuggingFace, convertissez en .cmf local et servez-le.",
|
||||
"import.searchPlaceholder": "Rechercher des modèles HuggingFace (par ex. qwen3, llama)…",
|
||||
"import.searching": "Recherche…",
|
||||
"import.noResults": "Aucun modèle trouvé.",
|
||||
"import.configure": "Configurer la conversion",
|
||||
"import.outName": "Nom de sortie",
|
||||
"import.outNameHint": "Nom du fichier .cmf (lettres, chiffres, - et _).",
|
||||
"import.quant": "Quantification",
|
||||
"import.convert": "Convertir et charger",
|
||||
"import.converting": "Conversion",
|
||||
"import.started": "Conversion lancée",
|
||||
"import.failed": "Échec de la conversion — voir le journal.",
|
||||
"import.register": "Enregistrer comme modèle local",
|
||||
"import.registered": "Enregistré sous « {id} ». Lancez cortiq-server pour le servir.",
|
||||
"import.jobs": "Téléchargements et conversions",
|
||||
"import.cancel": "Annuler",
|
||||
"import.cancelling": "Annulation — nettoyage du fichier incomplet…",
|
||||
"import.delete": "Supprimer",
|
||||
"import.deleteConfirm": "Supprimer ce modèle et son fichier .cmf converti du disque ?",
|
||||
"import.deleted": "Supprimé avec ses fichiers",
|
||||
"import.state.running": "en cours",
|
||||
"import.state.done": "terminé",
|
||||
"import.state.error": "erreur",
|
||||
"import.state.cancelled": "annulé",
|
||||
"import.featuredTitle": "Modèles .cmf prêts à l'emploi",
|
||||
"import.featuredSub": "Modèles .cmf déjà convertis du compte infosave sur HuggingFace — la liste est lue en direct, les nouveaux modèles apparaissent d'eux-mêmes.",
|
||||
"import.readyCmfBadge": ".cmf prêt",
|
||||
"import.directDownload": "Télécharger",
|
||||
"import.q.q8_2f": "8 bits à deux champs (𝒲×θ) — recommandé : meilleur rapport qualité/taille.",
|
||||
"import.q.q8_row": "8 bits par ligne — simple et robuste.",
|
||||
"import.q.q4": "4 bits par bloc — le plus compact, qualité moindre.",
|
||||
"import.q.q4_tiled": "4 bits en tuiles (Q4_TILED) — 1,66× plus rapide sur GPU Metal.",
|
||||
"import.q.vbit": "Variable 3–8 bits — budget de bits par expert.",
|
||||
"import.q.q1t": "Ternaire 1,58 bit (Q1T) — BitNet b1.58 avec valeurs aberrantes éparses.",
|
||||
"import.q.q1s": "1 bit épars (Q1S) — base 1 bit avec valeurs aberrantes holographiques.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — 1 bit avec diffusion d'erreur.",
|
||||
"import.q.q1": "1 bit (Q1) — format purement binaire.",
|
||||
"import.q.f16": "16 bits — sans quantification.",
|
||||
"import.q.f32": "32 bits — pleine précision (fichier volumineux).",
|
||||
"import.adv.title": "Paramètres avancés",
|
||||
"import.adv.auto": "auto",
|
||||
"import.adv.linearCore": "Cœur linéaire",
|
||||
"import.adv.linearHint": "Comment les couches d'attention linéaire sont reportées ou fusionnées.",
|
||||
"import.adv.nphase": "vmf_phase phases/tête",
|
||||
"import.adv.vbitShape": "Forme du budget VBIT",
|
||||
"import.adv.meanBits": "Bits moyens VBIT",
|
||||
"import.adv.shard": "Taille max. des shards (Go)",
|
||||
"import.adv.noShard": "pas de découpage",
|
||||
"import.adv.skipMtp": "ignorer la tête MTP",
|
||||
"import.adv.o1": "Attention O(1)",
|
||||
"import.adv.o1Hint": "Attention en flux à mémoire constante : le cache KV est libéré après le prefill, la mémoire ne croît donc pas avec de longs contextes. Les poids restent identiques octet pour octet — seule une indication de chargement est enregistrée ; vous pouvez revenir en arrière à tout moment (servir avec --o1 off). Validé sur Qwen3-0.6B : toutes couches ≈ ×1,18 de perplexité.",
|
||||
"import.adv.o1Off": "désactivé — attention exacte (par défaut)",
|
||||
"import.adv.o1All": "all — convertir chaque couche softmax",
|
||||
"import.adv.o1Deep": "deep N — seulement les N couches les plus profondes",
|
||||
"import.adv.o1Custom": "personnalisé — liste explicite de couches",
|
||||
"import.adv.o1DeepN": "N couches profondes",
|
||||
"import.adv.o1Layers": "Couches (séparées par des virgules)",
|
||||
"import.adv.o1mHint": "budget de landmarks (32 par défaut)",
|
||||
"import.adv.o1windowHint": "largeur de la fenêtre exacte (128 par défaut)",
|
||||
"import.adv.o1sinkHint": "clés sink exactes permanentes (4 par défaut)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "Configuration initiale",
|
||||
"onboarding.welcome": "Bienvenue dans Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Choisissez votre langue — vous pourrez la changer à tout moment dans l'en-tête.",
|
||||
"onboarding.next": "Suivant",
|
||||
"onboarding.back": "Retour",
|
||||
"onboarding.skip": "Passer pour l'instant",
|
||||
"onboarding.skipSetup": "Passer",
|
||||
"onboarding.finish": "Terminer la configuration",
|
||||
"onboarding.featuredTitle": "Modèles prêts à l'emploi (en un clic)",
|
||||
"onboarding.featuredSub": "Ce sont des modèles .cmf déjà convertis, publiés par infosave sur HuggingFace — il suffit de les télécharger, aucune conversion nécessaire. Choisissez-en un ou plusieurs pour démarrer immédiatement.",
|
||||
"onboarding.download": "Télécharger",
|
||||
"onboarding.hfTitle": "N'importe quel modèle HuggingFace — conversion automatique",
|
||||
"onboarding.hfSub": "Cherchez un modèle, choisissez une quantification, et nous le convertissons automatiquement en .cmf. Aucune modification manuelle de la configuration.",
|
||||
"onboarding.noJobs": "Aucun téléchargement pour l'instant.",
|
||||
"onboarding.doneTitle": "Tout est prêt",
|
||||
"onboarding.doneSub": "Les modèles téléchargés seront enregistrés automatiquement. Vous pourrez en ajouter d'autres dans Importer ou Modèles.",
|
||||
"onboarding.doneHint": "« Terminer la configuration » connecte chaque téléchargement achevé comme modèle local, démarre son serveur .cmf et y dirige le routage. Tout cela reste modifiable ensuite dans Modèles.",
|
||||
"onboarding.willRegister": "Seront connectés comme modèles locaux",
|
||||
"onboarding.nothingToRegister": "Aucun téléchargement terminé — rien à connecter. Revenez à l'étape précédente pour choisir un modèle.",
|
||||
"onboarding.registeredN": "{n} modèle(s) connecté(s) : {ids}",
|
||||
"onboarding.runtimeMissing": "Le runtime cortiq n'est pas installé — installez-le dans Paramètres → Modèles locaux pour les servir.",
|
||||
"onboarding.downloaded": "téléchargé",
|
||||
"onboarding.update": "Mettre à jour",
|
||||
"onboarding.deleteFile": "Supprimer le fichier .cmf du disque",
|
||||
"onboarding.confirmDelete": "Supprimer « {id} » ? Le fichier .cmf sera retiré du disque.",
|
||||
"onboarding.unregistered": "Également désenregistré : {ids}",
|
||||
};
|
||||
|
||||
Vendored
+54
-5
@@ -31,7 +31,7 @@ export default {
|
||||
"nav.routing": "Роутинг",
|
||||
"nav.protocols": "Протоколы",
|
||||
"nav.keys": "API-ключи",
|
||||
"nav.playground": "Playground",
|
||||
"nav.playground": "Песочница",
|
||||
"nav.settings": "Настройки",
|
||||
"nav.foot": "Шлюз · v{v}",
|
||||
"nav.billing": "Ключ роутера и оплата",
|
||||
@@ -80,7 +80,7 @@ export default {
|
||||
"dash.groupby.label": "по задаче",
|
||||
"dash.groupby.account": "по аккаунту",
|
||||
"dash.recent.title": "Последние запросы",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Playground.",
|
||||
"dash.recent.empty": "Запросов ещё нет. Попробуйте Песочницу.",
|
||||
"dash.col.time": "Время",
|
||||
"dash.col.model": "Модель",
|
||||
"dash.col.tier": "Уровень",
|
||||
@@ -138,6 +138,18 @@ export default {
|
||||
"models.form.apiKeyHint": "Хранится безопасно на шлюзе; больше не показывается.",
|
||||
"models.form.keyStored": "Ключ сохранён — оставьте пустым, чтобы не менять, или вставьте новый.",
|
||||
"models.form.keyFromEnv": "Используется ключ из переменной окружения.",
|
||||
"models.form.paramsTitle": "Параметры генерации, O(1) и режим размышления",
|
||||
"models.form.temperature": "Температура",
|
||||
"models.form.temperatureHint": "Температура генерации (0.0 — точные факты, 1.0+ — творчество).",
|
||||
"models.form.topP": "Top-P (Nucleus Sampling)",
|
||||
"models.form.topPHint": "Порог сэмплирования токенов (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Макс. токенов (Max Tokens)",
|
||||
"models.form.maxTokensHint": "Максимальное количество генерируемых токенов.",
|
||||
"models.form.thinkBudget": "Режим размышления / Think Budget",
|
||||
"models.form.thinkBudgetHint": "Бюджет токенов мыслей (0 = выключен, N > 0 = лимит рассуждений).",
|
||||
"models.form.systemPrompt": "Системный промпт по умолчанию",
|
||||
"models.form.systemPromptHint": "Автоматически подставляется, если запрос не содержит системное сообщение.",
|
||||
"models.form.systemPromptPlaceholder": "Вы — полезный ассистент...",
|
||||
"models.form.testKey": "Проверить и загрузить модели",
|
||||
"models.form.testing": "проверка…",
|
||||
"models.form.foundModels": "загружено моделей: {n}",
|
||||
@@ -207,7 +219,7 @@ export default {
|
||||
"keys.allowAll": "все",
|
||||
|
||||
// playground
|
||||
"pg.title": "Playground",
|
||||
"pg.title": "Песочница",
|
||||
"pg.subtitle": "Отправьте запрос через активный пайплайн и изучите решение роутера.",
|
||||
"pg.system": "Системный промпт",
|
||||
"pg.systemPlaceholder": "необязательное системное сообщение",
|
||||
@@ -266,7 +278,6 @@ export default {
|
||||
"settings.cmf.checkPort": "Проверить порт",
|
||||
"settings.cmf.portFree": "порт свободен",
|
||||
"settings.cmf.portBusy": "порт занят",
|
||||
"settings.cmf.portOwn": "занят текущим сервером шлюза — освободится после перезапуска, замена допустима",
|
||||
"settings.cmf.useSuggested": "занять {port}",
|
||||
"settings.cmf.serving": "работает: {model} на :{port}",
|
||||
"settings.cmf.intro": "Запускайте модели локально движком cortiq — работает даже офлайн или когда роутер недоступен.",
|
||||
@@ -374,7 +385,7 @@ export default {
|
||||
"import.failed": "Конвертация не удалась — смотрите лог.",
|
||||
"import.register": "Подключить как локальную модель",
|
||||
"import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
|
||||
"import.jobs": "Конвертации",
|
||||
"import.jobs": "Загрузки / Конвертации",
|
||||
"import.cancel": "Отменить",
|
||||
"import.cancelling": "Отмена — удаляю недописанный файл…",
|
||||
"import.delete": "Удалить",
|
||||
@@ -384,12 +395,22 @@ export default {
|
||||
"import.state.done": "готово",
|
||||
"import.state.error": "ошибка",
|
||||
"import.state.cancelled": "отменено",
|
||||
"import.featuredTitle": "Рекомендуемые готовые .cmf модели",
|
||||
"import.featuredSub": "Готовые .cmf модели из аккаунта infosave на HuggingFace — список читается вживую, новые модели появляются здесь сами.",
|
||||
"import.readyCmfBadge": "Готовая .cmf",
|
||||
"import.directDownload": "Скачать",
|
||||
"import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
|
||||
"import.q.q8_row": "8-бит построчно — просто, надёжно.",
|
||||
"import.q.q4": "4-бит блочно — компактнее, ниже качество.",
|
||||
"import.q.q4_tiled": "4-бит тайловый (Q4_TILED) — х1.66 быстрее на Metal GPU.",
|
||||
"import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
|
||||
"import.q.q1t": "1.58-бит тернарный (Q1T) — BitNet b1.58 с оутлайерами.",
|
||||
"import.q.q1s": "1-бит спарс (Q1S) — 1-бит база с голографическими оутлайерами.",
|
||||
"import.q.q1p": "1-бит PTQ (Q1P) — 1-бит с переносом ошибки (error diffusion).",
|
||||
"import.q.q1": "1-бит (Q1) — чистый бинарный формат.",
|
||||
"import.q.f16": "16-бит — без квантования.",
|
||||
"import.q.f32": "32-бит — полная точность (большой файл).",
|
||||
|
||||
"import.adv.title": "Расширенные настройки",
|
||||
"import.adv.auto": "авто",
|
||||
"import.adv.linearCore": "Линейное ядро",
|
||||
@@ -411,4 +432,32 @@ export default {
|
||||
"import.adv.o1mHint": "число опорных точек (по умолчанию 32)",
|
||||
"import.adv.o1windowHint": "ширина точного окна (по умолчанию 128)",
|
||||
"import.adv.o1sinkHint": "постоянные точные sink-ключи (по умолчанию 4)",
|
||||
|
||||
// онбординг (первый запуск)
|
||||
"onboarding.title": "Первоначальная настройка",
|
||||
"onboarding.welcome": "Добро пожаловать в Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "Выберите язык — его можно сменить в шапке в любой момент.",
|
||||
"onboarding.next": "Далее",
|
||||
"onboarding.back": "Назад",
|
||||
"onboarding.skip": "Пропустить",
|
||||
"onboarding.skipSetup": "Пропустить",
|
||||
"onboarding.finish": "Завершить",
|
||||
"onboarding.featuredTitle": "Готовые модели в один клик",
|
||||
"onboarding.featuredSub": "Это уже сконвертированные .cmf модели от infosave на HuggingFace — просто Скачать, без конвертации. Выберите одну или несколько для мгновенного старта.",
|
||||
"onboarding.download": "Скачать",
|
||||
"onboarding.hfTitle": "Любая модель с HuggingFace — автоконвертация",
|
||||
"onboarding.hfSub": "Найдите любую модель, выберите квантизацию — мы автоматически сконвертируем её в .cmf. Никаких ручных правок конфига.",
|
||||
"onboarding.noJobs": "Пока нет загрузок.",
|
||||
"onboarding.doneTitle": "Готово",
|
||||
"onboarding.doneSub": "Скачанные модели зарегистрируются автоматически. Добавить ещё можно во вкладке Импорт или Модели.",
|
||||
"onboarding.doneHint": "Кнопка «Завершить» подключит все скачанные модели как локальные, поднимет для них .cmf сервер и настроит роутинг. Всё это потом можно изменить во вкладке Модели.",
|
||||
"onboarding.willRegister": "Будут подключены как локальные модели",
|
||||
"onboarding.nothingToRegister": "Завершённых загрузок пока нет — подключать нечего. Вернитесь на шаг назад и выберите модель.",
|
||||
"onboarding.registeredN": "Подключено моделей: {n} — {ids}",
|
||||
"onboarding.runtimeMissing": "Среда cortiq не установлена — установите её в Настройки → Локальные модели, чтобы запускать эти модели.",
|
||||
"onboarding.downloaded": "скачана",
|
||||
"onboarding.update": "Обновить",
|
||||
"onboarding.deleteFile": "Удалить файл .cmf с диска",
|
||||
"onboarding.confirmDelete": "Удалить «{id}»? Файл .cmf будет удалён с диска.",
|
||||
"onboarding.unregistered": "Также отключено из моделей: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+188
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "Giriş TTL",
|
||||
"settings.cache.embed": "Gömme modeli",
|
||||
"settings.cache.note": "Anlamsal önbellek ayarları yeniden başlatmadan sonra geçerli olur.",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "Model içe aktar",
|
||||
|
||||
// dash
|
||||
"dash.clear": "Günlükleri temizle",
|
||||
"dash.clearConfirm": "Kaydedilmiş tüm istatistikler ve günlükler silinsin mi? Bu işlem geri alınamaz.",
|
||||
"dash.cleared": "Günlükler temizlendi",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "Bir id ya da model adı girin",
|
||||
"models.form.apiKey": "API anahtarı",
|
||||
"models.form.apiKeyPlaceholder": "API anahtarınızı yapıştırın",
|
||||
"models.form.apiKeyNotNeeded": "gerekmiyor (yerel sunucu)",
|
||||
"models.form.apiKeyHint": "Ağ geçidinde güvenle saklanır; bir daha gösterilmez.",
|
||||
"models.form.keyStored": "Kayıtlı bir anahtar var — korumak için boş bırakın ya da yenisini yapıştırın.",
|
||||
"models.form.keyFromEnv": "Ortam değişkenindeki anahtar kullanılıyor.",
|
||||
"models.form.paramsTitle": "Üretim ve düşünme parametreleri (O(1), MTP, sıcaklık)",
|
||||
"models.form.temperature": "Sıcaklık",
|
||||
"models.form.temperatureHint": "Üretim sıcaklığı (0.0 = belirlenimci, 1.0+ = yaratıcı).",
|
||||
"models.form.topP": "Top-P (nucleus örnekleme)",
|
||||
"models.form.topPHint": "Örnekleme eşiği (0.0 .. 1.0).",
|
||||
"models.form.maxTokens": "Maks. token",
|
||||
"models.form.maxTokensHint": "Üretilecek en fazla token sayısı.",
|
||||
"models.form.thinkBudget": "Düşünme modu / düşünme bütçesi",
|
||||
"models.form.thinkBudgetHint": "Akıl yürütme token bütçesi (0 = kapalı, N > 0 = en fazla akıl yürütme tokeni).",
|
||||
"models.form.systemPrompt": "Varsayılan sistem istemi",
|
||||
"models.form.systemPromptHint": "İstekte sistem istemi yoksa otomatik olarak başa eklenir.",
|
||||
"models.form.systemPromptPlaceholder": "Yardımsever bir asistansın…",
|
||||
"models.form.testKey": "Test et ve modelleri yükle",
|
||||
"models.form.testing": "kontrol ediliyor…",
|
||||
"models.form.foundModels": "{n} model yüklendi",
|
||||
"models.form.modelPlaceholder": "aramak için yazın…",
|
||||
"models.form.modelHint": "Anahtarınızla listeyi çekmek için “Test et ve modelleri yükle”ye tıklayın, sonra birini seçin.",
|
||||
"models.managed": "yönetilen",
|
||||
"models.managedHint": "Yönetilen CMF sunucusu tarafından sağlanır (Ayarlar → Yerel modeller). Yapılandırmayı orada yapın.",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "Yönlendiriciyi kullan (kapalı = yalnızca yerel modeller)",
|
||||
"settings.cmf": "Yerel modeller (CMF)",
|
||||
"settings.cmf.localOnly": "Yalnızca yerel modelleri kullan",
|
||||
"settings.cmf.manage": "Yerel model sunucusu çalıştır",
|
||||
"settings.cmf.autoInstall": "cortiq'i otomatik kur (crates.io)",
|
||||
"settings.cmf.autoUpdate": "cortiq'i otomatik güncelle",
|
||||
"settings.cmf.model": "Yerel .cmf model yolu",
|
||||
"settings.cmf.port": "Yerel sunucu portu",
|
||||
"settings.cmf.modelId": "Havuzdaki model id'si",
|
||||
"settings.cmf.note": "Yerel bir .cmf dosyasını cortiq ile sunar (github.com/infosave2007/cmf). Yönlendirici erişilemez olsa bile çalışır.",
|
||||
"settings.cmf.threads": "İş parçacıkları",
|
||||
"settings.cmf.gpu": "GPU kullan (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "İş parçacıkları çözme aşamasındaki matris-vektör çarpımlarını paralelleştirir (~3× hızlı; 0 = çalışma zamanı varsayılanı). GPU'ya aktarım daha çok uzun istemlerin prefill aşamasına yarar — çözme bellek bant genişliğine bağlı olduğundan Apple Silicon'da etkisi yaklaşık nötrdür.",
|
||||
"settings.cmf.pick": "bir model seçin",
|
||||
"settings.cmf.models": "Yerel modeller",
|
||||
"settings.cmf.modelsHelp": "Her satır, ağ geçidinin çalıştırdığı (kendi portunda kendi cortiq serve süreci olan) ve havuza kaydettiği bir modeldir. Kaç .cmf dosyanız ve boş portunuz varsa o kadar ekleyebilirsiniz.",
|
||||
"settings.cmf.rowHint": "id · .cmf dosyası · port · iş parçacığı · GPU. Değişiklikler yeniden başlatınca geçerli olur; satırı silmek o sunucuyu durdurur.",
|
||||
"settings.cmf.addServer": "+ Yerel model ekle",
|
||||
"settings.cmf.checkPort": "Portu kontrol et",
|
||||
"settings.cmf.portFree": "port boş",
|
||||
"settings.cmf.portBusy": "port kullanımda",
|
||||
"settings.cmf.useSuggested": "{port} kullan",
|
||||
"settings.cmf.serving": "{model} :{port} üzerinde sunuluyor",
|
||||
"settings.cmf.intro": "Modelleri cortiq motoruyla yerelde çalıştırın — çevrimdışıyken ya da yönlendirici çöktüğünde bile çalışır.",
|
||||
"settings.cmf.runtime": "cortiq çalışma zamanı",
|
||||
"settings.cmf.runtimeHelp": ".cmf modellerini yükleyip çalıştıran `cortiq` komut satırı aracı. Yerel sunucuyu başlatmadan önce bir kez kurun.",
|
||||
"settings.cmf.install": "cortiq'i kur",
|
||||
"settings.cmf.reinstall": "Yeniden kur / güncelle",
|
||||
"settings.cmf.installing": "kuruluyor… (derleniyor, birkaç dakika)",
|
||||
"settings.cmf.installedOk": "cortiq kuruldu",
|
||||
"settings.cmf.notInstalled": "kurulu değil",
|
||||
"settings.cmf.addModel": "Model ekle",
|
||||
"settings.cmf.addModelHelp": "Modeller “Model içe aktar” sekmesinde indirilip dönüştürülür (HuggingFace → .cmf). Sonra aşağıdaki yerel sunucuyu o .cmf dosyasına yönlendirin.",
|
||||
"settings.cmf.importNote": "Henüz model yok mu? Bir tane içe aktarın:",
|
||||
"settings.cmf.importLink": "İçe aktarmayı aç",
|
||||
"settings.cmf.modelHelp": "Sunulacak .cmf dosyasının yolu (ör. models/qwen.cmf), içe aktarma sekmesinde üretilir. Farklı görevler için birden çok model mi? Hepsini içe aktarıp bu yolu değiştirin.",
|
||||
"settings.cmf.portHelp": "Yerel model sunucusunun dinlediği TCP portu (varsayılan 8081). Yalnızca başka bir servisle çakışırsa değiştirin.",
|
||||
"settings.cmf.threadsHelp": "Üretim için kaç CPU çekirdeği kullanılacağı. Daha fazlası = daha hızlı (tek çekirdeğe göre ~3×); 0 seçimi çalışma zamanına bırakır.",
|
||||
"settings.cmf.gpuHelp": "Matris hesaplarını GPU'ya aktarır (Metal/wgpu). Apple Silicon'da çoğunlukla uzun istemleri hızlandırır — çözme bellek bant genişliğine bağlıdır.",
|
||||
"settings.cmf.localOnlyHelp": "Her isteği yerel modele yönlendirir; yönlendiriciyi ve tüm bulut sağlayıcılarını yok sayar.",
|
||||
"settings.cmf.manageHelp": "Ağ geçidi açılışta aşağıdaki .cmf için `cortiq serve` başlatır ve istekleri oraya yönlendirir. cortiq'in kurulu olması gerekir (adım 1). Yeniden başlatınca geçerli olur.",
|
||||
"settings.cmf.autoInstallHelp": "Açılışta cortiq yoksa crates.io üzerinden otomatik kurar.",
|
||||
"settings.cmf.autoUpdateHelp": "Açılışta daha yeni bir sürüm varsa cortiq'i en son sürüme yükseltir.",
|
||||
"settings.router.enabledHelp": "Her isteği cortiq yönlendiricisi sınıflandırsın (görev türü + karmaşıklık) ve model seçsin. Kapalı = yalnızca yerel yönlendirme tablosunu kullan.",
|
||||
"settings.router.verifyTlsHelp": "Yönlendiricinin TLS sertifikasını doğrula. Yalnızca kendinden imzalı ya da IP'ye bağlı sertifikada kapatın.",
|
||||
"settings.telemetry.metricsHelp": "Prometheus metriklerini GET /metrics adresinde yayınla.",
|
||||
"settings.cortiq.echoHelp": "Her yanıt gövdesine `cortiq` alanı ekle (görev, model, maliyet). Yanıt başlıkları her zaman gönderilir.",
|
||||
"settings.stats.enabledHelp": "Panoyu ve grafikleri beslemek için son istekleri kaydet.",
|
||||
"settings.cache.enabledHelp": "Anlamsal önbellek: yeni bir istem geçmiştekine çok benziyorsa kaydedilmiş yanıtı döndürür ve model çağrısını atlar. Havuzda bir gömme (embedding) modeli gerekir.",
|
||||
"settings.log.bodiesHelp": "İstek/yanıt mesaj gövdelerinin tamamını günlüğe yaz. Gizlilik için varsayılan olarak kapalı.",
|
||||
"settings.router.saveKey": "Anahtarı kaydet",
|
||||
"settings.router.keySaved": "Yönlendirici anahtarı kaydedildi",
|
||||
"settings.router.keyEmpty": "Önce anahtarı yapıştırın",
|
||||
"settings.router.secretHelp": "api.allaigate.com e-postanızdaki anahtarı yapıştırıp Anahtarı kaydet'e basın. Bu bir “API anahtarları” değeri DEĞİLDİR — onlar ağ geçidinizi çağıran istemciler içindir.",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — anahtar kayıtlı (değiştirmek için yenisini yapıştırın)",
|
||||
"settings.route.strategyHelp": "Sınıflandırma için yönlendiriciye gönderilen metin: last_user = yalnızca son kullanıcı mesajı; last_user_plus_system = son kullanıcı mesajı + sistem istemi; concat_all = tüm konuşma.",
|
||||
"settings.route.maxCharsHelp": "Bu metnin yönlendiriciye gönderilen en fazla karakter sayısı (fazlası kırpılır). Daha fazlası = daha isabetli sınıflandırma, ama daha yavaş ve pahalı yönlendirici çağrıları.",
|
||||
"settings.route.cacheTtlHelp": "Aynı istek için yönlendirme kararının, yönlendiriciye yeniden sorulmadan önce ne kadar önbellekte tutulacağı (ör. 60s, 5m).",
|
||||
"settings.route.profileHelp": "İstek belirtmediğinde kullanılacak varsayılan yönlendirme profili: cost-saver (en ucuz), balanced ya da quality-first (en iyi model). Yönlendiricinin hangi modeli seçeceğini etkiler.",
|
||||
|
||||
// import
|
||||
"import.subtitle": "HuggingFace'te arayın, yerel bir .cmf dosyasına dönüştürün ve sunun.",
|
||||
"import.searchPlaceholder": "HuggingFace modellerinde ara (ör. qwen3, llama)…",
|
||||
"import.searching": "Aranıyor…",
|
||||
"import.noResults": "Model bulunamadı.",
|
||||
"import.configure": "Dönüştürmeyi yapılandır",
|
||||
"import.outName": "Çıktı adı",
|
||||
"import.outNameHint": ".cmf dosya adı (harf, rakam, - ve _).",
|
||||
"import.quant": "Niceleme",
|
||||
"import.convert": "Dönüştür ve yükle",
|
||||
"import.converting": "Dönüştürülüyor",
|
||||
"import.started": "Dönüştürme başladı",
|
||||
"import.failed": "Dönüştürme başarısız — günlüğe bakın.",
|
||||
"import.register": "Yerel model olarak kaydet",
|
||||
"import.registered": "“{id}” olarak kaydedildi. Sunmak için cortiq-server'ı çalıştırın.",
|
||||
"import.jobs": "İndirmeler ve dönüştürmeler",
|
||||
"import.cancel": "İptal",
|
||||
"import.cancelling": "İptal ediliyor — yarım kalan dosya temizleniyor…",
|
||||
"import.delete": "Sil",
|
||||
"import.deleteConfirm": "Bu model ve dönüştürülmüş .cmf dosyası diskten silinsin mi?",
|
||||
"import.deleted": "Dosyalarıyla birlikte silindi",
|
||||
"import.state.running": "çalışıyor",
|
||||
"import.state.done": "tamam",
|
||||
"import.state.error": "hata",
|
||||
"import.state.cancelled": "iptal edildi",
|
||||
"import.featuredTitle": "Öne çıkan hazır .cmf modelleri",
|
||||
"import.featuredSub": "HuggingFace'teki infosave hesabından hazır .cmf modelleri — liste canlı okunur, yeni modeller kendiliğinden görünür.",
|
||||
"import.readyCmfBadge": "Hazır .cmf",
|
||||
"import.directDownload": "İndir",
|
||||
"import.q.q8_2f": "8 bit çift alan (𝒲×θ) — önerilen: en iyi kalite/boyut dengesi.",
|
||||
"import.q.q8_row": "Satır başına 8 bit — basit ve sağlam.",
|
||||
"import.q.q4": "4 bit blok — en küçük, kalite daha düşük.",
|
||||
"import.q.q4_tiled": "4 bit döşeli (Q4_TILED) — Metal GPU'da 1,66× daha hızlı.",
|
||||
"import.q.vbit": "Değişken 3–8 bit — uzman başına bit bütçesi.",
|
||||
"import.q.q1t": "1,58 bit üçlü (Q1T) — seyrek aykırı değerlerle BitNet b1.58.",
|
||||
"import.q.q1s": "1 bit seyrek (Q1S) — holografik aykırı değerlerle 1 bit taban.",
|
||||
"import.q.q1p": "1 bit PTQ (Q1P) — hata yayılımlı 1 bit.",
|
||||
"import.q.q1": "1 bit (Q1) — saf ikili biçim.",
|
||||
"import.q.f16": "16 bit — niceleme yok.",
|
||||
"import.q.f32": "32 bit — tam hassasiyet (büyük dosya).",
|
||||
"import.adv.title": "Gelişmiş ayarlar",
|
||||
"import.adv.auto": "otomatik",
|
||||
"import.adv.linearCore": "Doğrusal çekirdek",
|
||||
"import.adv.linearHint": "Doğrusal dikkat katmanlarının nasıl taşındığı/katlandığı.",
|
||||
"import.adv.nphase": "vmf_phase faz/başlık",
|
||||
"import.adv.vbitShape": "VBIT bütçe biçimi",
|
||||
"import.adv.meanBits": "VBIT ortalama bit",
|
||||
"import.adv.shard": "Parça üst sınırı (GB)",
|
||||
"import.adv.noShard": "parçalama yok",
|
||||
"import.adv.skipMtp": "MTP başlığını atla",
|
||||
"import.adv.o1": "O(1) dikkat",
|
||||
"import.adv.o1Hint": "Sabit bellekli akış dikkati: KV önbelleği prefill sonrası serbest bırakılır, böylece uzun bağlamlar belleği büyütmez. Ağırlıklar bayt bayt aynı kalır — yalnızca yükleme anına dair bir ipucu kaydedilir; istediğiniz zaman geri dönebilirsiniz (--o1 off ile sunun). Qwen3-0.6B üzerinde doğrulandı: tüm katmanlar ≈ ×1,18 şaşkınlık (perplexity).",
|
||||
"import.adv.o1Off": "kapalı — tam dikkat (varsayılan)",
|
||||
"import.adv.o1All": "all — her softmax katmanını dönüştür",
|
||||
"import.adv.o1Deep": "deep N — yalnızca en derin N katman",
|
||||
"import.adv.o1Custom": "özel — açık katman listesi",
|
||||
"import.adv.o1DeepN": "N derin katman",
|
||||
"import.adv.o1Layers": "Katmanlar (virgülle ayrılmış)",
|
||||
"import.adv.o1mHint": "landmark bütçesi (varsayılan 32)",
|
||||
"import.adv.o1windowHint": "tam pencere genişliği (varsayılan 128)",
|
||||
"import.adv.o1sinkHint": "kalıcı tam sink anahtarları (varsayılan 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "İlk kurulum",
|
||||
"onboarding.welcome": "Cortiq Gateway'e hoş geldiniz",
|
||||
"onboarding.welcomeSub": "Dilinizi seçin — üst çubuktan istediğiniz zaman değiştirebilirsiniz.",
|
||||
"onboarding.next": "İleri",
|
||||
"onboarding.back": "Geri",
|
||||
"onboarding.skip": "Şimdilik atla",
|
||||
"onboarding.skipSetup": "Atla",
|
||||
"onboarding.finish": "Kurulumu tamamla",
|
||||
"onboarding.featuredTitle": "Hazır modeller (tek tık)",
|
||||
"onboarding.featuredSub": "Bunlar HuggingFace'te infosave tarafından yayınlanan, önceden dönüştürülmüş .cmf modelleridir — sadece indirin, dönüştürme gerekmez. Hemen başlamak için bir ya da birkaçını seçin.",
|
||||
"onboarding.download": "İndir",
|
||||
"onboarding.hfTitle": "Herhangi bir HuggingFace modeli — otomatik dönüştürme",
|
||||
"onboarding.hfSub": "İstediğiniz modeli arayın, bir niceleme seçin; biz onu otomatik olarak .cmf biçimine dönüştürelim. Elle yapılandırma düzenlemesi yok.",
|
||||
"onboarding.noJobs": "Henüz indirme yok.",
|
||||
"onboarding.doneTitle": "Her şey hazır",
|
||||
"onboarding.doneSub": "İndirdiğiniz modeller otomatik olarak kaydedilir. Daha fazlasını İçe aktar ya da Modeller sekmesinden ekleyebilirsiniz.",
|
||||
"onboarding.doneHint": "“Kurulumu tamamla”, biten her indirmeyi yerel model olarak bağlar, .cmf sunucusunu başlatır ve yönlendirmeyi ona çevirir. Tümünü sonradan Modeller sekmesinden değiştirebilirsiniz.",
|
||||
"onboarding.willRegister": "Yerel model olarak bağlanacak",
|
||||
"onboarding.nothingToRegister": "Henüz tamamlanmış indirme yok — bağlanacak bir şey bulunmuyor. Bir adım geri dönüp model seçin.",
|
||||
"onboarding.registeredN": "{n} model bağlandı: {ids}",
|
||||
"onboarding.runtimeMissing": "cortiq çalışma zamanı kurulu değil — bunları sunabilmek için Ayarlar → Yerel modeller bölümünden kurun.",
|
||||
"onboarding.downloaded": "indirildi",
|
||||
"onboarding.update": "Güncelle",
|
||||
"onboarding.deleteFile": ".cmf dosyasını diskten sil",
|
||||
"onboarding.confirmDelete": "“{id}” silinsin mi? .cmf dosyası diskten kaldırılacak.",
|
||||
"onboarding.unregistered": "Ayrıca kaydı kaldırıldı: {ids}",
|
||||
};
|
||||
|
||||
Vendored
+188
@@ -280,4 +280,192 @@ export default {
|
||||
"settings.cache.ttl": "条目 TTL",
|
||||
"settings.cache.embed": "嵌入模型",
|
||||
"settings.cache.note": "缓存设置将在重启后生效。",
|
||||
|
||||
// ── translated from en.js (previously falling back to English) ──
|
||||
// nav
|
||||
"nav.import": "导入模型",
|
||||
|
||||
// dash
|
||||
"dash.clear": "清空日志",
|
||||
"dash.clearConfirm": "清空所有已记录的统计与日志?此操作无法撤销。",
|
||||
"dash.cleared": "日志已清空",
|
||||
|
||||
// models
|
||||
"models.form.idOrModel": "请输入 id 或模型名称",
|
||||
"models.form.apiKey": "API 密钥",
|
||||
"models.form.apiKeyPlaceholder": "粘贴你的 API 密钥",
|
||||
"models.form.apiKeyNotNeeded": "无需密钥(本地服务器)",
|
||||
"models.form.apiKeyHint": "安全保存在网关中,之后不会再次显示。",
|
||||
"models.form.keyStored": "已保存一个密钥 — 留空表示保留,或粘贴新的密钥替换。",
|
||||
"models.form.keyFromEnv": "正在使用环境变量中的密钥。",
|
||||
"models.form.paramsTitle": "生成与思考参数(O(1)、MTP、温度)",
|
||||
"models.form.temperature": "温度",
|
||||
"models.form.temperatureHint": "生成温度(0.0 = 确定性,1.0+ = 更有创造性)。",
|
||||
"models.form.topP": "Top-P(核采样)",
|
||||
"models.form.topPHint": "采样阈值(0.0 .. 1.0)。",
|
||||
"models.form.maxTokens": "最大 token 数",
|
||||
"models.form.maxTokensHint": "生成内容的最大 token 数。",
|
||||
"models.form.thinkBudget": "思考模式 / 思考预算",
|
||||
"models.form.thinkBudgetHint": "推理 token 预算(0 = 关闭,N > 0 = 推理 token 上限)。",
|
||||
"models.form.systemPrompt": "默认系统提示词",
|
||||
"models.form.systemPromptHint": "当请求中没有系统提示词时自动添加到前面。",
|
||||
"models.form.systemPromptPlaceholder": "你是一个乐于助人的助手……",
|
||||
"models.form.testKey": "测试并加载模型",
|
||||
"models.form.testing": "检查中……",
|
||||
"models.form.foundModels": "已加载 {n} 个模型",
|
||||
"models.form.modelPlaceholder": "输入以搜索……",
|
||||
"models.form.modelHint": "点击“测试并加载模型”用你的密钥拉取列表,然后选择一个。",
|
||||
"models.managed": "托管",
|
||||
"models.managedHint": "由托管的 CMF 服务器提供(设置 → 本地模型)。请在那里配置。",
|
||||
|
||||
// settings
|
||||
"settings.router.enabled": "使用路由器(关闭 = 仅本地模型)",
|
||||
"settings.cmf": "本地模型(CMF)",
|
||||
"settings.cmf.localOnly": "仅使用本地模型",
|
||||
"settings.cmf.manage": "运行本地模型服务器",
|
||||
"settings.cmf.autoInstall": "自动安装 cortiq(crates.io)",
|
||||
"settings.cmf.autoUpdate": "自动更新 cortiq",
|
||||
"settings.cmf.model": "本地 .cmf 模型路径",
|
||||
"settings.cmf.port": "本地服务器端口",
|
||||
"settings.cmf.modelId": "模型在池中的 id",
|
||||
"settings.cmf.note": "通过 cortiq 提供本地 .cmf 服务(github.com/infosave2007/cmf)。即使路由器不可用也能工作。",
|
||||
"settings.cmf.threads": "工作线程",
|
||||
"settings.cmf.gpu": "使用 GPU(Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "线程会并行化解码阶段的矩阵向量乘法(约快 3 倍;0 = 运行时默认值)。GPU 卸载主要加速长提示词的 prefill — 解码受内存带宽限制,因此在 Apple Silicon 上基本持平。",
|
||||
"settings.cmf.pick": "选择一个模型",
|
||||
"settings.cmf.models": "本地模型",
|
||||
"settings.cmf.modelsHelp": "每一行都是网关运行的一个模型(各自独立的 cortiq serve,各占一个端口)并注册到模型池。有多少 .cmf 文件和空闲端口就可以加多少行。",
|
||||
"settings.cmf.rowHint": "id · .cmf 文件 · 端口 · 线程 · GPU。改动在重启后生效;删除一行会停止对应的服务器。",
|
||||
"settings.cmf.addServer": "+ 添加本地模型",
|
||||
"settings.cmf.checkPort": "检查端口",
|
||||
"settings.cmf.portFree": "端口空闲",
|
||||
"settings.cmf.portBusy": "端口被占用",
|
||||
"settings.cmf.useSuggested": "使用 {port}",
|
||||
"settings.cmf.serving": "{model} 正在 :{port} 提供服务",
|
||||
"settings.cmf.intro": "用 cortiq 引擎在本地运行模型 — 即使离线或路由器故障也能工作。",
|
||||
"settings.cmf.runtime": "cortiq 运行时",
|
||||
"settings.cmf.runtimeHelp": "用于加载和运行 .cmf 模型的命令行工具 `cortiq`。启动本地服务器前先安装一次。",
|
||||
"settings.cmf.install": "安装 cortiq",
|
||||
"settings.cmf.reinstall": "重新安装 / 更新",
|
||||
"settings.cmf.installing": "安装中……(需要编译,几分钟)",
|
||||
"settings.cmf.installedOk": "cortiq 已安装",
|
||||
"settings.cmf.notInstalled": "未安装",
|
||||
"settings.cmf.addModel": "添加模型",
|
||||
"settings.cmf.addModelHelp": "模型在“导入模型”标签页下载并转换(HuggingFace → .cmf)。然后把下面的本地服务器指向该 .cmf 文件。",
|
||||
"settings.cmf.importNote": "还没有模型?去导入一个:",
|
||||
"settings.cmf.importLink": "打开导入",
|
||||
"settings.cmf.modelHelp": "要提供服务的 .cmf 文件路径(例如 models/qwen.cmf),由导入标签页生成。要为不同任务准备多个模型?导入它们并切换此路径。",
|
||||
"settings.cmf.portHelp": "本地模型服务器监听的 TCP 端口(默认 8081)。仅在与其他服务冲突时修改。",
|
||||
"settings.cmf.threadsHelp": "生成时使用多少个 CPU 核心。越多越快(相比单核约快 3 倍);0 表示交给运行时决定。",
|
||||
"settings.cmf.gpuHelp": "把矩阵运算卸载到 GPU(Metal/wgpu)。在 Apple Silicon 上主要加速长提示词 — 解码受内存带宽限制。",
|
||||
"settings.cmf.localOnlyHelp": "把每个请求都发往本地模型,忽略路由器和所有云端供应商。",
|
||||
"settings.cmf.manageHelp": "网关启动时会为下面的 .cmf 启动 `cortiq serve` 并把请求路由过去。需要已安装 cortiq(第 1 步)。重启后生效。",
|
||||
"settings.cmf.autoInstallHelp": "如果启动时缺少 cortiq,自动从 crates.io 安装。",
|
||||
"settings.cmf.autoUpdateHelp": "启动时如有更新版本,将 cortiq 升级到最新发布版。",
|
||||
"settings.router.enabledHelp": "让 cortiq 路由器对每个请求分类(任务类型 + 复杂度)并挑选模型。关闭 = 仅使用本地路由表。",
|
||||
"settings.router.verifyTlsHelp": "校验路由器的 TLS 证书。仅在使用自签名或绑定 IP 的证书时关闭。",
|
||||
"settings.telemetry.metricsHelp": "在 GET /metrics 暴露 Prometheus 指标。",
|
||||
"settings.cortiq.echoHelp": "在每个响应体中加入 `cortiq` 字段(任务、模型、成本)。响应头始终会发送。",
|
||||
"settings.stats.enabledHelp": "记录最近的请求,为仪表盘和图表提供数据。",
|
||||
"settings.cache.enabledHelp": "语义缓存:当新提示词与过去的某个非常相似时直接返回已保存的回答,跳过模型调用。需要池中有一个嵌入模型。",
|
||||
"settings.log.bodiesHelp": "记录请求/响应的完整消息体。出于隐私考虑默认关闭。",
|
||||
"settings.router.saveKey": "保存密钥",
|
||||
"settings.router.keySaved": "路由器密钥已保存",
|
||||
"settings.router.keyEmpty": "请先粘贴密钥",
|
||||
"settings.router.secretHelp": "粘贴 api.allaigate.com 邮件中的密钥,然后点击“保存密钥”。这不是“API 密钥”页中的值 — 那些是给调用你网关的客户端用的。",
|
||||
"settings.router.keyStoredMask": "•••••••••••• — 密钥已保存(再次粘贴可替换)",
|
||||
"settings.route.strategyHelp": "发送给路由器用于分类的文本:last_user = 仅最后一条用户消息;last_user_plus_system = 最后一条用户消息 + 系统提示词;concat_all = 整段对话。",
|
||||
"settings.route.maxCharsHelp": "发送给路由器的文本最大字符数(超出会截断)。越多分类越准,但路由器调用更慢也更贵。",
|
||||
"settings.route.cacheTtlHelp": "相同请求的路由决策缓存多久后才再次询问路由器(例如 60s、5m)。",
|
||||
"settings.route.profileHelp": "请求未指定时使用的默认路由档位:cost-saver(最省钱)、balanced 或 quality-first(最好的模型)。会影响路由器挑选哪个模型。",
|
||||
|
||||
// import
|
||||
"import.subtitle": "在 HuggingFace 上搜索模型,转换为本地 .cmf 并提供服务。",
|
||||
"import.searchPlaceholder": "搜索 HuggingFace 模型(例如 qwen3、llama)……",
|
||||
"import.searching": "搜索中……",
|
||||
"import.noResults": "未找到模型。",
|
||||
"import.configure": "配置转换",
|
||||
"import.outName": "输出名称",
|
||||
"import.outNameHint": ".cmf 文件名(字母、数字、- 和 _)。",
|
||||
"import.quant": "量化",
|
||||
"import.convert": "转换并加载",
|
||||
"import.converting": "转换中",
|
||||
"import.started": "转换已开始",
|
||||
"import.failed": "转换失败 — 请查看日志。",
|
||||
"import.register": "注册为本地模型",
|
||||
"import.registered": "已注册为 “{id}”。运行 cortiq-server 即可提供服务。",
|
||||
"import.jobs": "下载与转换",
|
||||
"import.cancel": "取消",
|
||||
"import.cancelling": "正在取消 — 清理未完成的文件……",
|
||||
"import.delete": "删除",
|
||||
"import.deleteConfirm": "从磁盘删除该模型及其转换好的 .cmf 文件?",
|
||||
"import.deleted": "已连同文件一起删除",
|
||||
"import.state.running": "进行中",
|
||||
"import.state.done": "完成",
|
||||
"import.state.error": "错误",
|
||||
"import.state.cancelled": "已取消",
|
||||
"import.featuredTitle": "推荐的开箱即用 .cmf 模型",
|
||||
"import.featuredSub": "来自 HuggingFace 上 infosave 账号的预转换 .cmf 模型 — 列表实时读取,新模型会自动出现。",
|
||||
"import.readyCmfBadge": "现成 .cmf",
|
||||
"import.directDownload": "下载",
|
||||
"import.q.q8_2f": "8 位双场(𝒲×θ)— 推荐:质量与体积的最佳平衡。",
|
||||
"import.q.q8_row": "8 位按行 — 简单稳健。",
|
||||
"import.q.q4": "4 位分块 — 体积最小,质量较低。",
|
||||
"import.q.q4_tiled": "4 位分块平铺(Q4_TILED)— 在 Metal GPU 上快 1.66 倍。",
|
||||
"import.q.vbit": "可变 3–8 位 — 按专家分配位宽预算。",
|
||||
"import.q.q1t": "1.58 位三值(Q1T)— BitNet b1.58,带稀疏离群值。",
|
||||
"import.q.q1s": "1 位稀疏(Q1S)— 1 位基底,带全息离群值。",
|
||||
"import.q.q1p": "1 位 PTQ(Q1P)— 1 位,带误差扩散。",
|
||||
"import.q.q1": "1 位(Q1)— 纯二值格式。",
|
||||
"import.q.f16": "16 位 — 不量化。",
|
||||
"import.q.f32": "32 位 — 全精度(文件很大)。",
|
||||
"import.adv.title": "高级设置",
|
||||
"import.adv.auto": "自动",
|
||||
"import.adv.linearCore": "线性核",
|
||||
"import.adv.linearHint": "线性注意力层如何被保留或折叠。",
|
||||
"import.adv.nphase": "vmf_phase 每头相位数",
|
||||
"import.adv.vbitShape": "VBIT 预算形状",
|
||||
"import.adv.meanBits": "VBIT 平均位宽",
|
||||
"import.adv.shard": "分片上限(GB)",
|
||||
"import.adv.noShard": "不分片",
|
||||
"import.adv.skipMtp": "跳过 MTP 头",
|
||||
"import.adv.o1": "O(1) 注意力",
|
||||
"import.adv.o1Hint": "常量内存的流式注意力:prefill 之后释放 KV 缓存,因此长上下文不会持续增加内存。权重保持逐字节一致 — 只记录一个加载时的提示;随时可以切回(启动时加 --o1 off)。已在 Qwen3-0.6B 上验证:全部层约为 ×1.18 困惑度。",
|
||||
"import.adv.o1Off": "关闭 — 精确注意力(默认)",
|
||||
"import.adv.o1All": "all — 转换所有 softmax 层",
|
||||
"import.adv.o1Deep": "deep N — 仅最深的 N 层",
|
||||
"import.adv.o1Custom": "自定义 — 显式指定层列表",
|
||||
"import.adv.o1DeepN": "最深的 N 层",
|
||||
"import.adv.o1Layers": "层(用逗号分隔)",
|
||||
"import.adv.o1mHint": "landmark 预算(默认 32)",
|
||||
"import.adv.o1windowHint": "精确窗口宽度(默认 128)",
|
||||
"import.adv.o1sinkHint": "常驻精确 sink key 数(默认 4)",
|
||||
|
||||
// onboarding
|
||||
"onboarding.title": "初始设置",
|
||||
"onboarding.welcome": "欢迎使用 Cortiq Gateway",
|
||||
"onboarding.welcomeSub": "选择你的语言 — 随时可以在顶栏更改。",
|
||||
"onboarding.next": "下一步",
|
||||
"onboarding.back": "上一步",
|
||||
"onboarding.skip": "暂时跳过",
|
||||
"onboarding.skipSetup": "跳过",
|
||||
"onboarding.finish": "完成设置",
|
||||
"onboarding.featuredTitle": "开箱即用的模型(一键)",
|
||||
"onboarding.featuredSub": "这些是 HuggingFace 上 infosave 发布的、已经转换好的 .cmf 模型 — 直接下载即可,无需转换。选择一个或多个即可立即开始。",
|
||||
"onboarding.download": "下载",
|
||||
"onboarding.hfTitle": "任意 HuggingFace 模型 — 自动转换",
|
||||
"onboarding.hfSub": "搜索任意模型,选择量化方式,我们会自动把它转换为 .cmf。无需手动修改配置。",
|
||||
"onboarding.noJobs": "还没有下载任务。",
|
||||
"onboarding.doneTitle": "全部就绪",
|
||||
"onboarding.doneSub": "下载好的模型会自动注册。之后可以在“导入模型”或“模型”中添加更多。",
|
||||
"onboarding.doneHint": "“完成设置”会把每个下载完成的模型接入为本地模型,启动它的 .cmf 服务器并把路由指向它。这些之后都可以在“模型”中修改。",
|
||||
"onboarding.willRegister": "将接入为本地模型",
|
||||
"onboarding.nothingToRegister": "还没有下载完成的模型 — 没有可接入的内容。返回上一步选一个模型。",
|
||||
"onboarding.registeredN": "已接入 {n} 个模型:{ids}",
|
||||
"onboarding.runtimeMissing": "cortiq 运行时尚未安装 — 请在 设置 → 本地模型 中安装后才能运行这些模型。",
|
||||
"onboarding.downloaded": "已下载",
|
||||
"onboarding.update": "更新",
|
||||
"onboarding.deleteFile": "从磁盘删除该 .cmf 文件",
|
||||
"onboarding.confirmDelete": "删除 “{id}”?该 .cmf 文件将从磁盘移除。",
|
||||
"onboarding.unregistered": "同时已取消注册:{ids}",
|
||||
};
|
||||
|
||||
Vendored
+14
-1
@@ -23,6 +23,10 @@ export function clear(node) {
|
||||
while (node && node.firstChild) node.removeChild(node.firstChild);
|
||||
}
|
||||
|
||||
export function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
export function mount(node, ...children) {
|
||||
clear(node);
|
||||
for (const c of children.flat())
|
||||
@@ -46,7 +50,14 @@ export function toast(msg, kind = "info", ms = 3200) {
|
||||
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
|
||||
export function modal(title, body, onSave, saveLabel) {
|
||||
const backdrop = h("div", { class: "modal-backdrop" });
|
||||
const close = () => backdrop.remove();
|
||||
const onKeydown = (e) => {
|
||||
if (e.key === "Escape") close();
|
||||
};
|
||||
const close = () => {
|
||||
backdrop.remove();
|
||||
document.body.classList.remove("modal-open");
|
||||
document.removeEventListener("keydown", onKeydown);
|
||||
};
|
||||
backdrop.addEventListener("click", (e) => {
|
||||
if (e.target === backdrop) close();
|
||||
});
|
||||
@@ -73,6 +84,8 @@ export function modal(title, body, onSave, saveLabel) {
|
||||
)
|
||||
);
|
||||
backdrop.appendChild(card);
|
||||
document.body.classList.add("modal-open");
|
||||
document.addEventListener("keydown", onKeydown);
|
||||
document.body.appendChild(backdrop);
|
||||
return { close };
|
||||
}
|
||||
|
||||
Vendored
+191
@@ -0,0 +1,191 @@
|
||||
// Ready-to-run .cmf models, read live from the infosave HuggingFace account via
|
||||
// /setup/featured — so a newly published model shows up without a code change.
|
||||
// Shared by the first-run wizard and the Import view: same cards, same download
|
||||
// / progress / delete behaviour in both places.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
|
||||
export async function loadFeatured() {
|
||||
try {
|
||||
const r = await api.setupFeatured();
|
||||
return r.models || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export function quantFromVariant(name) {
|
||||
const s = (name || "").toLowerCase();
|
||||
if (s.includes("q1t")) return "Q1T";
|
||||
if (s.includes("q2tp")) return "Q2TP";
|
||||
if (s.includes("q4tp")) return "Q4TP";
|
||||
if (s.includes("q4t")) return "Q4T";
|
||||
if (s.includes("q1")) return "Q1";
|
||||
if (s.includes("q4")) return "Q4";
|
||||
if (s.includes("q8")) return "Q8";
|
||||
return "";
|
||||
}
|
||||
|
||||
// Same thresholds as the backend's format_bytes, so card and job agree.
|
||||
export function fmtBytes(n) {
|
||||
if (!n && n !== 0) return "";
|
||||
if (n >= 1e9) return (n / 1e9).toFixed(1) + " GB";
|
||||
if (n >= 1e6) return Math.round(n / 1e6) + " MB";
|
||||
if (n >= 1e3) return Math.round(n / 1e3) + " KB";
|
||||
return n + " B";
|
||||
}
|
||||
|
||||
export function jobPct(j) {
|
||||
if (j.progress == null) return null;
|
||||
return Math.max(0, Math.min(100, Math.round(j.progress * 100)));
|
||||
}
|
||||
|
||||
/// "downloading · 42% · 1.4 / 3.2 GB" — whatever of it the job actually knows.
|
||||
/// A settled job shows its result, not the phase it stopped in.
|
||||
export function jobStatusText(j) {
|
||||
const parts = [];
|
||||
if (j.state === "running") {
|
||||
if (j.phase) parts.push(j.phase);
|
||||
const pct = jobPct(j);
|
||||
if (pct != null) parts.push(pct + "%");
|
||||
if (j.total_bytes) parts.push(`${fmtBytes(j.done_bytes || 0)} / ${fmtBytes(j.total_bytes)}`);
|
||||
} else if (j.state === "done" && j.size_bytes) {
|
||||
parts.push(fmtBytes(j.size_bytes));
|
||||
}
|
||||
return parts.join(" · ");
|
||||
}
|
||||
|
||||
export function jobBar(j) {
|
||||
const pct = jobPct(j);
|
||||
if (j.state === "done") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill ok", style: "width:100%" }));
|
||||
if (j.state !== "running") return h("div", { class: "pbar" }, h("div", { class: "pbar-fill err", style: "width:100%" }));
|
||||
// known percentage → determinate fill; unknown (listing files) → animated slider
|
||||
return h("div", { class: "pbar" },
|
||||
pct != null
|
||||
? h("div", { class: "pbar-fill det", style: `width:${Math.max(pct, 1)}%` })
|
||||
: h("div", { class: "pbar-fill run" })
|
||||
);
|
||||
}
|
||||
|
||||
/// HF gives `desc` as a bare pipeline tag ("text-generation") for live entries and
|
||||
/// a real sentence for curated ones. A one-word tag belongs in the badge row, not
|
||||
/// on a line of its own under every card.
|
||||
function descBadge(desc) {
|
||||
return desc && !desc.includes(" ") ? h("span", { class: "badge" }, desc) : null;
|
||||
}
|
||||
function descLine(desc) {
|
||||
return desc && desc.includes(" ") ? h("div", { class: "muted small", style: "margin:6px 0" }, desc) : null;
|
||||
}
|
||||
|
||||
/// One model. `ctx` = { hosts: Map(jobId → element), onDownload(m, v), onDelete(m, v) }.
|
||||
/// Live progress lands in the per-variant host so a poll can repaint it in place,
|
||||
/// without re-rendering the grid under the user's cursor.
|
||||
export function featuredCard(m, ctx) {
|
||||
const badge = m.badge || "CMF";
|
||||
const label = m.label || m.id;
|
||||
const desc = m.desc || "";
|
||||
const variants = m.variants && m.variants.length ? m.variants : null;
|
||||
const size = m.size || (m.size_bytes ? fmtBytes(m.size_bytes) : "");
|
||||
const isDownloaded = !!m.downloaded;
|
||||
const progressHost = (v) => {
|
||||
const host = h("div", { class: "card-prog" });
|
||||
const jid = (v && v.job) || m.job;
|
||||
if (jid) ctx.hosts.set(jid, host);
|
||||
return host;
|
||||
};
|
||||
// multi-variant (e.g. DeepSeek with parts-q2tp / parts-q4tp) → separate buttons per variant
|
||||
if (variants && variants.length > 1) {
|
||||
return h("div", { class: "hf-card featured-card" + (isDownloaded ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, isDownloaded ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
descBadge(desc),
|
||||
isDownloaded ? h("span", { class: "badge ok" }, "✓ " + t("onboarding.downloaded")) : null),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; flex-direction:column; gap:6px; margin-top:8px" },
|
||||
...variants.map((v) => {
|
||||
const q = quantFromVariant(v.name) || badge;
|
||||
const done = !!v.downloaded;
|
||||
const upd = !!v.update_available;
|
||||
const lab = done
|
||||
? (upd ? `↻ ${t("onboarding.update")} ${q} · ${v.size}` : `✓ ${q} · ${v.size}`)
|
||||
: `⬇ ${q} · ${v.size}`;
|
||||
const btn = h("button", {
|
||||
class: "btn sm " + (done && !upd ? "ghost" : "primary"),
|
||||
disabled: done && !upd ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v),
|
||||
title: v.name + (done ? " — " + t("onboarding.downloaded") : ""),
|
||||
}, lab);
|
||||
const row = h("div", { style: "display:flex; gap:6px; align-items:center; flex-wrap:wrap" }, btn);
|
||||
if (done) {
|
||||
const del = h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑");
|
||||
del.addEventListener("click", () => ctx.onDelete(m, v));
|
||||
row.appendChild(del);
|
||||
}
|
||||
return h("div", {}, row, progressHost(v));
|
||||
})
|
||||
)
|
||||
);
|
||||
}
|
||||
const v0 = variants && variants[0];
|
||||
const done0 = !!(v0 && v0.downloaded) || isDownloaded;
|
||||
const upd0 = !!(v0 && v0.update_available) || !!m.update_available;
|
||||
const q0 = quantFromVariant((v0 && v0.name) || "") || badge;
|
||||
const size0 = (v0 && v0.size) || size;
|
||||
const btn0 = h("button", {
|
||||
class: "btn sm " + (done0 && !upd0 ? "ghost" : "primary"),
|
||||
disabled: done0 && !upd0 ? true : null,
|
||||
onClick: () => ctx.onDownload(m, v0),
|
||||
title: done0 && !upd0 ? t("onboarding.downloaded") : ""
|
||||
}, done0
|
||||
? (upd0 ? `↻ ${t("onboarding.update")} ${q0} · ${size0}` : `✓ ${q0} · ${size0}`)
|
||||
: `⬇ ${q0}${size0 ? " · " + size0 : ""}`);
|
||||
const del0 = done0 ? h("button", { class: "btn sm ghost danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete")) : null;
|
||||
if (del0) del0.addEventListener("click", () => ctx.onDelete(m, v0));
|
||||
return h("div", { class: "hf-card featured-card" + (done0 ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, label, done0 ? h("span", { class: "badge ok", style: "margin-left:6px" }, "✓") : null),
|
||||
h("div", { class: "hf-meta" },
|
||||
h("span", { class: "badge" }, badge),
|
||||
size0 ? h("span", { class: "badge", style: "background:var(--bg); border-style:dashed" + (done0 ? "; opacity:.7" : "") }, (done0 ? "✓ " : "⬇ ") + size0) : null,
|
||||
descBadge(desc)
|
||||
),
|
||||
descLine(desc),
|
||||
h("div", { class: "muted small" }, m.id),
|
||||
h("div", { style: "display:flex; gap:6px; margin-top:8px; align-items:center; flex-wrap:wrap" }, btn0, del0),
|
||||
progressHost(v0)
|
||||
);
|
||||
}
|
||||
|
||||
/// Repaint the per-variant progress hosts from the current job list.
|
||||
export function paintFeaturedHosts(hosts, jobs, onCancel) {
|
||||
const byId = new Map(jobs.map((j) => [j.id, j]));
|
||||
for (const [id, host] of hosts) {
|
||||
const j = byId.get(id);
|
||||
if (!j || j.state !== "running") { mount(host); continue; }
|
||||
const cancel = h("button", { class: "btn sm ghost", style: "margin-left:auto", title: t("import.cancel") }, "✕");
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await onCancel(j); } catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
mount(host,
|
||||
jobBar(j),
|
||||
h("div", { style: "display:flex; align-items:center; gap:8px" },
|
||||
h("span", { class: "muted small" }, jobStatusText(j)),
|
||||
cancel
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Start the download for a featured variant. `local_name` is the stem the
|
||||
/// gateway itself derived, sent back verbatim so download / progress / delete
|
||||
/// can never disagree about which file this is.
|
||||
export async function startFeaturedDownload(m, v) {
|
||||
const fallbackBase = m.id.split("/").pop().toLowerCase().replace(/[^a-z0-9_-]/g, "-");
|
||||
const name = (v && v.local_name) || m.local_name || fallbackBase;
|
||||
const variant = v && v.name ? v.name : undefined;
|
||||
await api.startImport({ repo: m.id, quant: "Q1", name, variant });
|
||||
return variant;
|
||||
}
|
||||
Vendored
+88
-5
@@ -2,26 +2,36 @@
|
||||
// live progress view. Search HF, pick quantization + advanced params, watch
|
||||
// the conversion (multiple in parallel, resumable across tab switches), then
|
||||
// register the result as a local model.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { h, mount, toast, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobStatusText,
|
||||
} from "./featured.js";
|
||||
|
||||
const QUANTS = [
|
||||
["Q8_2F", "import.q.q8_2f"],
|
||||
["Q8_ROW", "import.q.q8_row"],
|
||||
["Q4_BLOCK", "import.q.q4"],
|
||||
["Q4_TILED", "import.q.q4_tiled"],
|
||||
["VBIT", "import.q.vbit"],
|
||||
["Q1T", "import.q.q1t"],
|
||||
["Q1S", "import.q.q1s"],
|
||||
["Q1P", "import.q.q1p"],
|
||||
["Q1", "import.q.q1"],
|
||||
["F16", "import.q.f16"],
|
||||
["F32", "import.q.f32"],
|
||||
];
|
||||
|
||||
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control,
|
||||
hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
function fmtNum(n) {
|
||||
if (n == null) return "—";
|
||||
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
|
||||
@@ -44,6 +54,10 @@ export function renderImport() {
|
||||
let selected = null;
|
||||
let lastModels = []; // cache of the last search, so re-highlighting needs no refetch
|
||||
let jobsTimer = null;
|
||||
let featured = []; // ready-made .cmf models from the infosave HF account
|
||||
let featuredHosts = new Map(); // job id → live progress slot inside a featured card
|
||||
let lastJobs = [];
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
|
||||
const searchIn = h("input", {
|
||||
class: "hf-search",
|
||||
@@ -245,6 +259,12 @@ export function renderImport() {
|
||||
}
|
||||
// Load + render all jobs. Runs on mount (before `root` is attached — so no
|
||||
// isConnected guard here) and on each poll tick.
|
||||
async function cancelJob(j) {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "warn");
|
||||
await refreshJobs();
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
let jobs = [];
|
||||
try {
|
||||
@@ -253,7 +273,14 @@ export function renderImport() {
|
||||
return;
|
||||
}
|
||||
jobs = jobs.filter((j) => j.state !== "cancelled"); // hide user-cancelled jobs
|
||||
lastJobs = jobs;
|
||||
renderJobs(jobs);
|
||||
// A settled download changes what the featured cards offer (✓ + delete), so
|
||||
// re-read the list on that transition — not on every poll tick.
|
||||
const settled = jobs.some((j) => jobStates.get(j.id) === "running" && j.state !== "running");
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) await renderFeatured();
|
||||
else paintFeaturedHosts(featuredHosts, jobs, cancelJob);
|
||||
if (jobs.some((j) => j.state === "running")) ensurePolling();
|
||||
else stopPolling();
|
||||
}
|
||||
@@ -280,7 +307,7 @@ export function renderImport() {
|
||||
(running && pct != null ? " · " + pct + "%" : "") +
|
||||
(j.size_bytes ? " · " + fmtBytes(j.size_bytes) : "")
|
||||
);
|
||||
const phase = h("div", { class: "muted small" }, j.phase || "");
|
||||
const phase = h("div", { class: "muted small" }, jobStatusText(j));
|
||||
const logEl = h("pre", { class: "conv-log" }, (j.log || []).slice(-10).join("\n"));
|
||||
|
||||
const actions = [];
|
||||
@@ -358,6 +385,59 @@ export function renderImport() {
|
||||
);
|
||||
}
|
||||
|
||||
// ── featured: the infosave account, read live so new releases just show up ──
|
||||
const featuredGrid = h("div", { class: "hf-grid" }, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
const reloadBtn = h("button", { class: "btn ghost sm" }, "↻ " + t("common.refresh"));
|
||||
const featuredHost = h(
|
||||
"div",
|
||||
{ class: "card featured-section" },
|
||||
h("div", { class: "card-head", style: "display:flex; align-items:center; gap:10px" },
|
||||
h("div", { class: "card-title", style: "margin:0" }, "⚡ " + t("import.featuredTitle")),
|
||||
h("div", { style: "flex:1" }),
|
||||
reloadBtn),
|
||||
h("div", { class: "muted small", style: "margin:2px 0 10px" }, t("import.featuredSub")),
|
||||
featuredGrid
|
||||
);
|
||||
|
||||
async function startFeatured(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function deleteFeatured(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!window.confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await renderFeatured();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function renderFeatured(reload = true) {
|
||||
if (reload) featured = await loadFeatured();
|
||||
featuredHosts = new Map();
|
||||
const ctx = { hosts: featuredHosts, onDownload: startFeatured, onDelete: deleteFeatured };
|
||||
mount(featuredGrid, ...(featured.length
|
||||
? featured.map((m) => featuredCard(m, ctx))
|
||||
: [h("div", { class: "muted pad" }, t("import.noResults"))]));
|
||||
paintFeaturedHosts(featuredHosts, lastJobs, cancelJob);
|
||||
}
|
||||
reloadBtn.addEventListener("click", async () => {
|
||||
reloadBtn.disabled = true;
|
||||
mount(featuredGrid, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
await renderFeatured();
|
||||
reloadBtn.disabled = false;
|
||||
});
|
||||
|
||||
const root = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -365,10 +445,13 @@ export function renderImport() {
|
||||
h("h2", {}, t("nav.import")),
|
||||
h("div", { class: "muted" }, t("import.subtitle"))),
|
||||
jobsHost, // progress panel on top, above the search
|
||||
featuredHost, // featured ready-to-run models
|
||||
h("div", { class: "card" }, searchIn, results),
|
||||
formHost
|
||||
);
|
||||
doSearch(); // initial trending list (mounts into `results`)
|
||||
renderFeatured(); // ready-made .cmf models, fetched live
|
||||
refreshJobs(); // resume: show any running/recent conversions immediately
|
||||
return root;
|
||||
}
|
||||
|
||||
|
||||
Vendored
+103
-22
@@ -1,5 +1,5 @@
|
||||
// Models — pool table with add/edit modal, probe, and per-model secret entry.
|
||||
import { h, mount, modal, toast, confirmDialog, money } from "../ui.js";
|
||||
import { h, mount, modal, toast, confirmDialog, money, opt } from "../ui.js";
|
||||
import { t } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import { appState } from "../app.js";
|
||||
@@ -7,9 +7,6 @@ import { appState } from "../app.js";
|
||||
function field(label, control, hint) {
|
||||
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
|
||||
}
|
||||
function opt(v, label, sel) {
|
||||
return h("option", { value: v, selected: sel ? true : null }, label || v);
|
||||
}
|
||||
|
||||
function keyBadge(src) {
|
||||
const map = { store: "store", env: "env", missing: "bad", none: "" };
|
||||
@@ -54,7 +51,16 @@ function modelForm(meta, existing) {
|
||||
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
|
||||
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
|
||||
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
|
||||
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
|
||||
const CAP_EMOJIS = {
|
||||
tools: "🔧 tools",
|
||||
vision: "👁️ vision",
|
||||
audio: "🎙️ audio",
|
||||
reasoning: "🧠 reasoning",
|
||||
json: "📄 json",
|
||||
caching: "⚡ caching"
|
||||
};
|
||||
|
||||
const capInputs = (meta.caps || Object.keys(CAP_EMOJIS)).map((c) => {
|
||||
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
|
||||
return { c, cb };
|
||||
});
|
||||
@@ -73,25 +79,55 @@ function modelForm(meta, existing) {
|
||||
capInputs.forEach(({ c, cb }) => { cb.checked = d.caps.includes(c); });
|
||||
keyIn.placeholder = d.needsKey ? t("models.form.apiKeyPlaceholder") : t("models.form.apiKeyNotNeeded");
|
||||
mount(keyStatus, "");
|
||||
// Automatically pre-fetch models (especially useful for OpenRouter which doesn't need a key)
|
||||
testKey(true);
|
||||
}
|
||||
providerSel.addEventListener("change", applyDefaults);
|
||||
if (!existing) applyDefaults();
|
||||
|
||||
// Auto-fill the id from the model name (until the user edits id by hand).
|
||||
if (!existing) {
|
||||
modelIn.addEventListener("input", () => {
|
||||
if (!idIn.value || idIn._auto) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
let cachedModels = [];
|
||||
|
||||
const autofillData = () => {
|
||||
console.log("autofillData triggered. modelIn.value:", modelIn.value);
|
||||
console.log("cachedModels length:", cachedModels.length);
|
||||
if (!existing && (!idIn.value || idIn._auto)) {
|
||||
idIn.value = slugId(modelIn.value);
|
||||
idIn._auto = true;
|
||||
console.log("auto-filled idIn:", idIn.value);
|
||||
}
|
||||
const found = cachedModels.find((m) => (m.id || m) === modelIn.value);
|
||||
console.log("found model:", found);
|
||||
if (found) {
|
||||
if (found.price_in !== undefined) {
|
||||
priceInIn.value = parseFloat(Number(found.price_in).toFixed(4));
|
||||
priceOutIn.value = parseFloat(Number(found.price_out).toFixed(4));
|
||||
console.log("Updated prices:", priceInIn.value, priceOutIn.value);
|
||||
}
|
||||
});
|
||||
if (found.caps && Array.isArray(found.caps)) {
|
||||
capInputs.forEach(({ c, cb }) => {
|
||||
cb.checked = found.caps.includes(c);
|
||||
});
|
||||
console.log("Updated caps:", found.caps);
|
||||
}
|
||||
}
|
||||
};
|
||||
modelIn.addEventListener("input", autofillData);
|
||||
modelIn.addEventListener("change", autofillData);
|
||||
|
||||
if (!existing) {
|
||||
idIn.addEventListener("input", () => { idIn._auto = false; });
|
||||
} else {
|
||||
console.log("Editing existing model, fetching testKey(true)");
|
||||
testKey(true);
|
||||
}
|
||||
|
||||
async function testKey() {
|
||||
if (!baseUrlIn.value.trim()) { mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?")); return; }
|
||||
async function testKey(silent = false) {
|
||||
if (!baseUrlIn.value.trim()) {
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?"));
|
||||
return;
|
||||
}
|
||||
testBtn.disabled = true;
|
||||
mount(keyStatus, t("models.form.testing"));
|
||||
if (!silent) mount(keyStatus, t("models.form.testing"));
|
||||
try {
|
||||
const r = await api.providerModels({
|
||||
provider: providerSel.value,
|
||||
@@ -100,24 +136,61 @@ function modelForm(meta, existing) {
|
||||
api_key_env: envFor(),
|
||||
});
|
||||
if (r.ok) {
|
||||
dl.replaceChildren(...(r.models || []).map((mm) => h("option", { value: mm })));
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, "✓ " + t("models.form.foundModels", { n: r.count })));
|
||||
modelIn.focus();
|
||||
cachedModels = r.models || [];
|
||||
dl.replaceChildren(...cachedModels.map((mm) => h("option", { value: mm.id || mm })));
|
||||
|
||||
let msg = "✓ " + t("models.form.foundModels", { n: r.count });
|
||||
if (r.balance !== undefined) {
|
||||
msg += ` (Balance: $${r.balance.toFixed(4)})`;
|
||||
}
|
||||
mount(keyStatus, h("span", { style: "color:var(--good)" }, msg));
|
||||
if (!silent) modelIn.focus();
|
||||
} else {
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
|
||||
}
|
||||
} catch (e) {
|
||||
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
|
||||
}
|
||||
testBtn.disabled = false;
|
||||
}
|
||||
testBtn.addEventListener("click", testKey);
|
||||
testBtn.addEventListener("click", () => testKey(false));
|
||||
keyIn.addEventListener("change", () => testKey(true));
|
||||
|
||||
const keyHint =
|
||||
existing && existing.key_source === "store" ? t("models.form.keyStored")
|
||||
: existing && existing.key_source === "env" ? t("models.form.keyFromEnv")
|
||||
: t("models.form.apiKeyHint");
|
||||
|
||||
// ⚙ Generation & Thinking Parameters
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: m.temperature ?? "" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: m.top_p ?? "" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: m.max_tokens ?? "" });
|
||||
const thinkIn = h("input", { type: "number", min: "0", placeholder: "1024", value: m.think_budget ?? "" });
|
||||
const o1Sel = h("select", {},
|
||||
h("option", { value: "" }, t("import.adv.auto")),
|
||||
h("option", { value: "all" }, "all — O(1) Attention"),
|
||||
h("option", { value: "deep" }, "deep — O(1) Deep"),
|
||||
h("option", { value: "off" }, "off — Standard"));
|
||||
if (m.o1) o1Sel.value = m.o1;
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: m.skip_mtp ? true : null });
|
||||
const sysPromptIn = h("textarea", { rows: 2, placeholder: t("models.form.systemPromptPlaceholder"), value: m.system_prompt || "" });
|
||||
|
||||
const advParams = h(
|
||||
"details",
|
||||
{ class: "adv-params", style: "margin-top:12px;padding:10px;border:1px solid var(--border);border-radius:6px;" },
|
||||
h("summary", { style: "font-weight:600;cursor:pointer;" }, "⚙ " + t("models.form.paramsTitle")),
|
||||
h("div", { class: "row", style: "margin-top:8px" },
|
||||
field(t("models.form.temperature"), tempIn, t("models.form.temperatureHint")),
|
||||
field(t("models.form.topP"), topPIn, t("models.form.topPHint"))),
|
||||
h("div", { class: "row" },
|
||||
field(t("models.form.maxTokens"), maxTokIn, t("models.form.maxTokensHint")),
|
||||
field(t("models.form.thinkBudget"), thinkIn, t("models.form.thinkBudgetHint"))),
|
||||
h("div", { class: "row" },
|
||||
field("O(1) Attention", o1Sel, t("import.adv.o1Hint")),
|
||||
field("MTP", h("label", { class: "check" }, skipMtpCb, t("import.adv.skipMtp")))),
|
||||
field(t("models.form.systemPrompt"), sysPromptIn, t("models.form.systemPromptHint"))
|
||||
);
|
||||
|
||||
const node = h(
|
||||
"div",
|
||||
{},
|
||||
@@ -128,7 +201,8 @@ function modelForm(meta, existing) {
|
||||
dl,
|
||||
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
|
||||
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c))))
|
||||
field(t("models.form.caps"), h("div", { class: "flex wrap", style: "gap:10px" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check", style: "margin:0" }, cb, CAP_EMOJIS[c] || c)))),
|
||||
advParams
|
||||
);
|
||||
|
||||
const getValue = () => {
|
||||
@@ -147,6 +221,13 @@ function modelForm(meta, existing) {
|
||||
price_out: parseFloat(priceOutIn.value) || 0,
|
||||
api_key_env: hasKey ? keyEnv : null,
|
||||
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: thinkIn.value !== "" ? parseInt(thinkIn.value) : null,
|
||||
o1: o1Sel.value || null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysPromptIn.value.trim() || null,
|
||||
},
|
||||
secret: typedKey,
|
||||
keyEnv,
|
||||
|
||||
Vendored
+332
@@ -0,0 +1,332 @@
|
||||
// First-run onboarding wizard: language → featured .cmf models (infosave) → HF search+convert → done.
|
||||
// No manual TOML edits: every action goes through the admin API.
|
||||
import { h, mount, toast } from "../ui.js";
|
||||
import { t, getLang, setLang, LANGS } from "../i18n.js";
|
||||
import { api } from "../api.js";
|
||||
import {
|
||||
loadFeatured as fetchFeatured,
|
||||
featuredCard,
|
||||
paintFeaturedHosts,
|
||||
startFeaturedDownload,
|
||||
jobBar,
|
||||
jobStatusText,
|
||||
fmtBytes,
|
||||
} from "./featured.js";
|
||||
|
||||
let step = 0; // 0 lang, 1 featured, 2 hf convert, 3 done
|
||||
let featured = [];
|
||||
let featuredLoaded = false;
|
||||
|
||||
function langStep(onNext) {
|
||||
const cards = LANGS.map((l) =>
|
||||
h("button", {
|
||||
class: "onb-lang" + (getLang() === l.code ? " sel" : ""),
|
||||
onClick: () => {
|
||||
setLang(l.code);
|
||||
cards.forEach((c) => c.classList.toggle("sel", c.dataset.code === l.code));
|
||||
toast(l.flag + " " + l.label, "ok");
|
||||
},
|
||||
"data-code": l.code,
|
||||
title: l.label,
|
||||
"aria-label": l.label,
|
||||
}, h("span", { class: "flag" }, l.flag), h("b", {}, l.label))
|
||||
);
|
||||
return h("div", { class: "onb-step" },
|
||||
h("h2", {}, t("onboarding.welcome")),
|
||||
h("p", { class: "muted" }, t("onboarding.welcomeSub")),
|
||||
h("div", { class: "onb-langs" }, ...cards),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn primary", onClick: onNext }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
export function renderOnboarding() {
|
||||
const root = h("div", { class: "onb-wrap" });
|
||||
let jobsPoll = null;
|
||||
let jobs = [];
|
||||
let jobStates = new Map(); // job id → last seen state, to spot completions
|
||||
let hosts = new Map(); // job id → card progress host (rebuilt on every render)
|
||||
let jobsHost = h("div", {});
|
||||
|
||||
// Job rows: bar, live status, and the actions that finish or undo a download.
|
||||
function jobsMini(list) {
|
||||
if (!list.length) return h("div", { class: "muted small" }, t("onboarding.noJobs"));
|
||||
return h("div", {},
|
||||
...list.map((j) => {
|
||||
const act = [];
|
||||
if (j.state === "done") {
|
||||
const btn = h("button", { class: "btn primary sm" }, "✓ " + t("import.register"));
|
||||
btn.addEventListener("click", async () => {
|
||||
btn.disabled = true;
|
||||
try {
|
||||
const r = await api.registerImport(j.id);
|
||||
toast(t("import.registered", { id: r.model_id }), "ok");
|
||||
} catch (e) { toast(e.message, "err"); btn.disabled = false; }
|
||||
});
|
||||
act.push(btn);
|
||||
}
|
||||
if (j.state === "running") {
|
||||
const cancel = h("button", { class: "btn ghost sm" }, "✕ " + t("import.cancel"));
|
||||
cancel.addEventListener("click", async () => {
|
||||
cancel.disabled = true;
|
||||
try { await api.cancelImport(j.id); toast(t("import.cancelling"), "ok"); await refreshJobs(); }
|
||||
catch (e) { toast(e.message, "err"); cancel.disabled = false; }
|
||||
});
|
||||
act.push(cancel);
|
||||
} else {
|
||||
// done / error / cancelled — remove the .cmf from disk and redo the step
|
||||
const del = h("button", { class: "btn ghost sm danger", title: t("onboarding.deleteFile") }, "🗑 " + t("import.delete"));
|
||||
del.addEventListener("click", () => deleteJob(j));
|
||||
act.push(del);
|
||||
}
|
||||
const known = ["running", "done", "error", "cancelled"].includes(j.state);
|
||||
return h("div", { class: "job", style: "margin:8px 0" },
|
||||
h("div", { class: "job-head" },
|
||||
h("b", {}, j.name || j.repo || j.id),
|
||||
h("span", { class: "badge" + (j.state === "done" ? " ok" : j.state === "running" ? "" : " bad") }, known ? t("import.state." + j.state) : j.state)
|
||||
),
|
||||
jobBar(j),
|
||||
h("div", { class: "muted small" }, jobStatusText(j)),
|
||||
h("pre", { class: "conv-log", style: "max-height:80px" }, (j.log || []).slice(-6).join("\n")),
|
||||
act.length ? h("div", { class: "actions" }, ...act) : null
|
||||
);
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
// Repaint only the live bits (card hosts + job list) — cheap enough for 1.5s.
|
||||
function paintJobs() {
|
||||
mount(jobsHost, jobs.length
|
||||
? h("div", { class: "card" }, h("div", { class: "card-title" }, t("import.jobs")), jobsMini(jobs))
|
||||
: h("div", {}));
|
||||
paintFeaturedHosts(hosts, jobs, async (j) => {
|
||||
await api.cancelImport(j.id);
|
||||
toast(t("import.cancelling"), "ok");
|
||||
await refreshJobs();
|
||||
});
|
||||
}
|
||||
|
||||
async function refreshJobs() {
|
||||
try { jobs = (await api.listImports()).jobs || []; } catch { jobs = []; }
|
||||
// A job that just settled changes what the cards offer (✓ + delete), so the
|
||||
// featured list has to be re-read — but only on the transition, not per poll.
|
||||
const settled = jobs.some((j) => {
|
||||
const prev = jobStates.get(j.id);
|
||||
return prev === "running" && j.state !== "running";
|
||||
});
|
||||
jobStates = new Map(jobs.map((j) => [j.id, j.state]));
|
||||
if (settled) {
|
||||
await loadFeatured();
|
||||
render(); // rebuilds the cards (✓ + delete) and repaints the job rows
|
||||
} else {
|
||||
paintJobs();
|
||||
}
|
||||
if (jobs.some((j) => j.state === "running")) {
|
||||
if (!jobsPoll) jobsPoll = setInterval(refreshJobs, 1500);
|
||||
} else {
|
||||
if (jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; }
|
||||
}
|
||||
}
|
||||
|
||||
async function loadFeatured() {
|
||||
featured = await fetchFeatured();
|
||||
featuredLoaded = true;
|
||||
}
|
||||
|
||||
function topNav(prev, next, extra) {
|
||||
return h("div", { class: "actions", style: "margin-bottom:12px; position:sticky; top:0; background:var(--bg); z-index:2; padding:8px 0; border-bottom:1px solid var(--border-soft); backdrop-filter:blur(4px)" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
extra ? extra : h("div", { style: "flex:1" }),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
}
|
||||
|
||||
async function startDownload(m, v) {
|
||||
try {
|
||||
const variant = await startFeaturedDownload(m, v);
|
||||
toast(t("import.started") + (variant ? ` · ${variant}` : ""), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
async function handleDelete(m, v) {
|
||||
const name = (v && v.local_name) || m.local_name || m.id;
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: name }))) return;
|
||||
try {
|
||||
const r = await api.deleteSetupFile(m.id, v && v.name, (v && v.local_name) || m.local_name);
|
||||
toast(t("import.deleted"), "ok");
|
||||
if (r && r.unregistered && r.unregistered.length) {
|
||||
toast(t("onboarding.unregistered", { ids: r.unregistered.join(", ") }), "ok");
|
||||
}
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
// Delete straight from a job row: the gateway knows that job's output path.
|
||||
async function deleteJob(j) {
|
||||
if (!confirm(t("onboarding.confirmDelete", { id: j.name || j.repo }))) return;
|
||||
try {
|
||||
await api.deleteImport(j.id);
|
||||
toast(t("import.deleted"), "ok");
|
||||
await loadFeatured();
|
||||
render();
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
}
|
||||
|
||||
function render() {
|
||||
const dots = [0, 1, 2, 3].map((i) => h("span", { class: "dot" + (i === step ? " on" : i < step ? " done" : "") }));
|
||||
const header = h("div", { class: "onb-header" },
|
||||
h("div", { class: "onb-dots" }, ...dots),
|
||||
h("div", { class: "onb-title" }, t("onboarding.title") + ` — ${step + 1}/4`)
|
||||
);
|
||||
|
||||
let body;
|
||||
const next = () => { step = Math.min(3, step + 1); render(); };
|
||||
const prev = () => { step = Math.max(0, step - 1); render(); };
|
||||
hosts = new Map(); // rebuilt by featuredCard below
|
||||
|
||||
if (step === 0) {
|
||||
const top = h("div", { class: "actions", style: "margin-bottom:12px; justify-content:flex-end" },
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
);
|
||||
body = h("div", {},
|
||||
top,
|
||||
langStep(next)
|
||||
);
|
||||
} else if (step === 1) {
|
||||
const top = topNav(prev, next, h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip")));
|
||||
const ctx = { hosts, onDownload: startDownload, onDelete: handleDelete };
|
||||
// the list is fetched from HuggingFace — say so instead of showing a blank grid
|
||||
const grid = featured.length
|
||||
? h("div", { class: "hf-grid" }, ...featured.map((m) => featuredCard(m, ctx)))
|
||||
: h("div", { class: "muted pad" }, featuredLoaded ? t("import.noResults") : t("import.searching"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
top,
|
||||
h("h2", {}, "📦 " + t("onboarding.featuredTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.featuredSub")),
|
||||
grid,
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn", onClick: async () => { await loadFeatured(); render(); await refreshJobs(); } }, t("common.refresh")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →"),
|
||||
h("button", { class: "btn ghost", onClick: () => { step = 3; render(); } }, t("onboarding.skip"))
|
||||
)
|
||||
);
|
||||
} else if (step === 2) {
|
||||
const top2 = topNav(prev, next);
|
||||
// lightweight HF search inline
|
||||
const inp = h("input", { placeholder: t("import.searchPlaceholder"), style: "width:100%" });
|
||||
const res = h("div", { class: "hf-grid" });
|
||||
const quantSel = h("select", {},
|
||||
h("option", { value: "Q8_2F" }, "Q8_2F — " + t("import.q.q8_2f")),
|
||||
h("option", { value: "Q4_BLOCK" }, "Q4 — " + t("import.q.q4")),
|
||||
h("option", { value: "Q1T" }, "Q1T — " + t("import.q.q1t")),
|
||||
h("option", { value: "Q1" }, "Q1 — " + t("import.q.q1")),
|
||||
);
|
||||
let selected = null;
|
||||
let debounce = null;
|
||||
async function doSearch() {
|
||||
const q = inp.value.trim();
|
||||
mount(res, h("div", { class: "muted pad" }, t("import.searching")));
|
||||
try {
|
||||
const r = await api.hfSearch(q, 16);
|
||||
const models = r.models || [];
|
||||
if (!models.length) { mount(res, h("div", { class: "muted pad" }, t("import.noResults"))); return; }
|
||||
mount(res, ...models.map((m) => {
|
||||
const card = h("div", { class: "hf-card" + (selected && selected.id === m.id ? " sel" : "") },
|
||||
h("div", { class: "hf-id" }, m.id),
|
||||
h("div", { class: "hf-meta" }, h("span", { class: "badge" }, m.pipeline_tag || "model"), h("span", { class: "muted" }, "♥ " + (m.likes ?? 0)))
|
||||
);
|
||||
card.addEventListener("click", () => { selected = m; doSearch(); });
|
||||
return card;
|
||||
}));
|
||||
} catch (e) { mount(res, h("div", { class: "err" }, String(e.message))); }
|
||||
}
|
||||
inp.addEventListener("input", () => { clearTimeout(debounce); debounce = setTimeout(doSearch, 300); });
|
||||
const go = h("button", { class: "btn primary" }, "⬇ " + t("import.convert"));
|
||||
go.addEventListener("click", async () => {
|
||||
if (!selected) { toast(t("import.noResults"), "warn"); return; }
|
||||
go.disabled = true;
|
||||
try {
|
||||
await api.startImport({ repo: selected.id, quant: quantSel.value, name: selected.id.split("/").pop().toLowerCase() });
|
||||
toast(t("import.started"), "ok");
|
||||
await refreshJobs();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
go.disabled = false;
|
||||
});
|
||||
// auto initial search
|
||||
setTimeout(doSearch, 100);
|
||||
body = h("div", { class: "onb-step" },
|
||||
top2,
|
||||
h("h2", {}, "🔍 " + t("onboarding.hfTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.hfSub")),
|
||||
h("div", { class: "card" }, inp, res),
|
||||
h("div", { class: "card", style: "margin-top:12px" },
|
||||
h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("import.quant")), quantSel)),
|
||||
h("div", { class: "actions" }, go)
|
||||
),
|
||||
jobsHost,
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
h("button", { class: "btn primary", onClick: next }, t("onboarding.next") + " →")
|
||||
)
|
||||
);
|
||||
} else {
|
||||
// done — finishing connects every completed download as a local model
|
||||
const ready = jobs.filter((j) => j.state === "done");
|
||||
const doneBtn = h("button", { class: "btn primary" }, t("onboarding.finish"));
|
||||
doneBtn.addEventListener("click", async () => {
|
||||
doneBtn.disabled = true;
|
||||
let r = null;
|
||||
try { r = await api.setupComplete(getLang(), true); } catch (e) { toast(e.message, "err"); }
|
||||
if (r && r.registered && r.registered.length) {
|
||||
toast(t("onboarding.registeredN", { n: r.registered.length, ids: r.registered.join(", ") }), "ok");
|
||||
if (!r.runtime_installed) toast(t("onboarding.runtimeMissing"), "warn");
|
||||
// let the toast be readable before the reload swaps the page
|
||||
await new Promise((res) => setTimeout(res, 900));
|
||||
}
|
||||
location.hash = "#/models";
|
||||
location.reload();
|
||||
});
|
||||
const skipBtn = h("button", { class: "btn ghost", onClick: async () => {
|
||||
try { await api.setupComplete(getLang(), false); } catch {}
|
||||
location.hash = "#/dashboard";
|
||||
} }, t("onboarding.skipSetup"));
|
||||
body = h("div", { class: "onb-step" },
|
||||
h("h2", {}, "✅ " + t("onboarding.doneTitle")),
|
||||
h("p", { class: "muted" }, t("onboarding.doneSub")),
|
||||
ready.length
|
||||
? h("div", { class: "card" },
|
||||
h("div", { class: "card-title" }, t("onboarding.willRegister")),
|
||||
...ready.map((j) => h("div", { class: "muted small" }, "• cmf-" + (j.name || j.repo) + (j.size_bytes ? ` · ${fmtBytes(j.size_bytes)}` : "")))
|
||||
)
|
||||
: h("div", { class: "muted small" }, t("onboarding.nothingToRegister")),
|
||||
jobsHost,
|
||||
h("div", { class: "callout" }, t("onboarding.doneHint")),
|
||||
h("div", { class: "actions" },
|
||||
h("button", { class: "btn ghost", onClick: prev }, "← " + t("onboarding.back")),
|
||||
skipBtn,
|
||||
doneBtn
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
mount(root, h("div", { class: "onb" }, header, body));
|
||||
paintJobs(); // fill the freshly built hosts with whatever we already know
|
||||
if (jobs.some((j) => j.state === "running") && !jobsPoll) {
|
||||
jobsPoll = setInterval(refreshJobs, 1500);
|
||||
}
|
||||
}
|
||||
|
||||
loadFeatured().then(() => { render(); refreshJobs(); });
|
||||
render();
|
||||
// cleanup poll on detach
|
||||
const obs = new MutationObserver(() => { if (!root.isConnected && jobsPoll) { clearInterval(jobsPoll); jobsPoll = null; obs.disconnect(); } });
|
||||
obs.observe(document.body, { childList: true, subtree: true });
|
||||
return root;
|
||||
}
|
||||
Vendored
+98
-8
@@ -245,11 +245,7 @@ export async function renderSettings() {
|
||||
try {
|
||||
const r = await api.cmfPortCheck(p);
|
||||
if (r.available) mount(st, h("span", { style: "color:var(--good)" }, "✓ " + t("settings.cmf.portFree")));
|
||||
else if (r.managed) {
|
||||
// held by this gateway's own managed server — replacing the model on
|
||||
// this port is fine, it frees up on restart
|
||||
mount(st, h("span", { style: "color:var(--accent)" }, "ℹ " + t("settings.cmf.portOwn")));
|
||||
} else {
|
||||
else {
|
||||
const kids = [h("span", { style: "color:var(--bad)" }, "✗ " + t("settings.cmf.portBusy"))];
|
||||
if (r.suggested) { const b = h("button", { class: "btn small", type: "button" }, t("settings.cmf.useSuggested", { port: r.suggested })); b.addEventListener("click", () => { portIn.value = r.suggested; mount(st, ""); }); kids.push(b); }
|
||||
mount(st, h("span", {}, ...kids));
|
||||
@@ -257,9 +253,103 @@ export async function renderSettings() {
|
||||
} catch (e) { mount(st, String(e.message || e)); }
|
||||
checkBtn.disabled = false;
|
||||
});
|
||||
const row = h("div", { class: "flex wrap", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), removeBtn, st);
|
||||
const entry = { node: row, read: () => ({ id: idIn.value.trim() || autoId(modelSel.value), model: modelSel.value.trim(), port: parseInt(portIn.value) || 8090, threads: parseInt(threadsIn.value) || 0, gpu: gpuIn.checked }) };
|
||||
|
||||
// ── O(1) Nyström Attention ──
|
||||
// NOT a "reasoning mode". This is a memory-efficient streaming attention
|
||||
// kernel (Nyström/landmark) that replaces standard KV-cache attention,
|
||||
// giving O(1) memory per decode step instead of O(n).
|
||||
const o1Sel = h("select", { style: "width:120px", title: "O(1) Nyström attention layer spec: which layers replace KV-cache with streaming Nyström attention" },
|
||||
h("option", { value: "" }, "O(1) авто"),
|
||||
h("option", { value: "all" }, "O(1) все слои"),
|
||||
h("option", { value: "deep" }, "O(1) глубокие"),
|
||||
h("option", { value: "off" }, "O(1) выкл."));
|
||||
if (sv.o1) o1Sel.value = sv.o1;
|
||||
const o1mIn = h("input", { type: "number", min: "4", placeholder: "32", value: sv.o1_m ?? "", style: "width:55px", title: "Landmark budget (≥4, default 32). More landmarks ≠ better: m=64 measured worse." });
|
||||
const o1wIn = h("input", { type: "number", min: "1", placeholder: "128", value: sv.o1_window ?? "", style: "width:60px", title: "Exact-window width — main quality lever (default 128)" });
|
||||
const o1sinkIn = h("input", { type: "number", min: "0", placeholder: "4", value: sv.o1_sink ?? "", style: "width:50px", title: "Permanent exact sink keys, StreamingLLM discipline (default 4)" });
|
||||
|
||||
// ── Generation parameters ──
|
||||
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: sv.temperature ?? "", style: "width:60px" });
|
||||
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: sv.top_p ?? "", style: "width:60px" });
|
||||
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: sv.max_tokens ?? "", style: "width:75px" });
|
||||
// think_budget = 0 means reasoning disabled (enable_thinking=false);
|
||||
// null = model decides; N>0 = token budget for the <think> block.
|
||||
const noThinkCb = h("input", { type: "checkbox", checked: (sv.think_budget === 0) ? true : null, title: "Отключить режим размышлений (think_budget=0, enable_thinking=false для Qwen3/3.5)" });
|
||||
const thinkIn = h("input", { type: "number", min: "1", placeholder: "авто", value: (sv.think_budget && sv.think_budget > 0) ? sv.think_budget : "", style: "width:75px", title: "Бюджет токенов на размышления (think_budget). Пустое = модель решает сама." });
|
||||
if (sv.think_budget === 0) thinkIn.disabled = true;
|
||||
noThinkCb.addEventListener("change", () => {
|
||||
thinkIn.disabled = noThinkCb.checked;
|
||||
if (noThinkCb.checked) thinkIn.value = "";
|
||||
});
|
||||
const sysIn = h("input", { placeholder: "System prompt (optional)", value: sv.system_prompt || "", style: "flex:1;min-width:200px" });
|
||||
|
||||
// ── MTP (Multi-Token Prediction): speculative decoding, NOT reasoning ──
|
||||
const skipMtpCb = h("input", { type: "checkbox", checked: sv.skip_mtp ? true : null, title: "Disable MTP speculative decoding (env CMF_MTP=0)" });
|
||||
|
||||
// ── Settings drawer ──
|
||||
const drawerStyle = "display:none;width:100%;margin-top:6px;padding:10px 12px;background:var(--bg-subtle);border-radius:6px;border:1px solid var(--border)";
|
||||
const groupStyle = "display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:4px 0";
|
||||
const labelStyle = "font-size:11px;color:var(--text-muted);font-weight:600;text-transform:uppercase;letter-spacing:0.5px;width:100%;margin-bottom:2px";
|
||||
|
||||
const drawer = h("div", { style: drawerStyle },
|
||||
// Group 1: O(1) Nyström Attention
|
||||
h("div", { style: labelStyle }, "O(1) Nyström Attention"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Слои:"), o1Sel,
|
||||
h("span", { class: "small muted" }, "m:"), o1mIn,
|
||||
h("span", { class: "small muted" }, "Window:"), o1wIn,
|
||||
h("span", { class: "small muted" }, "Sink:"), o1sinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 2: Generation
|
||||
h("div", { style: labelStyle }, "Генерация"),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "Temp:"), tempIn,
|
||||
h("span", { class: "small muted" }, "Top-P:"), topPIn,
|
||||
h("span", { class: "small muted" }, "Max токенов:"), maxTokIn
|
||||
),
|
||||
h("div", { style: groupStyle },
|
||||
h("span", { class: "small muted" }, "System prompt:"), sysIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 3: Reasoning (thinking)
|
||||
h("div", { style: labelStyle }, "Режим размышлений"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "Для моделей Qwen3/3.5: enable_thinking=false — модель отвечает напрямую без блока <think>" }, noThinkCb, "Отключить размышления"),
|
||||
h("span", { class: "small muted" }, "Think budget:"), thinkIn
|
||||
),
|
||||
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
|
||||
// Group 4: Runtime
|
||||
h("div", { style: labelStyle }, "Runtime"),
|
||||
h("div", { style: groupStyle },
|
||||
h("label", { class: "check small", title: "MTP — Multi-Token Prediction (спекулятивное декодирование). Отключение замедляет генерацию, но стабилизирует output." }, skipMtpCb, "Отключить MTP")
|
||||
)
|
||||
);
|
||||
|
||||
const gearBtn = h("button", { class: "btn small", type: "button", title: "⚙ Настройки модели" }, "⚙");
|
||||
gearBtn.addEventListener("click", () => {
|
||||
drawer.style.display = drawer.style.display === "none" ? "block" : "none";
|
||||
});
|
||||
|
||||
const row = h("div", { class: "flex wrap cmf-server-row", style: "gap:6px;align-items:center;margin:6px 0" },
|
||||
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), gearBtn, removeBtn, st, drawer);
|
||||
const entry = { node: row, read: () => ({
|
||||
id: idIn.value.trim() || autoId(modelSel.value),
|
||||
model: modelSel.value.trim(),
|
||||
port: parseInt(portIn.value) || 8090,
|
||||
threads: parseInt(threadsIn.value) || 0,
|
||||
gpu: gpuIn.checked,
|
||||
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
|
||||
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
|
||||
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
|
||||
think_budget: noThinkCb.checked ? 0 : (thinkIn.value !== "" ? parseInt(thinkIn.value) : null),
|
||||
o1: o1Sel.value || null,
|
||||
o1_m: o1mIn.value !== "" ? parseInt(o1mIn.value) : null,
|
||||
o1_window: o1wIn.value !== "" ? parseInt(o1wIn.value) : null,
|
||||
o1_sink: o1sinkIn.value !== "" ? parseInt(o1sinkIn.value) : null,
|
||||
skip_mtp: skipMtpCb.checked,
|
||||
system_prompt: sysIn.value.trim() || null,
|
||||
}) };
|
||||
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
|
||||
serverRows.push(entry);
|
||||
serversWrap.appendChild(row);
|
||||
|
||||
Reference in New Issue
Block a user