Compare commits

..
58 Commits
Author SHA1 Message Date
infosave2007 ad438a5dcb chore: bump v0.2.36
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-07-25 12:01:52 +03:00
infosave2007 45def5c6b6 fix(docker): model downloads failed in containers (curl exit 23, glibc mismatch)
- pin builder and runtime to the same Debian release (trixie): the floating
  rust:1-slim builder drifted to a newer glibc than the bookworm runtime, so
  freshly built images crashed on start with 'GLIBC_2.39 not found'
- install curl in the runtime image — ready-.cmf imports download with it
- keep all gateway state under /app/data (CORTIQ_GATEWAY_HOME) and run as
  root so bind-mounted host folders on Docker Desktop are writable; pass
  --user 10001 to harden when not bind-mounting
- persist /app/data via a cortiq-data volume in docker-compose
- fail imports immediately with the path and reason when the models dir
  cannot be created, and annotate curl exit codes 23/22 with readable hints
- exclude local models/*.cmf from the docker build context
2026-07-25 12:01:38 +03:00
infosave2007 5d8455b4a0 v0.2.35
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-07-23 21:25:45 +03:00
infosave2007 b255c60b0b chore: update lockfile 2026-07-23 16:41:58 +03:00
infosave2007 a75cacd0ec chore: bump v0.2.34
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-07-23 16:41:08 +03:00
infosave2007 850a54acdf fix: enable_thinking not sent in streaming path
Two bugs:
1. run_stream() never merged per-model/server config (think_budget,
   temperature, etc.) into the request — unlike run_inner(). Now both
   paths apply the same config merge.
2. to_wire() was missing the enable_thinking bool conversion line
   (lost in a prior revert). Restored: think_budget=0 → enable_thinking=false.
2026-07-23 16:34:49 +03:00
infosave2007 22110ac524 style(admin): apply cargo fmt to fix CI 2026-07-22 19:04:12 +03:00
infosave2007 b3b37d27f4 chore: update Cargo.lock 2026-07-22 18:59:46 +03:00
infosave2007 a7c8ed72c7 feat(admin): sync openrouter capabilities with emojis
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-07-22 18:59:31 +03:00
infosave2007 f2e6047bff fix: satisfy clippy in CMF import detection 2026-07-22 11:54:51 +03:00
infosave2007 0b28aa3d59 release: v0.2.32 mobile admin console 2026-07-22 11:45:00 +03:00
infosave2007 1414ad1b52 release: v0.2.31 - O(1) attention knobs, reasoning toggle & admin token persistence 2026-07-22 11:12:40 +03:00
infosave2007 94c8551206 feat(release): v0.2.30 — per-model generation, O(1) attention, MTP & reasoning parameters 2026-07-22 10:40:44 +03:00
infosave2007 6ecc709fd2 chore(release): v0.2.29 — fix duplicate managed model registration conflict 2026-07-21 23:39:47 +03:00
infosave2007 1661c19550 chore(release): v0.2.28 — auto-detect ready CMF quant & update section title 2026-07-21 23:35:14 +03:00
infosave2007 66ee32b233 chore(release): v0.2.27 — auto-spawn local CMF server on model registration 2026-07-21 23:15:33 +03:00
infosave2007 f6ee045989 chore(release): v0.2.26 — robust multi-GB streaming downloads & ready CMF quants 2026-07-21 23:08:02 +03:00
infosave2007 59e2eb3215 chore(release): v0.2.25 — direct stream download for ready CMF repos 2026-07-21 22:51:58 +03:00
infosave2007 5b743a4691 chore(release): v0.2.24 — new quantizations (q4t, q1t, q1s, q1p, q1) and featured CMF models 2026-07-21 22:45:43 +03:00
infosave2007 c59ed9c949 feat(import): O(1) attention options in Advanced (cortiq >= 0.2.0)
- New Advanced controls on the import page: O(1) attention spec (off / all /
  deep N / explicit layer list) plus the --o1-m / --o1-window / --o1-sink knobs
  with validated defaults (32 / 128 / 4) and explanatory hints.
- Backend passes --o1* to the native `cortiq convert`; fails fast with clear
  messages when combined with Python-only options, used on GGUF imports, or
  when the installed cortiq predates 0.2.0 (the hint needs the new runtime).
- The hint is recorded in the .cmf header (weights byte-identical); the runtime
  applies it at load. Verified end-to-end: converted a model with --o1 all and
  the runtime logged "o1 nystrom attention: 2/2 layer(s)".
2026-07-14 10:16:54 +03:00
infosave2007 b55e72789f feat(dashboard): windowed stats per range + clear-logs button
- Stats snapshot now computes the KPI totals AND the breakdown from the
  in-window per-minute buckets (previously they used all-time accumulators, so
  switching 1h/24h/7d changed only the chart, not the numbers). Buckets now
  carry failovers + per-group breakdown; Prometheus keeps all-time counters.
- New DELETE /admin/api/stats clears all in-memory stats and truncates the
  JSONL log; the dashboard gets a "Clear logs" button (with confirm).
2026-07-09 17:07:05 +03:00
infosave2007 0d1283c3fa docs(settings): add help tooltips to routing-input fields (strategy, max chars, cache TTL, profile) 2026-07-09 16:49:16 +03:00
infosave2007 0105a43ef5 feat(models): auto-derive id from model name; actionable probe auth errors
- The model id is now auto-filled from the model name as you type, and if left
  blank it's derived on save (slug of the last path segment, e.g.
  "qwen/qwen3.7-max" → "qwen3-7-max") — saving no longer errors on a blank id.
- probe/test auth failures (401 / "User not found" / "No auth") now append a
  hint to check the API key (for OpenRouter: use an inference key, not a
  provisioning/management key).
2026-07-09 16:30:48 +03:00
infosave2007 b8d48c31f7 fix(models): Add button, probe of managed models, delete list refresh
- ui.js: mount() now wraps non-node children in text nodes (like h()), so
  mount(el, "") no longer throws in appendChild — this is what broke the
  "Add model" form (applyDefaults cleared a status via mount(x,"")).
- probe_model: resolve managed local models too (they are synthesized into the
  registry, not [[models]]) — probing them no longer returns "model not found".
- models.js: await reload() after delete so the row is removed immediately.
2026-07-09 16:13:07 +03:00
infosave2007 196ff52f0f feat(models): friendlier add-model form — OpenRouter/LM Studio, direct key, test & load
- Providers: add openrouter (https://openrouter.ai/api/v1) and lmstudio
  (http://localhost:1234/v1), both OpenAI-compatible. Selecting a provider now
  auto-fills its verified base URL, cost tier and default capabilities.
- API key is typed directly in the form and stored securely as a secret under the
  provider's env name — no manual env var to manage.
- New POST /admin/api/provider/models validates the key and lists the provider's
  models; a "Test & load models" button fills a searchable dropdown. The model
  field now sits below the key (some providers list models from the key).
- Fuller capability set: tools, vision, audio, reasoning, json, caching.
2026-07-09 15:45:21 +03:00
infosave2007 e25814a54c feat(settings): pick a local model from a dropdown of existing .cmf files
- New GET /admin/api/cmf/files lists the .cmf files (and tensor-dir .cmf) under
  models_dir, with sizes.
- Settings → Local models: the per-row model field is now a dropdown of those
  files instead of a hand-typed path, and auto-fills the id from the filename.
2026-07-09 15:25:30 +03:00
infosave2007 0c20eb34f0 chore(release): v0.2.8 — multiple managed local models
- The gateway can now run several local .cmf models at once: a [[cmf.servers]]
  list, one supervised `cortiq serve` per entry on its own port, each registered
  in the pool and independently routable/pinnable. The single legacy
  local_model/local_port still works (back-compat via effective_servers()).
- Settings → Local models is now a rows editor (id · path · port + check · threads
  · GPU) with add/remove and per-server serving status.
- Deleting a managed model stops just its server and removes its file — unless
  another model still points at the same file (shared-file guard).
2026-07-09 12:00:10 +03:00
infosave2007 f8c1bb0914 chore(release): v0.2.7 — honest pins, visible managed model, physical delete
- A pinned playground model that is unavailable now errors clearly instead of
  silently answering from the managed local model (route_source no longer lies).
- The managed CMF model is surfaced in GET /admin/api/models (marked "managed",
  shown read-only in the UI) so the real running model is visible and pinnable
  without a hand-maintained [[models]] entry; a fresh install still lists nothing.
- Deleting a model now removes its local .cmf physically (file or tensor-dir);
  deleting the managed model also stops the server and clears the [cmf] config.
  Routing is stripped of the deleted id so the config still validates.
2026-07-08 21:14:03 +03:00
infosave2007 0f299b77e6 chore: fix clippy lints to green the lint CI
- import.rs / promotion.rs: sort_by(|a,b| b.k.cmp(&a.k)) -> sort_by_key(Reverse(k))
- pipeline.rs: skip_while(!p).next() -> find(p)
Behaviour-preserving; completes the CI fix started by the cargo fmt commit.
2026-07-08 20:09:54 +03:00
infosave2007 7c49510c46 chore: cargo fmt — fix lint 'Format' CI job (red since v0.2.3)
Whole-repo rustfmt drift; behaviour unchanged (whitespace/layout only).
All test platforms already passed; this greens the lint job.
2026-07-08 19:32:35 +03:00
infosave2007 31da84cc45 chore(release): v0.2.6 — Settings UX, empty pool by default, CMF port check
- Settings: "?" help tooltips on every option; router key auto-probes the
  connection right after saving (no more hanging "unavailable")
- Local models card reworked into 3 clear steps (install cortiq → add model →
  run locally); removed the hand-entered .cmf path / pool-id fields — models
  are added via the Import tab and the managed server registers them
- Fresh install ships an EMPTY model pool; the local .cmf is auto-registered by
  the managed CMF server, so it needs no static [[models]] entry
- New: port-availability check — GET /admin/api/cmf/port + "Check port" button
  with next-free-port suggestion. bind()+connect() probe so Docker-proxied /
  dual-stack ports (e.g. 8081) are correctly detected as occupied
- /admin/api/cmf now reports the serving model + port; card shows "▶ serving …"
- On-demand runtime install: POST /admin/api/cmf/install
2026-07-08 19:07:24 +03:00
infosave2007 749a47ce9b chore: ship the full working frontend + backend
The crates.io/release build shipped a STALE frontend (committed web/js was behind
the working tree), so the admin console hung on load. Commit the current working
web/js (app, api, dashboard, login, import view, i18n de/es/fr/tr/zh, css) and the
backend WIP (import.rs, admin/assets.rs) so the released console matches the dev
build. Verified: clean `cargo build --release`, empty-pool start, working
dashboard served.
2026-07-08 17:58:31 +03:00
infosave2007 d2b1d50eb8 chore(release): v0.2.4 2026-07-08 17:39:43 +03:00
infosave2007 d48349a144 feat: fresh install starts with an empty model pool
- config.rs: validate() no longer errors on zero models — an empty pool is
  allowed (warns instead), so the admin console comes up on a fresh install and
  the user adds models in /admin -> Models (or points [cmf] at a local .cmf).
- gateway.example.toml: ships an EMPTY pool (no seed cmf-local/local-embed, empty
  routing), so `cp example -> gateway.toml && run` shows an empty list — what a
  new user expects.
2026-07-08 17:39:43 +03:00
infosave2007 f05d26bfbb fix: add promotion + router-status modules referenced by the CMF integration
The v0.2.3 CMF-integration commit referenced crate::promotion (main.rs),
router_client::RouterLastStatus (state.rs) and router_client::classify_status
(admin), but those definitions were not part of the commit, so a clean build
failed (E0583/E0432/E0425). Add the two source files; cargo build --release now
compiles.
2026-07-08 16:59:58 +03:00
infosave2007 39f1fcb600 chore(release): v0.2.3 2026-07-08 16:39:30 +03:00
infosave2007 42377a75d4 config: example ships a local CMF model, drop cloud placeholders
The example config pool now shows one local CMF model (served by the managed
cortiq server) instead of local-qwen/gpt-4o-mini/claude-opus placeholders, with
routing pointing at it, and a documented [cmf] section (manage_server, threads,
gpu, ...). Add cloud/local providers by copying the [[models]] shape.
2026-07-08 16:39:30 +03:00
infosave2007 6cf2608fc8 admin: surface CMF worker threads + GPU toggle in Settings
Expose the two speed knobs in the /admin Settings panel instead of TOML-only:
- [cmf].gpu (new) -> exports CMF_GPU=1 to the managed cortiq serve (Metal/wgpu).
  On Apple Silicon decode is memory-bandwidth bound so this is ~neutral for
  generation (helps prefill); off by default.
- [cmf].threads (added earlier) now has a UI number input.

- config.rs: CmfCfg.gpu bool (serde default false).
- cmf_runtime.rs: spawn_serve sets CMF_GPU=1 when gpu is on.
- admin/mod.rs: put_settings flags needs_restart on threads/gpu change (the
  server is spawned once at startup with those env vars).
- settings.js: threads input + gpu checkbox + save payload + speed hint.
- i18n en/ru: threads/gpu/speedNote keys (other locales fall back to en, as the
  whole cmf section already does there).
2026-07-08 16:18:31 +03:00
infosave2007 e6d1a630c9 cmf: parallelise the managed local server (CMF_THREADS)
The gateway spawned `cortiq serve` with no thread hint, so the runtime ran the
decode loop single-threaded (~0.25 tok/s on a 15B CPU model). Export CMF_THREADS
from a new [cmf].threads config field (default 8) in spawn_serve.

Measured: 0.25 -> 0.81 tok/s (~3.2x) on qwopus-vbit7 (15B). Scaling is sublinear
(GatedDeltaNet state recurrence is serial per token, Amdahl-bound); a real jump
needs the GPU path.
2026-07-08 16:18:31 +03:00
infosave2007 8dd77ca7ff feat(cmf): integrate CMF as a managed local model server
Run models in the CMF (.cmf) format locally, via the native `cortiq`
CLI from crates.io — no Python and no external model server.

- cmf_runtime: install/update `cortiq-cli` from crates.io, spawn and
  health-check `cortiq serve` as a managed local backend.
- import: model-import jobs shell out to the native `cortiq convert`
  (HF repo -> .cmf, quant mapped Q8_2F->q8_2f / Q8_ROW->q8 / Q4->q4);
  the Python converter is used only for advanced/exotic options.
- pipeline: local-first routing — serve from the local CMF model when
  the router is disabled/unreachable, with a "local only" hard switch.
- registry/state: register the local CMF server as an OpenAI provider
  and track local candidates.
- config: data-dir resolution + CmfCfg (auto-install/update, managed
  server, host/port/model) and relaxed validation for the managed model.
- admin + settings UI (en/ru): "Local models (CMF)" settings card.
2026-07-08 16:18:31 +03:00
infosave2007 630149d141 docs: add docker-compose quickstart (gateway + Ollama) and GHCR badge
- docker-compose.yml + config/gateway.compose.toml (routes to a local Ollama service)
- README (EN+RU): ghcr badge + Docker Compose section
2026-07-01 13:40:24 +03:00
infosave2007 85fc508f04 chore(release): v0.2.2
Benchmarks (latency + accuracy), router docs + LAUNCH1 promo, Docker image on ghcr.
2026-07-01 11:28:56 +03:00
infosave2007 010171e05c feat: Docker image + ghcr publishing
- multi-stage Dockerfile (non-root, embeds SPA, default config, ~160MB) + .dockerignore
- .github/workflows/docker.yml: push to ghcr.io/infosave2007/cortiq-gateway
  (master -> :edge/:sha amd64; tags -> :X.Y.Z/:latest, linux/amd64+arm64)
- README (EN+RU): docker run instructions; CHANGELOG
- verified locally: image builds, container serves /healthz + /admin
2026-07-01 11:23:09 +03:00
infosave2007 39c56145ed docs: document the allaigate router + launch promo
- docs/ROUTER.md + docs/ROUTER.ru.md: what the semantic router does, hosted vs
  self-hosted, config, API contract, pricing, and the LAUNCH1 $1 Starter promo
- README (EN+RU): "The router (allaigate)" section with the promo callout
2026-07-01 10:35:29 +03:00
infosave2007 17c8186383 docs: add task-type accuracy benchmark (router vs keyword heuristic)
- bench/tasks.jsonl (37 natural-language prompts, 7 task types) + bench/accuracy.py
- measured on the live allaigate router: 100% (37/37) vs keyword heuristic 32%
- BENCHMARKS.md accuracy section; README (EN+RU) perf+accuracy tables and test matrix
- run.sh runs accuracy when CORTIQ_ROUTER_KEY is set
2026-07-01 07:45:16 +03:00
infosave2007 e773e18efd docs: add test/verification matrix to README (EN+RU) 2026-06-30 18:31:13 +03:00
infosave2007 6466006de6 docs: add latency benchmark vs LiteLLM/Portkey
- bench/ reproducible harness: instant mock backend + Apache Bench (ab -k -r)
- measures gateway overhead in passthrough mode (apples-to-apples)
- BENCHMARKS.md: methodology, caveats, results (cortiq ~57k rps p99 1ms vs
  Portkey ~5.8k p99 9ms vs LiteLLM 4-workers ~1.2k p99 59ms)
- README (EN+RU): Performance section + highlight
2026-06-30 18:11:26 +03:00
infosave2007 6a25d8ac08 feat: finish remaining inbound protocols (models, completions, native /route)
- GET /v1/models: pool + virtual cortiq-auto
- POST /v1/completions (legacy): non-stream + streaming text_completion
- POST /route: native passthrough returning the routing decision
- all inbound protocol adapters now implemented; admin meta + README/CHANGELOG updated
- bump to 0.2.1
2026-06-30 17:42:45 +03:00
infosave2007 e486617e51 chore(release): v0.2.0
Streaming (SSE), Anthropic provider/inbound, embeddings, semantic cache, MCP server.
2026-06-30 17:17:04 +03:00
infosave2007 729a8bfbba feat: MCP (Model Context Protocol) server
- POST /mcp (JSON-RPC 2.0): initialize, ping, tools/list, tools/call; notifications -> 202
- tools: cortiq_route (classify -> routing decision) and cortiq_chat (route -> answer)
- gated by protocols.mcp; admin meta marks it implemented
- docs: README/README.ru protocol tables, highlights and status updated for
  streaming, Anthropic, embeddings, semantic cache and MCP
2026-06-30 17:12:55 +03:00
infosave2007 5759de153b feat: semantic (embedding-based) response cache
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by
  routing signature; tracks hit rate and model cost saved
- pipeline: embed routing text, look up before the model call (route_source=cache on
  hit), insert successful responses; stats record cache hits
- config [cache] + admin settings card; dashboard shows hit-rate and savings
- embedding model resolved from the pool (or first kind="embedding")
2026-06-30 17:04:53 +03:00
infosave2007 bf8534f13d feat: Anthropic provider/inbound + embeddings
- Anthropic outbound provider: /v1/messages, system extraction, x-api-key +
  anthropic-version; streaming translates Anthropic SSE -> unified OpenAI wire format
- Anthropic inbound adapter (POST /v1/messages): non-streaming JSON + streaming SSE
  (OpenAI -> Anthropic event translation)
- Embeddings: POST /v1/embeddings inbound + Provider::embed(); embedding models
  resolvable from the pool (registry no longer skips them)
- protocols mounted always, gated on live flags; admin meta marks them implemented
2026-06-30 16:50:22 +03:00
infosave2007 70d8cde258 feat: streaming (SSE) end-to-end
- Provider::chat_stream; OpenAI provider streams via stream_options.include_usage
- pipeline.run_stream: shared route/select, failover before first byte
- /v1/chat/completions forwards SSE verbatim with X-Cortiq-* headers; usage tapped for stats
- admin /test/stream + live streaming in Playground (stream toggle, 7 langs)
2026-06-30 16:37:35 +03:00
infosave2007 3d71b03788 ci: build Windows binaries; add cross-platform CI test matrix
- release.yml: add x86_64-pc-windows-msvc target, OS-aware packaging (.exe + .zip)
- ci.yml: split into lint (fmt/clippy) + test matrix (ubuntu/windows/macos)
- use getrandom for the admin/key token CSPRNG (secure on Windows too)
2026-06-30 14:56:54 +03:00
infosave2007 492953d4c2 docs: add crates.io install instructions to both READMEs 2026-06-30 14:11:07 +03:00
infosave2007 3b0231b32d ci(release): use env-mapped token guard for crates.io publish step 2026-06-30 13:54:35 +03:00
infosave2007 fd2f2a165b chore(release): prepare v0.1.0 for crates.io
- bump version to 0.1.0; add crates.io metadata (include list, homepage, rust-version)
- add .github/workflows/release.yml: on tag v* → publish to crates.io + GitHub Release with binaries
- README: absolute links/images and badges so it renders on crates.io; add install section
- CHANGELOG: 0.1.0
2026-06-30 13:47:42 +03:00
infosave2007 cbc385f6c7 feat: universal LLM gateway with embedded multilingual admin console
- OpenAI-compatible gateway with intelligent routing via cortiq-router (allaigate)
- Embedded build-free admin SPA at /admin: 7 languages (en/ru/de/fr/es/zh/tr),
  light/dark themes, hot config reload (no restart)
- Admin API, per-request statistics + Prometheus /metrics, secret store, playground
- English documentation (primary) with Russian translations (*.ru.md)
2026-06-30 10:51:37 +03:00
32 changed files with 1530 additions and 127 deletions
+2
View File
@@ -1,4 +1,6 @@
target
models
*.cmf
.git
.github
docs
+66
View File
@@ -3,8 +3,74 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.36] - 2026-07-25
### Fixed
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.
## [0.2.35] - 2026-07-23
### Fixed
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.
## [0.2.34] - 2026-07-23
### Added
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.
### Fixed
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.
## [0.2.31] - 2026-07-22
### Added
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.
### Fixed
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.
## [0.2.30] - 2026-07-22
### Added
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.
### Fixed
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.
### Fixed
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").
### Fixed
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.
### Fixed
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.
### Fixed
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.
### Added
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).
## [Unreleased]
### Added
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
gateway + a local Ollama model in one command. GHCR badge in the READMEs.
Generated
+1 -1
View File
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]]
name = "cortiq-gateway"
version = "0.2.12"
version = "0.2.36"
dependencies = [
"anyhow",
"arc-swap",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "cortiq-gateway"
version = "0.2.12"
version = "0.2.36"
edition = "2021"
rust-version = "1.75"
license = "Apache-2.0"
+13 -5
View File
@@ -1,5 +1,7 @@
# ---- build stage ------------------------------------------------------------
FROM rust:1-slim AS builder
# pin builder and runtime to the same Debian release — a floating `rust:1-slim`
# drifts to a newer glibc than the runtime image and the binary refuses to start
FROM rust:1-slim-trixie AS builder
WORKDIR /app
# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
COPY Cargo.toml Cargo.lock ./
@@ -8,18 +10,24 @@ COPY web ./web
RUN cargo build --release
# ---- runtime stage ----------------------------------------------------------
FROM debian:bookworm-slim
FROM debian:trixie-slim
# curl is required at runtime — model imports download ready .cmf files with it
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& rm -rf /var/lib/apt/lists/* \
&& useradd -r -u 10001 cortiq \
&& mkdir -p /app/config
&& mkdir -p /app/config /app/data/models
COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
# ship the example as the default config so `docker run` works out of the box
COPY config/gateway.example.toml /app/config/gateway.toml
RUN chown -R 10001:10001 /app
WORKDIR /app
USER cortiq
# all gateway state (downloaded models, stats, secrets) lives under /app/data —
# mount a volume there to keep it across container recreates
ENV CORTIQ_GATEWAY_HOME=/app/data
# runs as root so writes to bind-mounted host folders (Docker Desktop on
# Windows/macOS) don't fail with permission errors (curl exit code 23);
# add `--user 10001` to docker run to harden when not using host bind mounts
EXPOSE 9000
ENTRYPOINT ["cortiq-gateway"]
CMD ["--config", "/app/config/gateway.toml"]
+6 -2
View File
@@ -54,7 +54,8 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
```bash
# Docker (GitHub Container Registry)
docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
-v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
-v "$PWD/config:/app/config" -v cortiq-data:/app/data \
ghcr.io/infosave2007/cortiq-gateway:latest
# → admin console at http://localhost:9000/admin
# from crates.io
@@ -67,7 +68,10 @@ cargo build --release # ./target/release/cortiq-gateway
```
The image ships a default config and runs out of the box; mount `/app/config` (with a
`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
`gateway.toml`) to use your own. Downloaded models and stats live in `/app/data` — mount
a volume there so they survive container recreates (a host folder works too, e.g.
`-v "$PWD/models:/app/data/models"`). Tags: `:latest`, `:X.Y.Z`, `:edge`
(linux/amd64 + arm64).
**Docker Compose** — gateway + a local Ollama model in one command:
+5
View File
@@ -17,6 +17,10 @@ services:
CORTIQ_ROUTER_KEY: ${CORTIQ_ROUTER_KEY:-}
volumes:
- ./config/gateway.compose.toml:/app/config/gateway.toml:ro
# downloaded .cmf models + stats persist here across recreates; swap the
# named volume for a host folder (e.g. ./models:/app/data/models) if you
# want the files visible next to the compose file
- cortiq-data:/app/data
depends_on:
- ollama
restart: unless-stopped
@@ -30,4 +34,5 @@ services:
restart: unless-stopped
volumes:
cortiq-data:
ollama:
+164 -19
View File
@@ -165,7 +165,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
"/admin/api/secrets",
get(list_secrets).put(set_secret).delete(clear_secret),
)
.route("/admin/api/stats", get(get_stats))
.route("/admin/api/stats", get(get_stats).delete(clear_stats))
.route("/admin/api/shadow", get(get_shadow))
.route("/admin/api/hf/search", get(hf_search))
.route("/admin/api/import", get(list_imports).post(start_import))
@@ -467,16 +467,111 @@ async fn provider_models(
}
// OpenAI + Anthropic both return { "data": [ { "id": ... } ] }.
let v: Value = serde_json::from_str(&body).unwrap_or(json!({}));
let mut models: Vec<String> = v["data"]
let mut models: Vec<Value> = v["data"]
.as_array()
.map(|arr| {
arr.iter()
.filter_map(|m| m["id"].as_str().map(|s| s.to_string()))
.filter_map(|m| {
let id = m["id"].as_str()?.to_string();
let mut obj = json!({ "id": id });
let mut caps = vec![];
if let Some(pricing) = m.get("pricing").filter(|v| !v.is_null()) {
if let Some(prompt) = pricing
.get("prompt")
.and_then(|s| s.as_str())
.and_then(|s| s.parse::<f64>().ok())
{
obj["price_in"] = json!(prompt * 1_000_000.0);
}
if let Some(comp) = pricing
.get("completion")
.and_then(|s| s.as_str())
.and_then(|s| s.parse::<f64>().ok())
{
obj["price_out"] = json!(comp * 1_000_000.0);
}
if pricing
.get("input_cache_read")
.filter(|v| !v.is_null())
.is_some()
|| pricing
.get("input_cache_write")
.filter(|v| !v.is_null())
.is_some()
{
caps.push("caching");
}
}
if let Some(params) = m.get("supported_parameters").and_then(|p| p.as_array()) {
let has_tools = params.iter().any(|p| {
p.as_str() == Some("tools") || p.as_str() == Some("tool_choice")
});
if has_tools {
caps.push("tools");
}
let has_json = params.iter().any(|p| p.as_str() == Some("response_format"));
if has_json {
caps.push("json");
}
let has_reasoning = params.iter().any(|p| {
p.as_str() == Some("reasoning")
|| p.as_str() == Some("include_reasoning")
});
if has_reasoning || m.get("reasoning").filter(|v| !v.is_null()).is_some() {
caps.push("reasoning");
}
}
if let Some(arch) = m.get("architecture").filter(|v| !v.is_null()) {
if let Some(inputs) =
arch.get("input_modalities").and_then(|i| i.as_array())
{
if inputs.iter().any(|i| i.as_str() == Some("image")) {
caps.push("vision");
}
if inputs.iter().any(|i| i.as_str() == Some("audio")) {
caps.push("audio");
}
}
}
if !caps.is_empty() {
obj["caps"] = json!(caps);
}
Some(obj)
})
.collect()
})
.unwrap_or_default();
models.sort();
ok(json!({ "ok": true, "count": models.len(), "models": models }))
models.sort_by(|a, b| a["id"].as_str().cmp(&b["id"].as_str()));
let mut resp = json!({ "ok": true, "count": models.len(), "models": models });
if b.provider == "openrouter" {
if let Some(k) = key {
if let Ok(cred_resp) = client
.get("https://openrouter.ai/api/v1/credits")
.header("authorization", format!("Bearer {k}"))
.send()
.await
{
if let Ok(cred_body) = cred_resp.text().await {
let cred_v: Value = serde_json::from_str(&cred_body).unwrap_or(json!({}));
if let Some(data) = cred_v.get("data") {
let total = data
.get("total_credits")
.and_then(|v| v.as_f64())
.unwrap_or(0.0);
let usage = data
.get("total_usage")
.and_then(|v| v.as_f64())
.unwrap_or(0.0);
resp["balance"] = json!(total - usage);
}
}
}
}
}
ok(resp)
}
async fn meta() -> ApiResult {
@@ -1197,6 +1292,13 @@ async fn get_stats(State(state): State<SharedState>, Query(q): Query<StatsQuery>
ok(snap)
}
/// Clear all recorded stats/logs (in-memory aggregates, series, recent requests)
/// and truncate the JSONL log — the dashboard "Clear logs" button.
async fn clear_stats(State(state): State<SharedState>) -> ApiResult {
state.stats.clear();
ok(json!({ "ok": true }))
}
/// Self-warming shadow loop: per-task-type promotion state, Wilson-LB of the
/// judged pass-rate, and whether the local model is serving it yet.
async fn get_shadow(State(state): State<SharedState>) -> ApiResult {
@@ -1297,21 +1399,64 @@ async fn register_import(State(state): State<SharedState>, Path(job): Path<Strin
.unwrap_or_else(|| "cmf-local".into());
let id = format!("cmf-{base}");
let mut cfg = current_cfg(&state);
if cfg.models.iter().any(|m| m.id == id) {
return Err(ApiError::bad(format!("model '{id}' already registered")));
let is_in_models = cfg.models.iter().any(|m| m.id == id);
let is_in_servers = cfg
.cmf
.servers
.iter()
.any(|s| s.id == id || s.model == j.output);
if is_in_models || is_in_servers {
return Err(ApiError::bad(format!("model '{id}' is already registered")));
}
if cfg.cmf.manage_server {
let host = if cfg.cmf.local_host.trim().is_empty() {
"127.0.0.1"
} else {
&cfg.cmf.local_host
};
let base_port = cfg.cmf.servers.iter().map(|s| s.port).max().unwrap_or(8089);
let port = next_free_port(host, base_port).unwrap_or_else(|| base_port + 1);
cfg.cmf.servers.push(crate::config::CmfServer {
id: id.clone(),
model: j.output.clone(),
port,
threads: cfg.cmf.threads,
gpu: cfg.cmf.gpu,
temperature: None,
top_p: None,
max_tokens: None,
think_budget: None,
system_prompt: None,
o1: None,
o1_m: None,
o1_window: None,
o1_sink: None,
skip_mtp: false,
});
} else {
cfg.models.push(crate::config::ModelCfg {
id: id.clone(),
provider: "openai".into(),
base_url: cfg.cmf.cortiq_server_url.clone(),
model: base,
cost_tier: "local".into(),
price_in: 0.0,
price_out: 0.0,
kind: "chat".into(),
api_key_env: None,
caps: Vec::new(),
temperature: None,
top_p: None,
max_tokens: None,
think_budget: None,
system_prompt: None,
o1: None,
skip_mtp: false,
});
}
cfg.models.push(crate::config::ModelCfg {
id: id.clone(),
provider: "openai".into(),
base_url: cfg.cmf.cortiq_server_url.clone(),
model: base,
cost_tier: "local".into(),
price_in: 0.0,
price_out: 0.0,
kind: "chat".into(),
api_key_env: None,
caps: Vec::new(),
});
state.reload(cfg)?;
ok(json!({ "ok": true, "model_id": id }))
}
+23 -3
View File
@@ -238,7 +238,7 @@ fn parse_ver(s: &str) -> (u32, u32, u32) {
)
}
fn version_lt(a: &str, b: &str) -> bool {
pub(crate) fn version_lt(a: &str, b: &str) -> bool {
parse_ver(a) < parse_ver(b)
}
@@ -290,11 +290,31 @@ fn spawn_serve(server: &CmfServer, cfg: &CmfCfg) -> Result<Child, String> {
if server.threads > 1 {
cmd.env("CMF_THREADS", server.threads.to_string());
}
// Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
// on unified memory, so mostly a prefill win; opt-in.
if server.gpu {
cmd.env("CMF_GPU", "1");
}
// O(1) Nyström attention: layers spec + sub-parameters.
if let Some(o1) = &server.o1 {
let s = o1.trim();
if !s.is_empty() {
// Pass even "off" so it explicitly overrides a file converter hint.
cmd.arg("--o1").arg(s);
}
}
if let Some(m) = server.o1_m {
cmd.arg("--o1-m").arg(m.to_string());
}
if let Some(w) = server.o1_window {
cmd.arg("--o1-window").arg(w.to_string());
}
if let Some(sink) = server.o1_sink {
cmd.arg("--o1-sink").arg(sink.to_string());
}
// MTP (Multi-Token Prediction) speculative decoding: the engine reads
// CMF_MTP env; "0" disables. There is no CLI flag for this.
if server.skip_mtp {
cmd.env("CMF_MTP", "0");
}
cmd.stdout(Stdio::null())
.stderr(Stdio::null())
.kill_on_drop(true)
+66 -2
View File
@@ -83,6 +83,36 @@ pub struct CmfServer {
/// Offload matvecs to the GPU (CMF_GPU=1).
#[serde(default)]
pub gpu: bool,
/// Default generation temperature (0.0 .. 2.0).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub temperature: Option<f32>,
/// Default nucleus sampling threshold (0.0 .. 1.0).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub top_p: Option<f32>,
/// Default maximum completion tokens.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_tokens: Option<u32>,
/// Reasoning / thinking token budget.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub think_budget: Option<u32>,
/// Default system prompt prepended if request lacks one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub system_prompt: Option<String>,
/// O(1) Nyström attention layer spec (all | deepN | i,j,k | off).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub o1: Option<String>,
/// O(1) landmark budget (default 32; ≥4).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub o1_m: Option<u32>,
/// O(1) exact-window width (default 128).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub o1_window: Option<u32>,
/// O(1) permanent exact sink keys (default 4, StreamingLLM discipline).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub o1_sink: Option<u32>,
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
#[serde(default)]
pub skip_mtp: bool,
}
/// CMF-format model factory: import a HuggingFace model, convert it to a
@@ -172,6 +202,16 @@ impl CmfCfg {
port: self.local_port,
threads: self.threads,
gpu: self.gpu,
temperature: None,
top_p: None,
max_tokens: None,
think_budget: None,
system_prompt: None,
o1: None,
o1_m: None,
o1_window: None,
o1_sink: None,
skip_mtp: false,
}];
}
Vec::new()
@@ -306,6 +346,27 @@ pub struct ModelCfg {
pub api_key_env: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub caps: Vec<String>,
/// Default generation temperature (0.0 .. 2.0).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub temperature: Option<f32>,
/// Default nucleus sampling threshold (0.0 .. 1.0).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub top_p: Option<f32>,
/// Default maximum completion tokens.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_tokens: Option<u32>,
/// Reasoning / thinking token budget.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub think_budget: Option<u32>,
/// Default system prompt prepended if request lacks one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub system_prompt: Option<String>,
/// O(1) constant-memory streaming attention hint (all, deep, custom, off).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub o1: Option<String>,
/// Skip Multi-Token Prediction (MTP) heads.
#[serde(default)]
pub skip_mtp: bool,
}
fn default_kind() -> String {
"chat".into()
@@ -461,10 +522,12 @@ pub struct AdminCfg {
/// Whether the admin panel and admin API (`/admin`, `/admin/api/*`) are enabled.
#[serde(default = "default_admin_enabled")]
pub enabled: bool,
/// Name of the env variable holding the admin token. If unset (or empty),
/// a token is generated at startup and printed to the log.
/// Name of the env variable holding the admin token.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub token_env: Option<String>,
/// Persistent admin token configured directly in toml.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub token: Option<String>,
/// Optional separate bind address for the admin surface (e.g. `127.0.0.1:9001`).
/// If unset, the admin interface lives on the main `listen` address under `/admin`.
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -475,6 +538,7 @@ impl Default for AdminCfg {
Self {
enabled: default_admin_enabled(),
token_env: None,
token: None,
listen: None,
}
}
+300 -4
View File
@@ -71,6 +71,20 @@ pub struct ImportParams {
pub shard_max_gb: Option<f32>,
#[serde(default)]
pub skip_mtp: bool,
/// O(1) Nyström attention hint (native converter, cortiq ≥ 0.2.0):
/// `all` | `deepN` | `i,j,k`. Weights pass through unchanged — the runtime
/// reads the hint at load. Empty/None/"off" = exact attention.
#[serde(default)]
pub o1: Option<String>,
/// Landmark budget for the o1 hint (validated default 32).
#[serde(default)]
pub o1_m: Option<usize>,
/// Exact-window width for the o1 hint (validated default 128).
#[serde(default)]
pub o1_window: Option<usize>,
/// Permanent exact sink keys for the o1 hint (validated default 4).
#[serde(default)]
pub o1_sink: Option<usize>,
}
fn default_quant() -> String {
"Q8_2F".into()
@@ -146,6 +160,14 @@ impl JobStore {
}
}
}
fn update_quant(&self, id: &str, quant: String) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
j.quant = quant.clone();
if !j.log.is_empty() && j.log[0].starts_with("→ downloading ready .cmf model") {
j.log[0] = format!("→ downloading ready .cmf model {} ({})", j.repo, quant);
}
}
}
fn set_state(&self, id: &str, state: &str) {
if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
j.state = state.into();
@@ -246,14 +268,172 @@ fn sanitize(s: &str) -> String {
/// Map the UI quant label to the native `cortiq` quant flag.
fn map_quant(q: &str) -> &'static str {
match q.to_ascii_uppercase().as_str() {
s if s.starts_with("Q4T") || s == "Q4_TILED" => "q4t",
s if s.starts_with("Q4") => "q4",
"Q8_2F" => "q8_2f",
"Q8_2F" | "Q82F" => "q8_2f",
"F16" | "FP16" => "f16",
"VBIT" => "vbit",
"Q1" => "q1",
"Q1P" | "Q1_PTQ" => "q1p",
"Q1S" | "Q1_MASK" => "q1s",
"Q1T" | "Q1_TERNARY" => "q1t",
_ => "q8",
}
}
fn is_cmf_repo(repo: &str) -> bool {
let lower = repo.to_ascii_lowercase();
lower.contains("cmf") || lower.ends_with(".cmf")
}
async fn download_cmf_repo(
repo: String,
output_abs: String,
hf_token: Option<String>,
store: Arc<JobStore>,
id: String,
cancel: Arc<Notify>,
) {
store.set_progress(&id, 0.0, "listing files".into());
let tree_url = format!("https://huggingface.co/api/models/{repo}/tree/main?recursive=true");
let client = reqwest::Client::new();
let mut req = client.get(&tree_url).header("User-Agent", "cortiq-gateway");
if let Some(t) = &hf_token {
req = req.bearer_auth(t);
}
let mut cmf_rel_path = None;
if let Ok(resp) = req.send().await {
if resp.status().is_success() {
if let Ok(json) = resp.json::<serde_json::Value>().await {
if let Some(arr) = json.as_array() {
for item in arr {
if let Some(path) = item.get("path").and_then(|p| p.as_str()) {
if path.to_ascii_lowercase().ends_with(".cmf") {
cmf_rel_path = Some(path.to_string());
break;
}
}
}
}
}
}
}
let cmf_path = match cmf_rel_path {
Some(p) => p,
None => {
let repo_name = repo.rsplit('/').next().unwrap_or(&repo);
format!("{repo_name}.cmf")
}
};
let path_lower = cmf_path.to_ascii_lowercase();
let detected_quant =
if path_lower.contains("-q1t.") || repo.to_ascii_lowercase().contains("2bit") {
"Q1T".to_string()
} else if path_lower.contains("-q1.") || repo.to_ascii_lowercase().contains("cmf") {
"Q1".to_string()
} else if path_lower.contains("-q4.") {
"Q4".to_string()
} else if path_lower.contains("-q8.") {
"Q8".to_string()
} else {
"READY_CMF".to_string()
};
store.update_quant(&id, detected_quant);
store.push_line(&id, format!("→ downloading {cmf_path} from HuggingFace..."));
store.set_progress(&id, 0.05, "downloading".into());
let download_url = format!("https://huggingface.co/{repo}/resolve/main/{cmf_path}");
let mut cmd = tokio::process::Command::new("curl");
cmd.arg("-f").arg("-L").arg("-#").arg("-o").arg(&output_abs);
if let Some(t) = &hf_token {
cmd.arg("-H").arg(format!("Authorization: Bearer {t}"));
}
cmd.arg(&download_url);
cmd.stderr(std::process::Stdio::piped());
cmd.stdout(std::process::Stdio::null());
let mut child = match cmd.spawn() {
Ok(c) => c,
Err(e) => {
store.push_line(&id, format!("✗ failed to start curl download: {e}"));
store.finish(&id, false);
store.drop_cancel(&id);
return;
}
};
let stderr = child.stderr.take();
let mut t_err = None;
if let Some(err_stream) = stderr {
let store_clone = store.clone();
let id_clone = id.clone();
t_err = Some(tokio::spawn(async move {
use tokio::io::AsyncReadExt;
let mut reader = err_stream;
let mut buf = [0u8; 256];
while let Ok(n) = reader.read(&mut buf).await {
if n == 0 {
break;
}
let text = String::from_utf8_lossy(&buf[..n]);
for word in text.split_whitespace() {
if let Some(num_str) = word.strip_suffix('%') {
if let Ok(pct) = num_str.parse::<f32>() {
let frac = (pct / 100.0).clamp(0.0, 1.0);
store_clone.set_progress(&id_clone, frac, "downloading".into());
}
}
}
}
}));
}
tokio::select! {
status_res = child.wait() => {
if let Some(t) = t_err { t.abort(); }
match status_res {
Ok(status) if status.success() => {
store.push_line(&id, "✓ done".into());
store.finish(&id, true);
}
Ok(status) => {
cleanup_output(&output_abs);
// curl 23 = write error: almost always a non-writable models dir
let hint = match status.code() {
Some(23) => " (write error — is the models dir writable?)",
Some(22) => " (HTTP error — repo/file not found or auth required)",
_ => "",
};
store.push_line(&id, format!("✗ download failed: exit code {:?}{hint}", status.code()));
store.finish(&id, false);
}
Err(e) => {
cleanup_output(&output_abs);
store.push_line(&id, format!("✗ download process error: {e}"));
store.finish(&id, false);
}
}
}
_ = cancel.notified() => {
let _ = child.start_kill();
let _ = child.wait().await;
if let Some(t) = t_err { t.abort(); }
cleanup_output(&output_abs);
store.push_line(&id, "✗ cancelled by user — partial output removed".into());
store.set_state(&id, "cancelled");
}
}
store.drop_cancel(&id);
}
/// into the job log. Fails fast if the converter script is missing.
pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Result<String, String> {
if p.repo.trim().is_empty() {
@@ -269,7 +449,41 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
cfg.converter
));
}
let _ = std::fs::create_dir_all(&cfg.models_dir);
// O(1) attention hint — native-converter feature (cortiq ≥ 0.2.0). Fail fast
// with a clear message instead of a mid-job "unexpected argument" error.
let o1 =
p.o1.as_deref()
.map(str::trim)
.filter(|s| !s.is_empty() && *s != "off")
.map(str::to_string);
if o1.is_some() {
if use_python {
return Err(
"O(1) attention is supported by the native converter only — remove the \
linear-core / v-bit shape / mean-bits options to use it"
.into(),
);
}
if p.repo.to_ascii_lowercase().contains("gguf") {
return Err("O(1) attention is not supported for GGUF imports yet".into());
}
let ver = crate::cmf_runtime::installed_version(&cfg.cortiq_bin);
if let Some(v) = &ver {
if crate::cmf_runtime::version_lt(v, "0.2.0") {
return Err(format!(
"O(1) attention needs cortiq ≥ 0.2.0 (installed: {v}) — update the runtime \
in Settings → Local models"
));
}
}
}
std::fs::create_dir_all(&cfg.models_dir).map_err(|e| {
format!(
"cannot create models dir {}: {e} — check that the gateway has write \
access (in Docker, mount a writable volume at the models path)",
cfg.models_dir
)
})?;
let base = if p.name.trim().is_empty() {
sanitize(p.repo.rsplit('/').next().unwrap_or(&p.repo))
} else {
@@ -284,13 +498,41 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
.unwrap_or_else(|_| output.clone());
let id = gen_id(&p.repo);
let is_cmf = is_cmf_repo(&p.repo);
let quant_display = if is_cmf {
let r = p.repo.to_lowercase();
if r.contains("2bit") || r.contains("q1t") {
"Q1T".to_string()
} else if r.contains("1.7bcmf")
|| r.contains("27bcmf")
|| r.contains("cmf")
|| p.quant.is_empty()
|| p.quant == "auto"
|| p.quant == "Q8_2F"
{
"Q1".to_string()
} else {
p.quant.clone()
}
} else if p.quant.is_empty() || p.quant == "auto" {
"q8".to_string()
} else {
p.quant.clone()
};
store.insert(Job {
id: id.clone(),
repo: p.repo.clone(),
quant: p.quant.clone(),
quant: quant_display.clone(),
output: output_abs.clone(),
state: "running".into(),
log: vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)],
log: if is_cmf {
vec![format!(
"→ downloading ready .cmf model {} ({})",
p.repo, quant_display
)]
} else {
vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)]
},
started: now(),
finished: None,
size_bytes: None,
@@ -306,6 +548,14 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
std::env::var(&cfg.hf_token_env).ok()
};
if is_cmf && !use_python {
let ret_id = id.clone();
tokio::spawn(async move {
download_cmf_repo(p.repo, output_abs, hf_token, store, id, cancel).await;
});
return Ok(ret_id);
}
// Program + args: `cortiq convert` by default, or the Python converter for
// advanced options. Both stream `@PROGRESS <frac>` markers into the log.
let (program, args, workdir): (String, Vec<String>, std::path::PathBuf) = if use_python {
@@ -384,6 +634,23 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
a.push("--hf-token".into());
a.push(t.clone());
}
// O(1) attention hint (weights unchanged; the runtime reads it at load)
if let Some(spec) = &o1 {
a.push("--o1".into());
a.push(spec.clone());
if let Some(m) = p.o1_m {
a.push("--o1-m".into());
a.push(m.to_string());
}
if let Some(w) = p.o1_window {
a.push("--o1-window".into());
a.push(w.to_string());
}
if let Some(s) = p.o1_sink {
a.push("--o1-sink".into());
a.push(s.to_string());
}
}
(
cfg.cortiq_bin.clone(),
a,
@@ -471,3 +738,32 @@ pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Resu
Ok(ret_id)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_map_quant_variants() {
assert_eq!(map_quant("Q8_2F"), "q8_2f");
assert_eq!(map_quant("q82f"), "q8_2f");
assert_eq!(map_quant("Q8_ROW"), "q8");
assert_eq!(map_quant("Q4_BLOCK"), "q4");
assert_eq!(map_quant("Q4_TILED"), "q4t");
assert_eq!(map_quant("q4t"), "q4t");
assert_eq!(map_quant("vbit"), "vbit");
assert_eq!(map_quant("q1"), "q1");
assert_eq!(map_quant("Q1P"), "q1p");
assert_eq!(map_quant("q1s"), "q1s");
assert_eq!(map_quant("Q1T"), "q1t");
assert_eq!(map_quant("F16"), "f16");
}
#[test]
fn test_is_cmf_repo_check() {
assert!(is_cmf_repo("infosave/Bonsai-8B_2bit_cmf"));
assert!(is_cmf_repo("infosave/Bonsai-1.7Bcmf"));
assert!(is_cmf_repo("user/model.cmf"));
assert!(!is_cmf_repo("Qwen/Qwen2.5-0.5B-Instruct"));
}
}
+12 -1
View File
@@ -61,6 +61,11 @@ fn resolve_admin_token(cfg: &Config, args: &Args) -> (String, bool) {
return (t.clone(), false);
}
}
if let Some(t) = &cfg.admin.token {
if !t.is_empty() {
return (t.clone(), false);
}
}
(admin::random_token(24), true)
}
@@ -94,6 +99,7 @@ async fn main() -> anyhow::Result<()> {
let mut app = Router::new()
.route("/healthz", get(|| async { "ok" }))
.route("/health", get(|| async { "ok" }))
.route("/readyz", get(|| async { "ready" }))
.route("/metrics", get(admin::metrics))
.merge(protocols::build_router());
@@ -104,7 +110,12 @@ async fn main() -> anyhow::Result<()> {
.fallback(admin::assets::fallback);
}
let app = app.with_state(state);
let cors = tower_http::cors::CorsLayer::new()
.allow_origin(tower_http::cors::Any)
.allow_methods(tower_http::cors::Any)
.allow_headers(tower_http::cors::Any);
let app = app.with_state(state).layer(cors);
let listener = tokio::net::TcpListener::bind(&listen).await?;
tracing::info!("cortiq-gateway listening on {listen}");
+1
View File
@@ -29,6 +29,7 @@ pub struct GenParams {
pub temperature: Option<f32>,
pub max_tokens: Option<u32>,
pub top_p: Option<f32>,
pub think_budget: Option<u32>,
pub stop: Vec<String>,
/// Fields we do not interpret but must proxy through to the provider.
pub passthrough: serde_json::Map<String, serde_json::Value>,
+117 -2
View File
@@ -341,7 +341,64 @@ impl Pipeline {
route_source
);
match provider.chat(req.clone()).await {
let mut req_call = req.clone();
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
if req_call.params.temperature.is_none() {
req_call.params.temperature = m_cfg.temperature;
}
if req_call.params.top_p.is_none() {
req_call.params.top_p = m_cfg.top_p;
}
if req_call.params.max_tokens.is_none() {
req_call.params.max_tokens = m_cfg.max_tokens;
}
if req_call.params.think_budget.is_none() {
req_call.params.think_budget = m_cfg.think_budget;
}
if let Some(sys) = &m_cfg.system_prompt {
if !sys.trim().is_empty()
&& !req_call.messages.iter().any(|m| m.role == "system")
{
req_call.messages.insert(
0,
Message {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
},
);
}
}
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
if req_call.params.temperature.is_none() {
req_call.params.temperature = s_cfg.temperature;
}
if req_call.params.top_p.is_none() {
req_call.params.top_p = s_cfg.top_p;
}
if req_call.params.max_tokens.is_none() {
req_call.params.max_tokens = s_cfg.max_tokens;
}
if req_call.params.think_budget.is_none() {
req_call.params.think_budget = s_cfg.think_budget;
}
if let Some(sys) = &s_cfg.system_prompt {
if !sys.trim().is_empty()
&& !req_call.messages.iter().any(|m| m.role == "system")
{
req_call.messages.insert(
0,
Message {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
},
);
}
}
}
match provider.chat(req_call).await {
Ok(mut resp) => {
let cost =
provider.price(resp.usage.prompt_tokens, resp.usage.completion_tokens);
@@ -415,7 +472,65 @@ impl Pipeline {
model_id,
route_source
);
match provider.chat_stream(req.clone()).await {
// Merge per-model/server config (same as run_inner).
let mut req_call = req.clone();
if let Some(m_cfg) = live.cfg.models.iter().find(|m| m.id == model_id) {
if req_call.params.temperature.is_none() {
req_call.params.temperature = m_cfg.temperature;
}
if req_call.params.top_p.is_none() {
req_call.params.top_p = m_cfg.top_p;
}
if req_call.params.max_tokens.is_none() {
req_call.params.max_tokens = m_cfg.max_tokens;
}
if req_call.params.think_budget.is_none() {
req_call.params.think_budget = m_cfg.think_budget;
}
if let Some(sys) = &m_cfg.system_prompt {
if !sys.trim().is_empty()
&& !req_call.messages.iter().any(|m| m.role == "system")
{
req_call.messages.insert(
0,
Message {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
},
);
}
}
} else if let Some(s_cfg) = live.cfg.cmf.servers.iter().find(|s| s.id == model_id) {
if req_call.params.temperature.is_none() {
req_call.params.temperature = s_cfg.temperature;
}
if req_call.params.top_p.is_none() {
req_call.params.top_p = s_cfg.top_p;
}
if req_call.params.max_tokens.is_none() {
req_call.params.max_tokens = s_cfg.max_tokens;
}
if req_call.params.think_budget.is_none() {
req_call.params.think_budget = s_cfg.think_budget;
}
if let Some(sys) = &s_cfg.system_prompt {
if !sys.trim().is_empty()
&& !req_call.messages.iter().any(|m| m.role == "system")
{
req_call.messages.insert(
0,
Message {
role: "system".into(),
content: sys.clone(),
tool_calls: Vec::new(),
},
);
}
}
}
match provider.chat_stream(req_call).await {
Ok(stream) => {
let info = RouteInfo {
task_label: decision.task_label.clone(),
+1
View File
@@ -101,6 +101,7 @@ async fn handler(
temperature: req.temperature,
max_tokens: req.max_tokens,
top_p: req.top_p,
think_budget: None,
stop: Vec::new(),
passthrough: Default::default(),
},
+5
View File
@@ -49,6 +49,10 @@ struct OpenAiChatRequest {
#[serde(default)]
top_p: Option<f32>,
#[serde(default)]
think_budget: Option<u32>,
#[serde(default)]
reasoning_effort: Option<u32>,
#[serde(default)]
stream: bool,
#[serde(default)]
tools: Vec<serde_json::Value>,
@@ -93,6 +97,7 @@ async fn handler(
temperature: req.temperature,
max_tokens: req.max_tokens,
top_p: req.top_p,
think_budget: req.think_budget.or(req.reasoning_effort),
stop: Vec::new(),
passthrough: req.rest,
},
+1
View File
@@ -82,6 +82,7 @@ async fn handler(
temperature: req.temperature,
max_tokens: req.max_tokens,
top_p: req.top_p,
think_budget: None,
stop: Vec::new(),
passthrough: Default::default(),
},
+15
View File
@@ -42,9 +42,20 @@ impl OpenAiProvider {
if let Some(t) = req.params.temperature {
body["temperature"] = t.into();
}
if let Some(p) = req.params.top_p {
body["top_p"] = p.into();
}
if let Some(m) = req.params.max_tokens {
body["max_tokens"] = m.into();
}
eprintln!("[to_wire] think_budget={:?}", req.params.think_budget);
if let Some(tb) = req.params.think_budget {
body["think_budget"] = tb.into();
body["reasoning_effort"] = tb.into();
// cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
body["enable_thinking"] = (tb > 0).into();
eprintln!("[to_wire] SET enable_thinking={}", tb > 0);
}
if !req.tools.is_empty() {
body["tools"] = serde_json::Value::Array(req.tools.clone());
}
@@ -52,6 +63,10 @@ impl OpenAiProvider {
for (k, v) in &req.params.passthrough {
body[k] = v.clone();
}
eprintln!(
"[to_wire] enable_thinking={:?}",
body.get("enable_thinking")
);
body
}
}
+7
View File
@@ -49,6 +49,13 @@ impl Registry {
kind: "chat".into(),
api_key_env: None,
caps: Vec::new(),
temperature: s.temperature,
top_p: s.top_p,
max_tokens: s.max_tokens,
think_budget: s.think_budget,
system_prompt: s.system_prompt.clone(),
o1: s.o1.clone(),
skip_mtp: s.skip_mtp,
};
by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
}
+8
View File
@@ -145,6 +145,14 @@ impl AppState {
let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
new_cfg.save(&self.config_path)?;
self.live.store(Arc::new(live));
let cmf_rt = self.cmf.clone();
let cmf_cfg = new_cfg.cmf.clone();
tokio::spawn(async move {
cmf_rt.stop().await;
crate::cmf_runtime::manage(cmf_rt, cmf_cfg).await;
});
Ok(())
}
+93 -41
View File
@@ -83,9 +83,23 @@ impl Totals {
self.cost_usd += r.cost_usd;
self.latency_ms_sum += r.latency_ms;
}
/// Accumulate another Totals into this one (for windowed aggregation).
fn merge(&mut self, o: &Totals) {
self.requests += o.requests;
self.ok += o.ok;
self.errors += o.errors;
self.failovers += o.failovers;
self.prompt_tokens += o.prompt_tokens;
self.completion_tokens += o.completion_tokens;
self.cost_usd += o.cost_usd;
self.latency_ms_sum += o.latency_ms_sum;
}
}
/// Per-minute bucket for time-series charts.
/// Per-minute bucket. Beyond the chart fields it carries the per-group breakdown
/// and failovers so the admin snapshot can be computed for ANY time window by
/// summing the in-window buckets (the internal fields are not serialized into
/// the `series` payload).
#[derive(Clone, Debug, Default, Serialize)]
pub struct Bucket {
pub minute: u64, // unix seconds, rounded down to the minute
@@ -95,15 +109,25 @@ pub struct Bucket {
pub completion_tokens: u64,
pub cost_usd: f64,
pub latency_ms_sum: u64,
#[serde(skip)]
pub failovers: u64,
#[serde(skip)]
pub by_model: HashMap<String, Totals>,
#[serde(skip)]
pub by_account: HashMap<String, Totals>,
#[serde(skip)]
pub by_tier: HashMap<String, Totals>,
#[serde(skip)]
pub by_label: HashMap<String, Totals>,
}
#[derive(Default)]
struct Inner {
/// All-time totals + per-model (used by the Prometheus counters, which must
/// be monotonic — the windowed admin snapshot is derived from the buckets).
total: Totals,
by_model: HashMap<String, Totals>,
by_account: HashMap<String, Totals>,
by_tier: HashMap<String, Totals>,
by_label: HashMap<String, Totals>,
/// Per-minute buckets — the single source for the windowed admin snapshot.
buckets: VecDeque<Bucket>,
recent: VecDeque<RequestRecord>,
}
@@ -166,31 +190,14 @@ impl Stats {
}
fn apply_aggregates(inner: &mut Inner, rec: &RequestRecord, retention_secs: u64) {
// all-time counters for Prometheus; the windowed snapshot is derived
// from the buckets below.
inner.total.apply(rec);
inner
.by_model
.entry(rec.model_id.clone())
.or_default()
.apply(rec);
inner
.by_account
.entry(if rec.account.is_empty() {
"anonymous".to_string()
} else {
rec.account.clone()
})
.or_default()
.apply(rec);
inner
.by_tier
.entry(rec.tier.clone())
.or_default()
.apply(rec);
inner
.by_label
.entry(rec.task_label.clone())
.or_default()
.apply(rec);
let minute = rec.ts - (rec.ts % 60);
match inner.buckets.back_mut() {
@@ -250,13 +257,31 @@ impl Stats {
.filter(|b| b.minute >= cutoff)
.collect();
let breakdown_src = match groupby {
"account" => &inner.by_account,
"tier" => &inner.by_tier,
"label" => &inner.by_label,
_ => &inner.by_model,
};
let mut breakdown: Vec<serde_json::Value> = breakdown_src
// Windowed totals + breakdown = sum of the in-window buckets, so the KPIs
// and the breakdown recompute whenever the range changes.
let mut win = Totals::default();
let mut group: HashMap<String, Totals> = HashMap::new();
for b in &series {
win.requests += b.requests;
win.errors += b.errors;
win.ok += b.requests.saturating_sub(b.errors);
win.failovers += b.failovers;
win.prompt_tokens += b.prompt_tokens;
win.completion_tokens += b.completion_tokens;
win.cost_usd += b.cost_usd;
win.latency_ms_sum += b.latency_ms_sum;
let src = match groupby {
"account" => &b.by_account,
"tier" => &b.by_tier,
"label" => &b.by_label,
_ => &b.by_model,
};
for (k, t) in src {
group.entry(k.clone()).or_default().merge(t);
}
}
let mut breakdown: Vec<serde_json::Value> = group
.iter()
.map(|(k, t)| {
serde_json::json!({
@@ -280,17 +305,17 @@ impl Stats {
serde_json::json!({
"totals": {
"requests": inner.total.requests,
"ok": inner.total.ok,
"errors": inner.total.errors,
"failovers": inner.total.failovers,
"prompt_tokens": inner.total.prompt_tokens,
"completion_tokens": inner.total.completion_tokens,
"total_tokens": inner.total.prompt_tokens + inner.total.completion_tokens,
"cost_usd": inner.total.cost_usd,
"avg_latency_ms": avg(inner.total.latency_ms_sum, inner.total.requests),
"success_rate": if inner.total.requests > 0 {
inner.total.ok as f64 / inner.total.requests as f64
"requests": win.requests,
"ok": win.ok,
"errors": win.errors,
"failovers": win.failovers,
"prompt_tokens": win.prompt_tokens,
"completion_tokens": win.completion_tokens,
"total_tokens": win.prompt_tokens + win.completion_tokens,
"cost_usd": win.cost_usd,
"avg_latency_ms": avg(win.latency_ms_sum, win.requests),
"success_rate": if win.requests > 0 {
win.ok as f64 / win.requests as f64
} else { 0.0 },
},
"groupby": groupby,
@@ -299,6 +324,18 @@ impl Stats {
})
}
/// Reset all in-memory stats and truncate the JSONL log — the "clear logs"
/// admin action.
pub fn clear(&self) {
{
let mut inner = self.inner.lock().unwrap();
*inner = Inner::default();
}
if let Some(path) = &self.file {
let _ = std::fs::write(path, "");
}
}
/// Recent requests (newest first), with pagination.
pub fn recent(&self, limit: usize, offset: usize) -> Vec<RequestRecord> {
let inner = self.inner.lock().unwrap();
@@ -358,10 +395,25 @@ fn fill_bucket(b: &mut Bucket, r: &RequestRecord) {
if r.outcome != "ok" {
b.errors += 1;
}
if r.failover {
b.failovers += 1;
}
b.prompt_tokens += r.prompt_tokens as u64;
b.completion_tokens += r.completion_tokens as u64;
b.cost_usd += r.cost_usd;
b.latency_ms_sum += r.latency_ms;
// per-group breakdown, so the snapshot can be windowed by summing buckets
b.by_model.entry(r.model_id.clone()).or_default().apply(r);
b.by_account
.entry(if r.account.is_empty() {
"anonymous".to_string()
} else {
r.account.clone()
})
.or_default()
.apply(r);
b.by_tier.entry(r.tier.clone()).or_default().apply(r);
b.by_label.entry(r.task_label.clone()).or_default().apply(r);
}
fn avg(sum: u64, n: u64) -> u64 {
+167
View File
@@ -397,3 +397,170 @@ tbody tr:hover td { background: var(--hover); }
.mono.kv { display: flex; gap: 6px; padding: 1px 0; white-space: normal; overflow-wrap: anywhere; word-break: break-word; }
.mono.kv .kv-k { color: var(--muted, #9aa4b2); flex: 0 0 auto; }
.mono.kv .kv-v { font-weight: 600; overflow-wrap: anywhere; word-break: break-word; min-width: 0; }
/* ---- responsive shell + touch layout ----------------------------------- */
.menu-toggle, .nav-close, .mobile-nav-backdrop { display: none; }
.hamburger { width: 18px; display: flex; flex-direction: column; gap: 4px; }
.hamburger i { display: block; height: 2px; border-radius: 2px; background: currentColor; }
body.modal-open { overflow: hidden; }
@media (max-width: 980px) {
.app { display: block; min-height: 100dvh; }
.main { min-height: 100dvh; }
.menu-toggle { display: inline-flex; flex: 0 0 auto; }
.nav-close { display: inline-flex; margin-left: auto; flex: 0 0 auto; font-size: 24px; line-height: 1; }
.brand-copy { min-width: 0; }
.sidebar {
display: flex;
position: fixed;
inset: 0 auto 0 0;
width: min(84vw, 310px);
height: 100dvh;
padding: max(18px, env(safe-area-inset-top)) 14px max(14px, env(safe-area-inset-bottom));
z-index: 70;
box-shadow: 18px 0 45px rgba(0, 0, 0, .42);
transform: translateX(-104%);
visibility: hidden;
transition: transform .22s ease, visibility 0s linear .22s;
overflow-y: auto;
}
.app.nav-open .sidebar {
transform: translateX(0);
visibility: visible;
transition: transform .22s ease;
}
.mobile-nav-backdrop {
display: block;
position: fixed;
inset: 0;
z-index: 65;
width: 100%;
height: 100%;
border: 0;
background: rgba(3, 7, 18, .62);
opacity: 0;
visibility: hidden;
pointer-events: none;
transition: opacity .22s ease, visibility 0s linear .22s;
}
.app.nav-open .mobile-nav-backdrop {
opacity: 1;
visibility: visible;
pointer-events: auto;
transition: opacity .22s ease;
}
body.mobile-nav-open { overflow: hidden; }
.topbar { padding: 10px 18px; min-height: 58px; gap: 10px; }
.topbar .title { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.topbar .right { gap: 8px; min-width: 0; }
.content { max-width: none; padding: 22px 20px 52px; }
}
@media (max-width: 720px) {
body { font-size: 14px; }
.status-pills { display: none; }
.topbar { padding: 9px 12px; }
.content { padding: 18px 14px 42px; }
.page-head { margin-bottom: 16px; }
.page-head > .flex { align-items: stretch; flex-direction: column; }
.page-head > .flex > .btn { width: 100%; }
.page-head p { line-height: 1.45; }
.card { padding: 15px; margin-bottom: 14px; border-radius: 12px; }
.card .card-head { align-items: flex-start; flex-wrap: wrap; }
.card .card-head .right { width: 100%; margin-left: 0; flex-wrap: wrap; }
.grid { gap: 14px; }
.kpis { gap: 10px; margin-bottom: 14px; }
.kpi { padding: 12px; border-radius: 11px; }
.kpi .big { font-size: 21px; }
.row { display: grid; grid-template-columns: minmax(0, 1fr); gap: 0; }
.row > * { min-width: 0; width: 100%; }
input, select, textarea { min-height: 44px; font-size: 16px; }
input[type="checkbox"], input[type="radio"] { min-height: 0; }
.check { min-height: 36px; margin-bottom: 8px; }
.btn, .icon-btn { min-height: 44px; }
.btn { padding: 10px 14px; }
.btn.sm, .btn.small { min-height: 38px; padding: 8px 11px; }
.table-wrap {
width: 100%;
overflow-x: auto;
overscroll-behavior-x: contain;
-webkit-overflow-scrolling: touch;
scrollbar-width: thin;
}
.table-wrap table { min-width: max-content; }
th, td { padding: 10px; }
.tierrow { align-items: flex-start; flex-direction: column; gap: 8px; }
.tierrow .tname { width: auto; }
.chips { width: 100%; }
.togglerow { align-items: flex-start; }
.pg { gap: 14px; }
.routecard .kv { flex-wrap: wrap; }
.routecard .kv .v { max-width: 100%; overflow-wrap: anywhere; }
.answer { padding: 12px; }
.seg { max-width: 100%; overflow-x: auto; }
.seg button { min-height: 40px; white-space: nowrap; }
.hf-grid { grid-template-columns: 1fr; max-height: 56dvh; }
.job-head, .actions { align-items: stretch; flex-wrap: wrap; }
.actions .btn { flex: 1 1 150px; }
.key-row { flex-direction: column; }
.key-row .btn { width: 100%; }
.cmf-server-row { align-items: stretch !important; padding: 10px; border: 1px solid var(--border-soft); border-radius: 10px; }
.cmf-server-row > input, .cmf-server-row > select { width: 100% !important; min-width: 0 !important; flex: 1 1 100% !important; }
.cmf-server-row > .btn { flex: 1 1 calc(50% - 6px); }
.cmf-server-row > .small { flex: 1 1 100%; }
.modal-backdrop { align-items: flex-end; padding: 8px 0 0; }
.modal {
max-width: none;
max-height: calc(100dvh - 8px);
overflow-y: auto;
padding: 18px 15px max(14px, env(safe-area-inset-bottom));
border-radius: 18px 18px 0 0;
}
.modal .modal-foot {
position: sticky;
bottom: calc(-1 * max(14px, env(safe-area-inset-bottom)));
margin: 12px -15px 0;
padding: 12px 15px max(14px, env(safe-area-inset-bottom));
background: color-mix(in srgb, var(--card) 94%, transparent);
backdrop-filter: blur(10px);
border-top: 1px solid var(--border-soft);
}
.modal .modal-foot .btn { flex: 1; }
.toasts { left: 12px; right: 12px; bottom: max(12px, env(safe-area-inset-bottom)); }
.toast { min-width: 0; width: 100%; }
.login-wrap { align-items: flex-start; padding: max(20px, env(safe-area-inset-top)) 14px; }
.login-card { padding: 22px 18px; margin-top: 8dvh; }
}
@media (max-width: 480px) {
.topbar .title { font-size: 14px; }
.topbar .right { gap: 6px; }
.lang-select { width: 54px; min-height: 44px; padding: 6px; }
.icon-btn { width: 42px; height: 42px; }
.kpis { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.kpi .lbl { line-height: 1.25; }
.barlist .b { grid-template-columns: minmax(0, 1fr) auto; gap: 6px 10px; }
.barlist .b .track { grid-column: 1 / -1; grid-row: 2; }
.barlist .b .val { min-width: 0; }
.empty { padding: 28px 12px; }
}
@media (max-width: 360px) {
.topbar .title { display: none; }
.kpis { grid-template-columns: 1fr; }
}
@media (prefers-reduced-motion: reduce) {
.sidebar, .mobile-nav-backdrop { transition: none; }
}
+1 -1
View File
@@ -19,6 +19,6 @@
<noscript>This console requires JavaScript.</noscript>
<div id="app"><div class="boot">Loading…</div></div>
<div id="toasts" class="toasts"></div>
<script type="module" src="/admin/js/app.js"></script>
<script type="module" src="/admin/js/app.js?v=3"></script>
</body>
</html>
+1
View File
@@ -87,6 +87,7 @@ export const api = {
// stats
stats: (range, groupby) => j("GET", `/stats?range=${range}&groupby=${groupby}`),
clearStats: () => j("DELETE", "/stats"),
requests: (limit = 50, offset = 0) => j("GET", `/requests?limit=${limit}&offset=${offset}`),
// playground
+39 -5
View File
@@ -4,7 +4,7 @@ import { t, getLang, setLang, onLangChange, LANGS } from "./i18n.js";
import { api, getToken, setToken, setUnauthorizedHandler } from "./api.js";
import { renderLogin } from "./views/login.js";
import { renderDashboard } from "./views/dashboard.js";
import { renderModels } from "./views/models.js";
import { renderModels } from "./views/models.js?v=3";
import { renderImport } from "./views/import.js";
import { renderRouting } from "./views/routing.js";
import { renderProtocols } from "./views/protocols.js";
@@ -28,6 +28,17 @@ export const appState = { meta: null, health: null, version: "" };
// allaigate site — where router keys are issued and topped up
export const SITE_URL = "https://api.allaigate.com/";
let mobileNavOpen = false;
function setMobileNav(open) {
mobileNavOpen = !!open;
const app = document.querySelector(".app");
if (app) app.classList.toggle("nav-open", mobileNavOpen);
document.body.classList.toggle("mobile-nav-open", mobileNavOpen);
const toggle = document.querySelector(".menu-toggle");
if (toggle) toggle.setAttribute("aria-expanded", String(mobileNavOpen));
}
function currentRoute() {
const id = location.hash.replace(/^#\/?/, "") || "dashboard";
return ROUTES.find((r) => r.id === id) || ROUTES[0];
@@ -42,17 +53,18 @@ const FAVICON =
function sidebar(activeId) {
return h(
"aside",
{ class: "sidebar", id: "sidebar" },
{ class: "sidebar", id: "sidebar", "aria-label": "Main navigation" },
h(
"div",
{ class: "brand" },
h("img", { src: FAVICON, alt: "" }),
h("div", {}, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub")))
h("div", { class: "brand-copy" }, h("b", {}, "allaigate"), h("div", { class: "sub" }, t("brand.sub"))),
h("button", { class: "icon-btn nav-close", type: "button", title: "Close menu", "aria-label": "Close menu", onClick: () => setMobileNav(false) }, "×")
),
...ROUTES.map((r) =>
h(
"a",
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id },
{ class: "nav-item" + (r.id === activeId ? " active" : ""), href: "#/" + r.id, onClick: () => setMobileNav(false) },
h("span", { class: "ico" }, r.icon),
t("nav." + r.id)
)
@@ -85,7 +97,7 @@ function statusPills() {
: t("status.checking");
return h(
"div",
{ class: "flex" },
{ class: "flex status-pills" },
h(
"span",
{ class: "pill" },
@@ -136,6 +148,19 @@ function topbar(activeId) {
return h(
"header",
{ class: "topbar" },
h(
"button",
{
class: "icon-btn menu-toggle",
type: "button",
title: "Menu",
"aria-label": "Open menu",
"aria-controls": "sidebar",
"aria-expanded": String(mobileNavOpen),
onClick: () => setMobileNav(!mobileNavOpen),
},
h("span", { class: "hamburger", "aria-hidden": "true" }, h("i"), h("i"), h("i"))
),
h("div", { class: "title" }, t("nav." + activeId)),
h(
"div",
@@ -175,12 +200,15 @@ async function renderView() {
function renderShell() {
const route = currentRoute();
mobileNavOpen = false;
document.body.classList.remove("mobile-nav-open");
mount(
document.getElementById("app"),
h(
"div",
{ class: "app" },
sidebar(route.id),
h("button", { class: "mobile-nav-backdrop", type: "button", tabindex: "-1", "aria-label": "Close menu", onClick: () => setMobileNav(false) }),
h("main", { class: "main" }, topbar(route.id), h("div", { class: "content", id: "view" }))
)
);
@@ -243,6 +271,12 @@ onLangChange(() => {
window.addEventListener("hashchange", () => {
if (document.querySelector(".app")) renderShell();
});
window.addEventListener("keydown", (e) => {
if (e.key === "Escape" && mobileNavOpen) setMobileNav(false);
});
window.addEventListener("resize", () => {
if (window.innerWidth > 980 && mobileNavOpen) setMobileNav(false);
});
setInterval(() => {
if (document.querySelector(".app")) refreshHealth();
}, 20000);
+40 -1
View File
@@ -59,6 +59,9 @@ export default {
// dashboard
"dash.title": "Dashboard",
"dash.subtitle": "Live traffic, cost and routing analytics for your model pool.",
"dash.clear": "Clear logs",
"dash.clearConfirm": "Clear all recorded stats and logs? This cannot be undone.",
"dash.cleared": "Logs cleared",
"dash.range": "Range",
"dash.range.1h": "1h",
"dash.range.24h": "24h",
@@ -136,6 +139,18 @@ export default {
"models.form.apiKeyHint": "Stored securely on the gateway; it is not shown again.",
"models.form.keyStored": "A key is saved — leave blank to keep it, or paste a new one.",
"models.form.keyFromEnv": "Using a key from an environment variable.",
"models.form.paramsTitle": "Generation & Thinking Parameters (O(1), MTP, Temperature)",
"models.form.temperature": "Temperature",
"models.form.temperatureHint": "Generation temperature (0.0 = deterministic, 1.0+ = creative).",
"models.form.topP": "Top-P (Nucleus Sampling)",
"models.form.topPHint": "Sampling threshold (0.0 .. 1.0).",
"models.form.maxTokens": "Max Tokens",
"models.form.maxTokensHint": "Maximum completion tokens to generate.",
"models.form.thinkBudget": "Thinking Mode / Think Budget",
"models.form.thinkBudgetHint": "Reasoning token budget (0 = disabled, N > 0 = max reasoning tokens).",
"models.form.systemPrompt": "Default System Prompt",
"models.form.systemPromptHint": "Prepend automatically if request lacks a system prompt.",
"models.form.systemPromptPlaceholder": "You are a helpful assistant...",
"models.form.testKey": "Test & load models",
"models.form.testing": "checking…",
"models.form.foundModels": "{n} models loaded",
@@ -316,6 +331,10 @@ export default {
"settings.route.maxChars": "Max chars",
"settings.route.cacheTtl": "Cache TTL",
"settings.route.profile": "Default profile",
"settings.route.strategyHelp": "What text is sent to the router for classification: last_user = only the last user message; last_user_plus_system = last user message + system prompt; concat_all = the whole conversation.",
"settings.route.maxCharsHelp": "Max characters of that text sent to the router (longer is truncated). More = more accurate classification, but slower/pricier router calls.",
"settings.route.cacheTtlHelp": "How long a routing decision is cached for an identical request before the router is asked again (e.g. 60s, 5m).",
"settings.route.profileHelp": "Default routing profile when a request doesn't specify one: cost-saver (cheapest), balanced, or quality-first (best model). Biases which model the router picks.",
"settings.breaker": "Circuit breaker",
"settings.breaker.threshold": "Error threshold",
"settings.breaker.cooldown": "Cooldown",
@@ -367,7 +386,7 @@ export default {
"import.failed": "Conversion failed — see the log.",
"import.register": "Register as local model",
"import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
"import.jobs": "Conversions",
"import.jobs": "Downloads & Conversions",
"import.cancel": "Cancel",
"import.cancelling": "Cancelling — cleaning up partial output…",
"import.delete": "Delete",
@@ -377,12 +396,21 @@ export default {
"import.state.done": "done",
"import.state.error": "error",
"import.state.cancelled": "cancelled",
"import.featuredTitle": "Featured Ready-to-Run .cmf Models",
"import.readyCmfBadge": "Ready .cmf",
"import.directDownload": "Download",
"import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
"import.q.q8_row": "8-bit per-row — simple, robust.",
"import.q.q4": "4-bit block — smallest, lower quality.",
"import.q.q4_tiled": "4-bit tiled (Q4_TILED) — 1.66x faster on Metal GPU.",
"import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
"import.q.q1t": "1.58-bit ternary (Q1T) — BitNet b1.58 with sparse outliers.",
"import.q.q1s": "1-bit sparse (Q1S) — 1-bit base with holographic outliers.",
"import.q.q1p": "1-bit PTQ (Q1P) — 1-bit with error diffusion.",
"import.q.q1": "1-bit (Q1) — pure binary format.",
"import.q.f16": "16-bit — no quantization.",
"import.q.f32": "32-bit — full precision (large file).",
"import.adv.title": "Advanced settings",
"import.adv.auto": "auto",
"import.adv.linearCore": "Linear core",
@@ -393,4 +421,15 @@ export default {
"import.adv.shard": "Shard max (GB)",
"import.adv.noShard": "no sharding",
"import.adv.skipMtp": "skip MTP head",
"import.adv.o1": "O(1) attention",
"import.adv.o1Hint": "Constant-memory streaming attention: the KV cache is released after prefill, so long contexts don't grow memory. Weights stay byte-identical — only a load-time hint is recorded; you can switch back anytime (serve with --o1 off). Validated on Qwen3-0.6B: all layers ≈ ×1.18 perplexity.",
"import.adv.o1Off": "off — exact attention (default)",
"import.adv.o1All": "all — convert every softmax layer",
"import.adv.o1Deep": "deep N — only the N deepest layers",
"import.adv.o1Custom": "custom — explicit layer list",
"import.adv.o1DeepN": "N deep layers",
"import.adv.o1Layers": "Layers (comma-separated)",
"import.adv.o1mHint": "landmark budget (default 32)",
"import.adv.o1windowHint": "exact-window width (default 128)",
"import.adv.o1sinkHint": "permanent exact sink keys (default 4)",
};
+40 -1
View File
@@ -58,6 +58,9 @@ export default {
// dashboard
"dash.title": "Дашборд",
"dash.subtitle": "Трафик в реальном времени, расходы и аналитика роутинга по пулу моделей.",
"dash.clear": "Очистить логи",
"dash.clearConfirm": "Очистить всю статистику и логи? Отменить нельзя.",
"dash.cleared": "Логи очищены",
"dash.range": "Период",
"dash.range.1h": "1ч",
"dash.range.24h": "24ч",
@@ -135,6 +138,18 @@ export default {
"models.form.apiKeyHint": "Хранится безопасно на шлюзе; больше не показывается.",
"models.form.keyStored": "Ключ сохранён — оставьте пустым, чтобы не менять, или вставьте новый.",
"models.form.keyFromEnv": "Используется ключ из переменной окружения.",
"models.form.paramsTitle": "Параметры генерации, O(1) и режим размышления",
"models.form.temperature": "Температура",
"models.form.temperatureHint": "Температура генерации (0.0 — точные факты, 1.0+ — творчество).",
"models.form.topP": "Top-P (Nucleus Sampling)",
"models.form.topPHint": "Порог сэмплирования токенов (0.0 .. 1.0).",
"models.form.maxTokens": "Макс. токенов (Max Tokens)",
"models.form.maxTokensHint": "Максимальное количество генерируемых токенов.",
"models.form.thinkBudget": "Режим размышления / Think Budget",
"models.form.thinkBudgetHint": "Бюджет токенов мыслей (0 = выключен, N > 0 = лимит рассуждений).",
"models.form.systemPrompt": "Системный промпт по умолчанию",
"models.form.systemPromptHint": "Автоматически подставляется, если запрос не содержит системное сообщение.",
"models.form.systemPromptPlaceholder": "Вы — полезный ассистент...",
"models.form.testKey": "Проверить и загрузить модели",
"models.form.testing": "проверка…",
"models.form.foundModels": "загружено моделей: {n}",
@@ -315,6 +330,10 @@ export default {
"settings.route.maxChars": "Макс. символов",
"settings.route.cacheTtl": "TTL кэша",
"settings.route.profile": "Профиль по умолчанию",
"settings.route.strategyHelp": "Какой текст отправляется роутеру для классификации: last_user — только последнее сообщение пользователя; last_user_plus_system — последнее сообщение + системный промпт; concat_all — вся переписка.",
"settings.route.maxCharsHelp": "Максимум символов этого текста, отправляемого роутеру (длиннее — обрезается). Больше = точнее классификация, но медленнее/дороже вызовы роутера.",
"settings.route.cacheTtlHelp": "Сколько кэшируется решение роутинга для одинакового запроса, прежде чем роутер спрашивается снова (напр. 60s, 5m).",
"settings.route.profileHelp": "Профиль роутинга по умолчанию, если запрос его не задаёт: cost-saver (дешевле), balanced (баланс) или quality-first (качество). Влияет на выбор модели роутером.",
"settings.breaker": "Автоматический выключатель",
"settings.breaker.threshold": "Порог ошибок",
"settings.breaker.cooldown": "Время восстановления",
@@ -366,7 +385,7 @@ export default {
"import.failed": "Конвертация не удалась — смотрите лог.",
"import.register": "Подключить как локальную модель",
"import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
"import.jobs": "Конвертации",
"import.jobs": "Загрузки / Конвертации",
"import.cancel": "Отменить",
"import.cancelling": "Отмена — удаляю недописанный файл…",
"import.delete": "Удалить",
@@ -376,12 +395,21 @@ export default {
"import.state.done": "готово",
"import.state.error": "ошибка",
"import.state.cancelled": "отменено",
"import.featuredTitle": "Рекомендуемые готовые .cmf модели",
"import.readyCmfBadge": "Готовая .cmf",
"import.directDownload": "Скачать",
"import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
"import.q.q8_row": "8-бит построчно — просто, надёжно.",
"import.q.q4": "4-бит блочно — компактнее, ниже качество.",
"import.q.q4_tiled": "4-бит тайловый (Q4_TILED) — х1.66 быстрее на Metal GPU.",
"import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
"import.q.q1t": "1.58-бит тернарный (Q1T) — BitNet b1.58 с оутлайерами.",
"import.q.q1s": "1-бит спарс (Q1S) — 1-бит база с голографическими оутлайерами.",
"import.q.q1p": "1-бит PTQ (Q1P) — 1-бит с переносом ошибки (error diffusion).",
"import.q.q1": "1-бит (Q1) — чистый бинарный формат.",
"import.q.f16": "16-бит — без квантования.",
"import.q.f32": "32-бит — полная точность (большой файл).",
"import.adv.title": "Расширенные настройки",
"import.adv.auto": "авто",
"import.adv.linearCore": "Линейное ядро",
@@ -392,4 +420,15 @@ export default {
"import.adv.shard": "Шард макс (ГБ)",
"import.adv.noShard": "без шардинга",
"import.adv.skipMtp": "пропустить MTP-голову",
"import.adv.o1": "O(1)-внимание",
"import.adv.o1Hint": "Потоковое внимание с постоянной памятью: KV-кэш освобождается после prefill, длинный контекст не раздувает память. Веса не меняются — записывается лишь подсказка на загрузку; вернуться можно всегда (запуск с --o1 off). Проверено на Qwen3-0.6B: все слои ≈ ×1.18 перплексии.",
"import.adv.o1Off": "выкл — точное внимание (по умолчанию)",
"import.adv.o1All": "all — конвертировать все softmax-слои",
"import.adv.o1Deep": "deep N — только N самых глубоких слоёв",
"import.adv.o1Custom": "вручную — явный список слоёв",
"import.adv.o1DeepN": "N глубоких слоёв",
"import.adv.o1Layers": "Слои (через запятую)",
"import.adv.o1mHint": "число опорных точек (по умолчанию 32)",
"import.adv.o1windowHint": "ширина точного окна (по умолчанию 128)",
"import.adv.o1sinkHint": "постоянные точные sink-ключи (по умолчанию 4)",
};
+14 -1
View File
@@ -23,6 +23,10 @@ export function clear(node) {
while (node && node.firstChild) node.removeChild(node.firstChild);
}
export function opt(v, label, sel) {
return h("option", { value: v, selected: sel ? true : null }, label || v);
}
export function mount(node, ...children) {
clear(node);
for (const c of children.flat())
@@ -46,7 +50,14 @@ export function toast(msg, kind = "info", ms = 3200) {
// Modal dialog. `body` is a DOM node; `onSave` returns a Promise; returns nothing.
export function modal(title, body, onSave, saveLabel) {
const backdrop = h("div", { class: "modal-backdrop" });
const close = () => backdrop.remove();
const onKeydown = (e) => {
if (e.key === "Escape") close();
};
const close = () => {
backdrop.remove();
document.body.classList.remove("modal-open");
document.removeEventListener("keydown", onKeydown);
};
backdrop.addEventListener("click", (e) => {
if (e.target === backdrop) close();
});
@@ -73,6 +84,8 @@ export function modal(title, body, onSave, saveLabel) {
)
);
backdrop.appendChild(card);
document.body.classList.add("modal-open");
document.addEventListener("keydown", onKeydown);
document.body.appendChild(backdrop);
return { close };
}
+19 -5
View File
@@ -1,5 +1,5 @@
// Dashboard — KPIs, time-series charts, breakdown, recent requests, health.
import { h, mount, money, num, ms, pct, timeAgo } from "../ui.js";
import { h, mount, money, num, ms, pct, timeAgo, toast, confirmDialog } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { lineChart, barList } from "../charts.js";
@@ -53,10 +53,24 @@ export async function renderDashboard() {
"div",
{ class: "flex" },
h("div", { class: "grow" }, h("h2", {}, t("dash.title")), h("p", {}, t("dash.subtitle"))),
seg(RANGES, range, (r) => {
range = r;
load();
}, (r) => t("dash.range." + r))
h("div", { class: "flex", style: "gap:8px;align-items:center" },
seg(RANGES, range, (r) => {
range = r;
load();
}, (r) => t("dash.range." + r)),
h("button", {
class: "btn sm danger",
onClick: async () => {
if (!(await confirmDialog(t("dash.clearConfirm")))) return;
try {
await api.clearStats();
toast(t("dash.cleared"), "good");
await load();
} catch (e) {
toast(String(e.message || e), "bad");
}
},
}, t("dash.clear")))
)
),
+97 -4
View File
@@ -2,7 +2,7 @@
// live progress view. Search HF, pick quantization + advanced params, watch
// the conversion (multiple in parallel, resumable across tab switches), then
// register the result as a local model.
import { h, mount, toast } from "../ui.js";
import { h, mount, toast, opt } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
@@ -10,18 +10,27 @@ const QUANTS = [
["Q8_2F", "import.q.q8_2f"],
["Q8_ROW", "import.q.q8_row"],
["Q4_BLOCK", "import.q.q4"],
["Q4_TILED", "import.q.q4_tiled"],
["VBIT", "import.q.vbit"],
["Q1T", "import.q.q1t"],
["Q1S", "import.q.q1s"],
["Q1P", "import.q.q1p"],
["Q1", "import.q.q1"],
["F16", "import.q.f16"],
["F32", "import.q.f32"],
];
const FEATURED_REPOS = [
{ id: "infosave/Bonsai-1.7Bcmf", label: "Bonsai 1.7B CMF", badge: "1.7B · Q1", quant: "Q1" },
{ id: "infosave/Bonsai-8B_2bit_cmf", label: "Bonsai 8B 2-bit CMF", badge: "8B · Q1T", quant: "Q1T" },
{ id: "infosave/Bonsai-27Bcmf", label: "Bonsai 27B CMF", badge: "27B · Q1", quant: "Q1" },
];
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control,
hint ? h("div", { class: "hint" }, hint) : null);
}
function opt(v, label, sel) {
return h("option", { value: v, selected: sel ? true : null }, label || v);
}
function fmtNum(n) {
if (n == null) return "—";
if (n >= 1e6) return (n / 1e6).toFixed(1) + "M";
@@ -125,6 +134,32 @@ export function renderImport() {
const meanBitsIn = h("input", { type: "number", step: "0.1", placeholder: "5.0" });
const shardIn = h("input", { type: "number", step: "0.5", placeholder: t("import.adv.noShard") });
const skipMtp = h("input", { type: "checkbox" });
// O(1) attention (cortiq ≥ 0.2.0): constant-memory streaming attention hint,
// weights byte-identical — the runtime applies it at load.
const o1Sel = h("select", {},
opt("", t("import.adv.o1Off")),
opt("all", t("import.adv.o1All")),
opt("deep", t("import.adv.o1Deep")),
opt("custom", t("import.adv.o1Custom")));
const o1DeepIn = h("input", { type: "number", min: "1", placeholder: "12" });
const o1CustomIn = h("input", { placeholder: "0,4,8" });
const o1M = h("input", { type: "number", min: "1", placeholder: "32" });
const o1Window = h("input", { type: "number", min: "0", placeholder: "128" });
const o1Sink = h("input", { type: "number", min: "0", placeholder: "4" });
const o1SpecRow = h("div", { class: "row", style: "display:none" },
field(t("import.adv.o1DeepN"), o1DeepIn),
field(t("import.adv.o1Layers"), o1CustomIn));
const o1Knobs = h("div", { class: "row", style: "display:none" },
field("--o1-m", o1M, t("import.adv.o1mHint")),
field("--o1-window", o1Window, t("import.adv.o1windowHint")),
field("--o1-sink", o1Sink, t("import.adv.o1sinkHint")));
o1Sel.addEventListener("change", () => {
const on = !!o1Sel.value;
o1Knobs.style.display = on ? "" : "none";
o1SpecRow.style.display = o1Sel.value === "deep" || o1Sel.value === "custom" ? "" : "none";
o1DeepIn.parentElement.style.display = o1Sel.value === "deep" ? "" : "none";
o1CustomIn.parentElement.style.display = o1Sel.value === "custom" ? "" : "none";
});
const advBody = h(
"div",
{ class: "adv-body", style: "display:none" },
@@ -137,6 +172,9 @@ export function renderImport() {
h("div", { class: "row" },
field(t("import.adv.shard"), shardIn),
field("MTP", h("label", { class: "check" }, skipMtp, t("import.adv.skipMtp")))),
field(t("import.adv.o1"), o1Sel, t("import.adv.o1Hint")),
o1SpecRow,
o1Knobs,
);
const advToggle = h("button", { class: "btn ghost sm" }, "⚙ " + t("import.adv.title"));
advToggle.addEventListener("click", () => {
@@ -156,6 +194,19 @@ export function renderImport() {
if (vbitSel.value) params.vbit_shape = vbitSel.value;
if (meanBitsIn.value) params.mean_bits = parseFloat(meanBitsIn.value);
if (shardIn.value) params.shard_max_gb = parseFloat(shardIn.value);
if (o1Sel.value) {
// spec: all | deepN | explicit layer list
params.o1 =
o1Sel.value === "deep" ? "deep" + (parseInt(o1DeepIn.value) || 12)
: o1Sel.value === "custom" ? o1CustomIn.value.trim()
: "all";
if (!params.o1) delete params.o1;
else {
if (o1M.value) params.o1_m = parseInt(o1M.value);
if (o1Window.value) params.o1_window = parseInt(o1Window.value);
if (o1Sink.value) params.o1_sink = parseInt(o1Sink.value);
}
}
goBtn.disabled = true;
try {
await api.startImport(params);
@@ -316,6 +367,46 @@ export function renderImport() {
);
}
function featuredCard(item) {
const card = h(
"div",
{ class: "hf-card featured-card" },
h("div", { class: "row", style: "align-items:center; justify-content:space-between;" },
h("div", {},
h("div", { class: "hf-id" }, item.label),
h("div", { class: "hf-meta" },
h("span", { class: "badge" }, item.badge),
h("span", { class: "muted" }, item.id)
)
),
h("button", { class: "btn primary sm" }, "⬇ " + t("import.directDownload"))
)
);
card.querySelector("button").addEventListener("click", async (e) => {
e.stopPropagation();
try {
await api.startImport({
repo: item.id,
quant: item.quant || "READY_CMF",
name: item.id.split("/").pop().toLowerCase(),
});
toast(t("import.started"), "ok");
refreshJobs();
jobsHost.scrollIntoView({ behavior: "smooth", block: "nearest" });
} catch (err) {
toast(err.message, "err");
}
});
return card;
}
const featuredHost = h(
"div",
{ class: "card featured-section" },
h("div", { class: "card-title" }, "⚡ " + t("import.featuredTitle")),
h("div", { class: "hf-grid" }, ...FEATURED_REPOS.map(featuredCard))
);
const root = h(
"div",
{},
@@ -323,6 +414,7 @@ export function renderImport() {
h("h2", {}, t("nav.import")),
h("div", { class: "muted" }, t("import.subtitle"))),
jobsHost, // progress panel on top, above the search
featuredHost, // featured ready-to-run models
h("div", { class: "card" }, searchIn, results),
formHost
);
@@ -330,3 +422,4 @@ export function renderImport() {
refreshJobs(); // resume: show any running/recent conversions immediately
return root;
}
+103 -22
View File
@@ -1,5 +1,5 @@
// Models — pool table with add/edit modal, probe, and per-model secret entry.
import { h, mount, modal, toast, confirmDialog, money } from "../ui.js";
import { h, mount, modal, toast, confirmDialog, money, opt } from "../ui.js";
import { t } from "../i18n.js";
import { api } from "../api.js";
import { appState } from "../app.js";
@@ -7,9 +7,6 @@ import { appState } from "../app.js";
function field(label, control, hint) {
return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
}
function opt(v, label, sel) {
return h("option", { value: v, selected: sel ? true : null }, label || v);
}
function keyBadge(src) {
const map = { store: "store", env: "env", missing: "bad", none: "" };
@@ -54,7 +51,16 @@ function modelForm(meta, existing) {
const tierSel = h("select", {}, ...(meta.cost_tiers || ["cheap"]).map((c) => opt(c, c, c === m.cost_tier)));
const priceInIn = h("input", { type: "number", step: "0.01", value: m.price_in ?? 0 });
const priceOutIn = h("input", { type: "number", step: "0.01", value: m.price_out ?? 0 });
const capInputs = (meta.caps || ["tools", "vision"]).map((c) => {
const CAP_EMOJIS = {
tools: "🔧 tools",
vision: "👁️ vision",
audio: "🎙️ audio",
reasoning: "🧠 reasoning",
json: "📄 json",
caching: "⚡ caching"
};
const capInputs = (meta.caps || Object.keys(CAP_EMOJIS)).map((c) => {
const cb = h("input", { type: "checkbox", checked: (m.caps || []).includes(c) ? true : null });
return { c, cb };
});
@@ -73,25 +79,55 @@ function modelForm(meta, existing) {
capInputs.forEach(({ c, cb }) => { cb.checked = d.caps.includes(c); });
keyIn.placeholder = d.needsKey ? t("models.form.apiKeyPlaceholder") : t("models.form.apiKeyNotNeeded");
mount(keyStatus, "");
// Automatically pre-fetch models (especially useful for OpenRouter which doesn't need a key)
testKey(true);
}
providerSel.addEventListener("change", applyDefaults);
if (!existing) applyDefaults();
// Auto-fill the id from the model name (until the user edits id by hand).
if (!existing) {
modelIn.addEventListener("input", () => {
if (!idIn.value || idIn._auto) {
idIn.value = slugId(modelIn.value);
idIn._auto = true;
let cachedModels = [];
const autofillData = () => {
console.log("autofillData triggered. modelIn.value:", modelIn.value);
console.log("cachedModels length:", cachedModels.length);
if (!existing && (!idIn.value || idIn._auto)) {
idIn.value = slugId(modelIn.value);
idIn._auto = true;
console.log("auto-filled idIn:", idIn.value);
}
const found = cachedModels.find((m) => (m.id || m) === modelIn.value);
console.log("found model:", found);
if (found) {
if (found.price_in !== undefined) {
priceInIn.value = parseFloat(Number(found.price_in).toFixed(4));
priceOutIn.value = parseFloat(Number(found.price_out).toFixed(4));
console.log("Updated prices:", priceInIn.value, priceOutIn.value);
}
});
if (found.caps && Array.isArray(found.caps)) {
capInputs.forEach(({ c, cb }) => {
cb.checked = found.caps.includes(c);
});
console.log("Updated caps:", found.caps);
}
}
};
modelIn.addEventListener("input", autofillData);
modelIn.addEventListener("change", autofillData);
if (!existing) {
idIn.addEventListener("input", () => { idIn._auto = false; });
} else {
console.log("Editing existing model, fetching testKey(true)");
testKey(true);
}
async function testKey() {
if (!baseUrlIn.value.trim()) { mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?")); return; }
async function testKey(silent = false) {
if (!baseUrlIn.value.trim()) {
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, t("models.form.baseUrl") + "?"));
return;
}
testBtn.disabled = true;
mount(keyStatus, t("models.form.testing"));
if (!silent) mount(keyStatus, t("models.form.testing"));
try {
const r = await api.providerModels({
provider: providerSel.value,
@@ -100,24 +136,61 @@ function modelForm(meta, existing) {
api_key_env: envFor(),
});
if (r.ok) {
dl.replaceChildren(...(r.models || []).map((mm) => h("option", { value: mm })));
mount(keyStatus, h("span", { style: "color:var(--good)" }, "✓ " + t("models.form.foundModels", { n: r.count })));
modelIn.focus();
cachedModels = r.models || [];
dl.replaceChildren(...cachedModels.map((mm) => h("option", { value: mm.id || mm })));
let msg = "✓ " + t("models.form.foundModels", { n: r.count });
if (r.balance !== undefined) {
msg += ` (Balance: $${r.balance.toFixed(4)})`;
}
mount(keyStatus, h("span", { style: "color:var(--good)" }, msg));
if (!silent) modelIn.focus();
} else {
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + (r.error || t("common.error"))));
}
} catch (e) {
mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
if (!silent) mount(keyStatus, h("span", { style: "color:var(--bad)" }, "✗ " + String(e.message || e)));
}
testBtn.disabled = false;
}
testBtn.addEventListener("click", testKey);
testBtn.addEventListener("click", () => testKey(false));
keyIn.addEventListener("change", () => testKey(true));
const keyHint =
existing && existing.key_source === "store" ? t("models.form.keyStored")
: existing && existing.key_source === "env" ? t("models.form.keyFromEnv")
: t("models.form.apiKeyHint");
// ⚙ Generation & Thinking Parameters
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: m.temperature ?? "" });
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: m.top_p ?? "" });
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: m.max_tokens ?? "" });
const thinkIn = h("input", { type: "number", min: "0", placeholder: "1024", value: m.think_budget ?? "" });
const o1Sel = h("select", {},
h("option", { value: "" }, t("import.adv.auto")),
h("option", { value: "all" }, "all — O(1) Attention"),
h("option", { value: "deep" }, "deep — O(1) Deep"),
h("option", { value: "off" }, "off — Standard"));
if (m.o1) o1Sel.value = m.o1;
const skipMtpCb = h("input", { type: "checkbox", checked: m.skip_mtp ? true : null });
const sysPromptIn = h("textarea", { rows: 2, placeholder: t("models.form.systemPromptPlaceholder"), value: m.system_prompt || "" });
const advParams = h(
"details",
{ class: "adv-params", style: "margin-top:12px;padding:10px;border:1px solid var(--border);border-radius:6px;" },
h("summary", { style: "font-weight:600;cursor:pointer;" }, "⚙ " + t("models.form.paramsTitle")),
h("div", { class: "row", style: "margin-top:8px" },
field(t("models.form.temperature"), tempIn, t("models.form.temperatureHint")),
field(t("models.form.topP"), topPIn, t("models.form.topPHint"))),
h("div", { class: "row" },
field(t("models.form.maxTokens"), maxTokIn, t("models.form.maxTokensHint")),
field(t("models.form.thinkBudget"), thinkIn, t("models.form.thinkBudgetHint"))),
h("div", { class: "row" },
field("O(1) Attention", o1Sel, t("import.adv.o1Hint")),
field("MTP", h("label", { class: "check" }, skipMtpCb, t("import.adv.skipMtp")))),
field(t("models.form.systemPrompt"), sysPromptIn, t("models.form.systemPromptHint"))
);
const node = h(
"div",
{},
@@ -128,7 +201,8 @@ function modelForm(meta, existing) {
dl,
h("div", { class: "row" }, field(t("models.form.kind"), kindSel), field(t("models.form.costTier"), tierSel)),
h("div", { class: "row" }, field(t("models.form.priceIn"), priceInIn), field(t("models.form.priceOut"), priceOutIn)),
field(t("models.form.caps"), h("div", { class: "flex wrap" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check" }, cb, c))))
field(t("models.form.caps"), h("div", { class: "flex wrap", style: "gap:10px" }, ...capInputs.map(({ c, cb }) => h("label", { class: "check", style: "margin:0" }, cb, CAP_EMOJIS[c] || c)))),
advParams
);
const getValue = () => {
@@ -147,6 +221,13 @@ function modelForm(meta, existing) {
price_out: parseFloat(priceOutIn.value) || 0,
api_key_env: hasKey ? keyEnv : null,
caps: capInputs.filter(({ cb }) => cb.checked).map(({ c }) => c),
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
think_budget: thinkIn.value !== "" ? parseInt(thinkIn.value) : null,
o1: o1Sel.value || null,
skip_mtp: skipMtpCb.checked,
system_prompt: sysPromptIn.value.trim() || null,
},
secret: typedKey,
keyEnv,
+102 -6
View File
@@ -253,9 +253,103 @@ export async function renderSettings() {
} catch (e) { mount(st, String(e.message || e)); }
checkBtn.disabled = false;
});
const row = h("div", { class: "flex wrap", style: "gap:6px;align-items:center;margin:6px 0" },
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), removeBtn, st);
const entry = { node: row, read: () => ({ id: idIn.value.trim() || autoId(modelSel.value), model: modelSel.value.trim(), port: parseInt(portIn.value) || 8090, threads: parseInt(threadsIn.value) || 0, gpu: gpuIn.checked }) };
// ── O(1) Nyström Attention ──
// NOT a "reasoning mode". This is a memory-efficient streaming attention
// kernel (Nyström/landmark) that replaces standard KV-cache attention,
// giving O(1) memory per decode step instead of O(n).
const o1Sel = h("select", { style: "width:120px", title: "O(1) Nyström attention layer spec: which layers replace KV-cache with streaming Nyström attention" },
h("option", { value: "" }, "O(1) авто"),
h("option", { value: "all" }, "O(1) все слои"),
h("option", { value: "deep" }, "O(1) глубокие"),
h("option", { value: "off" }, "O(1) выкл."));
if (sv.o1) o1Sel.value = sv.o1;
const o1mIn = h("input", { type: "number", min: "4", placeholder: "32", value: sv.o1_m ?? "", style: "width:55px", title: "Landmark budget (≥4, default 32). More landmarks ≠ better: m=64 measured worse." });
const o1wIn = h("input", { type: "number", min: "1", placeholder: "128", value: sv.o1_window ?? "", style: "width:60px", title: "Exact-window width — main quality lever (default 128)" });
const o1sinkIn = h("input", { type: "number", min: "0", placeholder: "4", value: sv.o1_sink ?? "", style: "width:50px", title: "Permanent exact sink keys, StreamingLLM discipline (default 4)" });
// ── Generation parameters ──
const tempIn = h("input", { type: "number", step: "0.05", min: "0", max: "2.0", placeholder: "0.7", value: sv.temperature ?? "", style: "width:60px" });
const topPIn = h("input", { type: "number", step: "0.05", min: "0", max: "1.0", placeholder: "0.9", value: sv.top_p ?? "", style: "width:60px" });
const maxTokIn = h("input", { type: "number", min: "1", placeholder: "2048", value: sv.max_tokens ?? "", style: "width:75px" });
// think_budget = 0 means reasoning disabled (enable_thinking=false);
// null = model decides; N>0 = token budget for the <think> block.
const noThinkCb = h("input", { type: "checkbox", checked: (sv.think_budget === 0) ? true : null, title: "Отключить режим размышлений (think_budget=0, enable_thinking=false для Qwen3/3.5)" });
const thinkIn = h("input", { type: "number", min: "1", placeholder: "авто", value: (sv.think_budget && sv.think_budget > 0) ? sv.think_budget : "", style: "width:75px", title: "Бюджет токенов на размышления (think_budget). Пустое = модель решает сама." });
if (sv.think_budget === 0) thinkIn.disabled = true;
noThinkCb.addEventListener("change", () => {
thinkIn.disabled = noThinkCb.checked;
if (noThinkCb.checked) thinkIn.value = "";
});
const sysIn = h("input", { placeholder: "System prompt (optional)", value: sv.system_prompt || "", style: "flex:1;min-width:200px" });
// ── MTP (Multi-Token Prediction): speculative decoding, NOT reasoning ──
const skipMtpCb = h("input", { type: "checkbox", checked: sv.skip_mtp ? true : null, title: "Disable MTP speculative decoding (env CMF_MTP=0)" });
// ── Settings drawer ──
const drawerStyle = "display:none;width:100%;margin-top:6px;padding:10px 12px;background:var(--bg-subtle);border-radius:6px;border:1px solid var(--border)";
const groupStyle = "display:flex;flex-wrap:wrap;gap:8px;align-items:center;padding:4px 0";
const labelStyle = "font-size:11px;color:var(--text-muted);font-weight:600;text-transform:uppercase;letter-spacing:0.5px;width:100%;margin-bottom:2px";
const drawer = h("div", { style: drawerStyle },
// Group 1: O(1) Nyström Attention
h("div", { style: labelStyle }, "O(1) Nyström Attention"),
h("div", { style: groupStyle },
h("span", { class: "small muted" }, "Слои:"), o1Sel,
h("span", { class: "small muted" }, "m:"), o1mIn,
h("span", { class: "small muted" }, "Window:"), o1wIn,
h("span", { class: "small muted" }, "Sink:"), o1sinkIn
),
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
// Group 2: Generation
h("div", { style: labelStyle }, "Генерация"),
h("div", { style: groupStyle },
h("span", { class: "small muted" }, "Temp:"), tempIn,
h("span", { class: "small muted" }, "Top-P:"), topPIn,
h("span", { class: "small muted" }, "Max токенов:"), maxTokIn
),
h("div", { style: groupStyle },
h("span", { class: "small muted" }, "System prompt:"), sysIn
),
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
// Group 3: Reasoning (thinking)
h("div", { style: labelStyle }, "Режим размышлений"),
h("div", { style: groupStyle },
h("label", { class: "check small", title: "Для моделей Qwen3/3.5: enable_thinking=false — модель отвечает напрямую без блока <think>" }, noThinkCb, "Отключить размышления"),
h("span", { class: "small muted" }, "Think budget:"), thinkIn
),
h("hr", { style: "border:none;border-top:1px solid var(--border);margin:6px 0" }),
// Group 4: Runtime
h("div", { style: labelStyle }, "Runtime"),
h("div", { style: groupStyle },
h("label", { class: "check small", title: "MTP — Multi-Token Prediction (спекулятивное декодирование). Отключение замедляет генерацию, но стабилизирует output." }, skipMtpCb, "Отключить MTP")
)
);
const gearBtn = h("button", { class: "btn small", type: "button", title: "⚙ Настройки модели" }, "⚙");
gearBtn.addEventListener("click", () => {
drawer.style.display = drawer.style.display === "none" ? "block" : "none";
});
const row = h("div", { class: "flex wrap cmf-server-row", style: "gap:6px;align-items:center;margin:6px 0" },
idIn, modelSel, portIn, checkBtn, threadsIn, h("label", { class: "flex", style: "gap:4px" }, gpuIn, "GPU"), gearBtn, removeBtn, st, drawer);
const entry = { node: row, read: () => ({
id: idIn.value.trim() || autoId(modelSel.value),
model: modelSel.value.trim(),
port: parseInt(portIn.value) || 8090,
threads: parseInt(threadsIn.value) || 0,
gpu: gpuIn.checked,
temperature: tempIn.value !== "" ? parseFloat(tempIn.value) : null,
top_p: topPIn.value !== "" ? parseFloat(topPIn.value) : null,
max_tokens: maxTokIn.value !== "" ? parseInt(maxTokIn.value) : null,
think_budget: noThinkCb.checked ? 0 : (thinkIn.value !== "" ? parseInt(thinkIn.value) : null),
o1: o1Sel.value || null,
o1_m: o1mIn.value !== "" ? parseInt(o1mIn.value) : null,
o1_window: o1wIn.value !== "" ? parseInt(o1wIn.value) : null,
o1_sink: o1sinkIn.value !== "" ? parseInt(o1sinkIn.value) : null,
skip_mtp: skipMtpCb.checked,
system_prompt: sysIn.value.trim() || null,
}) };
removeBtn.addEventListener("click", () => { const i = serverRows.indexOf(entry); if (i >= 0) serverRows.splice(i, 1); row.remove(); });
serverRows.push(entry);
serversWrap.appendChild(row);
@@ -442,9 +536,11 @@ export async function renderSettings() {
"div",
{ class: "card" },
h("div", { class: "card-head" }, h("h3", {}, t("settings.route"))),
field(t("settings.route.strategy"), strategySel),
h("div", { class: "row" }, field(t("settings.route.maxChars"), maxChars), field(t("settings.route.cacheTtl"), cacheTtl)),
field(t("settings.route.profile"), profileSel)
field(t("settings.route.strategy"), strategySel, null, t("settings.route.strategyHelp")),
h("div", { class: "row" },
field(t("settings.route.maxChars"), maxChars, null, t("settings.route.maxCharsHelp")),
field(t("settings.route.cacheTtl"), cacheTtl, null, t("settings.route.cacheTtlHelp"))),
field(t("settings.route.profile"), profileSel, null, t("settings.route.profileHelp"))
),
h(
"div",