- config.rs: validate() no longer errors on zero models — an empty pool is
allowed (warns instead), so the admin console comes up on a fresh install and
the user adds models in /admin -> Models (or points [cmf] at a local .cmf).
- gateway.example.toml: ships an EMPTY pool (no seed cmf-local/local-embed, empty
routing), so `cp example -> gateway.toml && run` shows an empty list — what a
new user expects.
The v0.2.3 CMF-integration commit referenced crate::promotion (main.rs),
router_client::RouterLastStatus (state.rs) and router_client::classify_status
(admin), but those definitions were not part of the commit, so a clean build
failed (E0583/E0432/E0425). Add the two source files; cargo build --release now
compiles.
The example config pool now shows one local CMF model (served by the managed
cortiq server) instead of local-qwen/gpt-4o-mini/claude-opus placeholders, with
routing pointing at it, and a documented [cmf] section (manage_server, threads,
gpu, ...). Add cloud/local providers by copying the [[models]] shape.
Expose the two speed knobs in the /admin Settings panel instead of TOML-only:
- [cmf].gpu (new) -> exports CMF_GPU=1 to the managed cortiq serve (Metal/wgpu).
On Apple Silicon decode is memory-bandwidth bound so this is ~neutral for
generation (helps prefill); off by default.
- [cmf].threads (added earlier) now has a UI number input.
- config.rs: CmfCfg.gpu bool (serde default false).
- cmf_runtime.rs: spawn_serve sets CMF_GPU=1 when gpu is on.
- admin/mod.rs: put_settings flags needs_restart on threads/gpu change (the
server is spawned once at startup with those env vars).
- settings.js: threads input + gpu checkbox + save payload + speed hint.
- i18n en/ru: threads/gpu/speedNote keys (other locales fall back to en, as the
whole cmf section already does there).
The gateway spawned `cortiq serve` with no thread hint, so the runtime ran the
decode loop single-threaded (~0.25 tok/s on a 15B CPU model). Export CMF_THREADS
from a new [cmf].threads config field (default 8) in spawn_serve.
Measured: 0.25 -> 0.81 tok/s (~3.2x) on qwopus-vbit7 (15B). Scaling is sublinear
(GatedDeltaNet state recurrence is serial per token, Amdahl-bound); a real jump
needs the GPU path.
Run models in the CMF (.cmf) format locally, via the native `cortiq`
CLI from crates.io — no Python and no external model server.
- cmf_runtime: install/update `cortiq-cli` from crates.io, spawn and
health-check `cortiq serve` as a managed local backend.
- import: model-import jobs shell out to the native `cortiq convert`
(HF repo -> .cmf, quant mapped Q8_2F->q8_2f / Q8_ROW->q8 / Q4->q4);
the Python converter is used only for advanced/exotic options.
- pipeline: local-first routing — serve from the local CMF model when
the router is disabled/unreachable, with a "local only" hard switch.
- registry/state: register the local CMF server as an OpenAI provider
and track local candidates.
- config: data-dir resolution + CmfCfg (auto-install/update, managed
server, host/port/model) and relaxed validation for the managed model.
- admin + settings UI (en/ru): "Local models (CMF)" settings card.
- docs/ROUTER.md + docs/ROUTER.ru.md: what the semantic router does, hosted vs
self-hosted, config, API contract, pricing, and the LAUNCH1 $1 Starter promo
- README (EN+RU): "The router (allaigate)" section with the promo callout
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- bench/tasks.jsonl (37 natural-language prompts, 7 task types) + bench/accuracy.py
- measured on the live allaigate router: 100% (37/37) vs keyword heuristic 32%
- BENCHMARKS.md accuracy section; README (EN+RU) perf+accuracy tables and test matrix
- run.sh runs accuracy when CORTIQ_ROUTER_KEY is set
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- GET /v1/models: pool + virtual cortiq-auto
- POST /v1/completions (legacy): non-stream + streaming text_completion
- POST /route: native passthrough returning the routing decision
- all inbound protocol adapters now implemented; admin meta + README/CHANGELOG updated
- bump to 0.2.1
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- POST /mcp (JSON-RPC 2.0): initialize, ping, tools/list, tools/call; notifications -> 202
- tools: cortiq_route (classify -> routing decision) and cortiq_chat (route -> answer)
- gated by protocols.mcp; admin meta marks it implemented
- docs: README/README.ru protocol tables, highlights and status updated for
streaming, Anthropic, embeddings, semantic cache and MCP
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by
routing signature; tracks hit rate and model cost saved
- pipeline: embed routing text, look up before the model call (route_source=cache on
hit), insert successful responses; stats record cache hits
- config [cache] + admin settings card; dashboard shows hit-rate and savings
- embedding model resolved from the pool (or first kind="embedding")
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- release.yml: add x86_64-pc-windows-msvc target, OS-aware packaging (.exe + .zip)
- ci.yml: split into lint (fmt/clippy) + test matrix (ubuntu/windows/macos)
- use getrandom for the admin/key token CSPRNG (secure on Windows too)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- bump version to 0.1.0; add crates.io metadata (include list, homepage, rust-version)
- add .github/workflows/release.yml: on tag v* → publish to crates.io + GitHub Release with binaries
- README: absolute links/images and badges so it renders on crates.io; add install section
- CHANGELOG: 0.1.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
