Two bugs:
1. run_stream() never merged per-model/server config (think_budget,
temperature, etc.) into the request — unlike run_inner(). Now both
paths apply the same config merge.
2. to_wire() was missing the enable_thinking bool conversion line
(lost in a prior revert). Restored: think_budget=0 → enable_thinking=false.
- New Advanced controls on the import page: O(1) attention spec (off / all /
deep N / explicit layer list) plus the --o1-m / --o1-window / --o1-sink knobs
with validated defaults (32 / 128 / 4) and explanatory hints.
- Backend passes --o1* to the native `cortiq convert`; fails fast with clear
messages when combined with Python-only options, used on GGUF imports, or
when the installed cortiq predates 0.2.0 (the hint needs the new runtime).
- The hint is recorded in the .cmf header (weights byte-identical); the runtime
applies it at load. Verified end-to-end: converted a model with --o1 all and
the runtime logged "o1 nystrom attention: 2/2 layer(s)".
- Stats snapshot now computes the KPI totals AND the breakdown from the
in-window per-minute buckets (previously they used all-time accumulators, so
switching 1h/24h/7d changed only the chart, not the numbers). Buckets now
carry failovers + per-group breakdown; Prometheus keeps all-time counters.
- New DELETE /admin/api/stats clears all in-memory stats and truncates the
JSONL log; the dashboard gets a "Clear logs" button (with confirm).
- The model id is now auto-filled from the model name as you type, and if left
blank it's derived on save (slug of the last path segment, e.g.
"qwen/qwen3.7-max" → "qwen3-7-max") — saving no longer errors on a blank id.
- probe/test auth failures (401 / "User not found" / "No auth") now append a
hint to check the API key (for OpenRouter: use an inference key, not a
provisioning/management key).
- ui.js: mount() now wraps non-node children in text nodes (like h()), so
mount(el, "") no longer throws in appendChild — this is what broke the
"Add model" form (applyDefaults cleared a status via mount(x,"")).
- probe_model: resolve managed local models too (they are synthesized into the
registry, not [[models]]) — probing them no longer returns "model not found".
- models.js: await reload() after delete so the row is removed immediately.
- Providers: add openrouter (https://openrouter.ai/api/v1) and lmstudio
(http://localhost:1234/v1), both OpenAI-compatible. Selecting a provider now
auto-fills its verified base URL, cost tier and default capabilities.
- API key is typed directly in the form and stored securely as a secret under the
provider's env name — no manual env var to manage.
- New POST /admin/api/provider/models validates the key and lists the provider's
models; a "Test & load models" button fills a searchable dropdown. The model
field now sits below the key (some providers list models from the key).
- Fuller capability set: tools, vision, audio, reasoning, json, caching.
- New GET /admin/api/cmf/files lists the .cmf files (and tensor-dir .cmf) under
models_dir, with sizes.
- Settings → Local models: the per-row model field is now a dropdown of those
files instead of a hand-typed path, and auto-fills the id from the filename.
- The gateway can now run several local .cmf models at once: a [[cmf.servers]]
list, one supervised `cortiq serve` per entry on its own port, each registered
in the pool and independently routable/pinnable. The single legacy
local_model/local_port still works (back-compat via effective_servers()).
- Settings → Local models is now a rows editor (id · path · port + check · threads
· GPU) with add/remove and per-server serving status.
- Deleting a managed model stops just its server and removes its file — unless
another model still points at the same file (shared-file guard).
- A pinned playground model that is unavailable now errors clearly instead of
silently answering from the managed local model (route_source no longer lies).
- The managed CMF model is surfaced in GET /admin/api/models (marked "managed",
shown read-only in the UI) so the real running model is visible and pinnable
without a hand-maintained [[models]] entry; a fresh install still lists nothing.
- Deleting a model now removes its local .cmf physically (file or tensor-dir);
deleting the managed model also stops the server and clears the [cmf] config.
Routing is stripped of the deleted id so the config still validates.
- import.rs / promotion.rs: sort_by(|a,b| b.k.cmp(&a.k)) -> sort_by_key(Reverse(k))
- pipeline.rs: skip_while(!p).next() -> find(p)
Behaviour-preserving; completes the CI fix started by the cargo fmt commit.
- Settings: "?" help tooltips on every option; router key auto-probes the
connection right after saving (no more hanging "unavailable")
- Local models card reworked into 3 clear steps (install cortiq → add model →
run locally); removed the hand-entered .cmf path / pool-id fields — models
are added via the Import tab and the managed server registers them
- Fresh install ships an EMPTY model pool; the local .cmf is auto-registered by
the managed CMF server, so it needs no static [[models]] entry
- New: port-availability check — GET /admin/api/cmf/port + "Check port" button
with next-free-port suggestion. bind()+connect() probe so Docker-proxied /
dual-stack ports (e.g. 8081) are correctly detected as occupied
- /admin/api/cmf now reports the serving model + port; card shows "▶ serving …"
- On-demand runtime install: POST /admin/api/cmf/install
The crates.io/release build shipped a STALE frontend (committed web/js was behind
the working tree), so the admin console hung on load. Commit the current working
web/js (app, api, dashboard, login, import view, i18n de/es/fr/tr/zh, css) and the
backend WIP (import.rs, admin/assets.rs) so the released console matches the dev
build. Verified: clean `cargo build --release`, empty-pool start, working
dashboard served.
- config.rs: validate() no longer errors on zero models — an empty pool is
allowed (warns instead), so the admin console comes up on a fresh install and
the user adds models in /admin -> Models (or points [cmf] at a local .cmf).
- gateway.example.toml: ships an EMPTY pool (no seed cmf-local/local-embed, empty
routing), so `cp example -> gateway.toml && run` shows an empty list — what a
new user expects.
The v0.2.3 CMF-integration commit referenced crate::promotion (main.rs),
router_client::RouterLastStatus (state.rs) and router_client::classify_status
(admin), but those definitions were not part of the commit, so a clean build
failed (E0583/E0432/E0425). Add the two source files; cargo build --release now
compiles.
The example config pool now shows one local CMF model (served by the managed
cortiq server) instead of local-qwen/gpt-4o-mini/claude-opus placeholders, with
routing pointing at it, and a documented [cmf] section (manage_server, threads,
gpu, ...). Add cloud/local providers by copying the [[models]] shape.
Expose the two speed knobs in the /admin Settings panel instead of TOML-only:
- [cmf].gpu (new) -> exports CMF_GPU=1 to the managed cortiq serve (Metal/wgpu).
On Apple Silicon decode is memory-bandwidth bound so this is ~neutral for
generation (helps prefill); off by default.
- [cmf].threads (added earlier) now has a UI number input.
- config.rs: CmfCfg.gpu bool (serde default false).
- cmf_runtime.rs: spawn_serve sets CMF_GPU=1 when gpu is on.
- admin/mod.rs: put_settings flags needs_restart on threads/gpu change (the
server is spawned once at startup with those env vars).
- settings.js: threads input + gpu checkbox + save payload + speed hint.
- i18n en/ru: threads/gpu/speedNote keys (other locales fall back to en, as the
whole cmf section already does there).
The gateway spawned `cortiq serve` with no thread hint, so the runtime ran the
decode loop single-threaded (~0.25 tok/s on a 15B CPU model). Export CMF_THREADS
from a new [cmf].threads config field (default 8) in spawn_serve.
Measured: 0.25 -> 0.81 tok/s (~3.2x) on qwopus-vbit7 (15B). Scaling is sublinear
(GatedDeltaNet state recurrence is serial per token, Amdahl-bound); a real jump
needs the GPU path.
Run models in the CMF (.cmf) format locally, via the native `cortiq`
CLI from crates.io — no Python and no external model server.
- cmf_runtime: install/update `cortiq-cli` from crates.io, spawn and
health-check `cortiq serve` as a managed local backend.
- import: model-import jobs shell out to the native `cortiq convert`
(HF repo -> .cmf, quant mapped Q8_2F->q8_2f / Q8_ROW->q8 / Q4->q4);
the Python converter is used only for advanced/exotic options.
- pipeline: local-first routing — serve from the local CMF model when
the router is disabled/unreachable, with a "local only" hard switch.
- registry/state: register the local CMF server as an OpenAI provider
and track local candidates.
- config: data-dir resolution + CmfCfg (auto-install/update, managed
server, host/port/model) and relaxed validation for the managed model.
- admin + settings UI (en/ru): "Local models (CMF)" settings card.
- docs/ROUTER.md + docs/ROUTER.ru.md: what the semantic router does, hosted vs
self-hosted, config, API contract, pricing, and the LAUNCH1 $1 Starter promo
- README (EN+RU): "The router (allaigate)" section with the promo callout
- bench/tasks.jsonl (37 natural-language prompts, 7 task types) + bench/accuracy.py
- measured on the live allaigate router: 100% (37/37) vs keyword heuristic 32%
- BENCHMARKS.md accuracy section; README (EN+RU) perf+accuracy tables and test matrix
- run.sh runs accuracy when CORTIQ_ROUTER_KEY is set
- GET /v1/models: pool + virtual cortiq-auto
- POST /v1/completions (legacy): non-stream + streaming text_completion
- POST /route: native passthrough returning the routing decision
- all inbound protocol adapters now implemented; admin meta + README/CHANGELOG updated
- bump to 0.2.1
- POST /mcp (JSON-RPC 2.0): initialize, ping, tools/list, tools/call; notifications -> 202
- tools: cortiq_route (classify -> routing decision) and cortiq_chat (route -> answer)
- gated by protocols.mcp; admin meta marks it implemented
- docs: README/README.ru protocol tables, highlights and status updated for
streaming, Anthropic, embeddings, semantic cache and MCP
- cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by
routing signature; tracks hit rate and model cost saved
- pipeline: embed routing text, look up before the model call (route_source=cache on
hit), insert successful responses; stats record cache hits
- config [cache] + admin settings card; dashboard shows hit-rate and savings
- embedding model resolved from the pool (or first kind="embedding")
infosave2007