cortiq-gateway/CHANGELOG.md

Changelog

All notable changes to this project are documented here. The format is based on Keep a Changelog.

[0.2.40] - 2026-08-08

Added

• Per-key spend accounting on the API-keys page: requests, tokens in/out and cost per key for a selectable window (24h / 7d / 30d) plus a cumulative all-time cost column — "which department spends what" at a glance.
• Bounded stats on disk: once stats.jsonl outgrows stats.max_file_mb (default 16, 0 disables), records older than the retention window fold into cumulative per-account totals in a small sidecar file (tmp+rename on both) and leave the log. All-time accounting survives compaction and restarts; nothing is ever counted twice.
• Per-complexity models inside a task rule: each task is edited as rows of "model + complexity" pairs — [routing.tasks.code] low=["cheap"] high=["best"] any=[...]. The tier-specific pick leads its own tier, any (and legacy plain arrays) leads everywhere, the global tier chain and default always remain as fallback, and high verdicts still escalate over any-tier pins.

Changed

• Benchmark hard probes are generated from wider interdisciplinary pools (6 families each) and retry once with a fresh template — the router's novelty decay no longer flips them red.

Fixed

• The task-rule badge is honest: it appears only when the rule actually chose the head of the candidate chain, not whenever a rule merely exists for the task.

[0.2.39] - 2026-08-07

Added

• Benchmark, redesigned into the console (⏱ sidebar entry): two explicit modes. Smart routing (default) sends every probe through cortiq-auto and judges the ROUTER — task label vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge — with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a "requests went to" per-model distribution. Individual models compares pinned models head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run × and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
• Hard probes that exercise the high tier: two route-judged tests (science / ops pools) generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings decay with the router's novelty signal. Pass = the router said high and the request escalated; the tiny token budget keeps them at cents even on the priciest model.
• One-paste harness setup on the API-keys page: Claude Code (native Anthropic /v1/messages, now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline — each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses already installed on its machine (✓ on the tab, "already configured for this gateway" status).
• Stale-tab detection: open consoles notice a gateway restart via the meta start marker and show a reload banner instead of silently running an old interface.
• Log viewer Clear button (DELETE /admin/api/logs).

Changed

• High complexity escalates over task rules: at tier high the tier chain leads and the task rule demotes to fallback — a code → cheap-model rule no longer pins the hardest prompts to the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
• Honest stream costs: the final cost rides a closing SSE event (cortiq_cost_usd) since the initial header is written before generation; money renders with 4 decimals (last two smaller) and grows precision below $0.0001 instead of collapsing to $0.0000.
• Oracle-sized timeouts: router default 800 ms → 15 s (hosted escalations take ~10 s); provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall cap so long generations and streams are never cut.
• Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-<think>), router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
• New-key dialog simplified to account / rate limit / optional custom value.

Fixed

• cortiq bench results showed "error" with newer cortiq releases that pretty-print --json: the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers steady decode speed (the old heuristic could report allocs_per_token as tok/s).

[0.2.38] - 2026-08-07

Added

• Image & video generation in the Playground: three tabs now — Chat, Image, Video. Image runs cortiq imagine over a local text-to-image .cmf (Lumina) with width/height/steps/CFG/seed, live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser with one-click PNG export. Video runs cortiq animate (MiniMax-H3) with frames/steps/quality and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens opportunistically when ffmpeg exists). Local .cmf files are classified chat/image/video via cortiq info; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56.
• Task-based routing: [routing.tasks] maps a router task label to an ordered model list. A matching rule's models lead the candidate chain and the complexity chain follows as fallback, so a task pin never weakens failover. The Routing page builds the editor from the account's live taxonomy (GET /v1/taxonomies), shows each task's share of 7-day traffic, and supports custom labels. Responses whose chain was headed by a rule report route_source "router+task-rule".
• Generation gallery under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete), and generations queue up (with position shown) instead of being rejected.
• Chat is a real dialogue: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live metrics: token counts, tok/s and time-to-first-token.
• Exact stream token counts when the model server provides usage; the gateway fallback counts content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate.
• Jobs survive restarts: import and media job lists persist to <data>/jobs-*.json and are reconciled on start — the Register button no longer vanishes because the gateway restarted.
• Admin console, ops batch: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy); a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download; a Restart-gateway button; an update-available notice (crates.io checked twice a day); config export as the real gateway.toml and TOML import (validated, hot-reloaded); "Connect your app" snippets (curl/Python/JS) on the keys page; one-click cortiq bench --json on the Models page.
• Docker ships the cortiq binary (built in the builder stage, plus libvulkan1): local .cmf chat models and media generation now work in containers, where auto-install never could.
• Security: an open admin console listening on 0.0.0.0 outside Docker warns in the log and shows a dismissible banner.

Fixed

• The default config pointed at a placeholder localhost:8080 router with a nonexistent taxonomy, leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now targets the hosted allaigate router and degrades gracefully until a key is set.
• The wizard's Finish lands on the Playground instead of the Models list.

[0.2.37] - 2026-08-06

Added

• Zero-argument startup: cortiq-gateway now runs with no flags. The config is created on first launch (repo config/gateway.toml when run from a checkout, otherwise ~/.cortiq-gateway/gateway.toml, or $CORTIQ_GATEWAY_HOME), the browser opens by itself, and a busy port is swapped for the next free one. --no-browser / --no-port-search opt out; --config still takes an explicit path.
• No admin token required by default: the console opens without auth. Set --admin-token, [admin].token or [admin].token_env to protect it.
• First-run wizard (/admin#/onboarding): pick a language, download a ready-made .cmf model, and finish — Finish connects every completed download as a local model, starts its cortiq serve and points routing.default at it. Nothing to edit by hand.
• Ready-made models are read live from the infosave HuggingFace account in both the wizard and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a delete button. Newly published models appear without a gateway update.
• Windows-friendly data dir: %APPDATA% / %LOCALAPPDATA% / %USERPROFILE% are used before falling back, so running the exe from Downloads never writes into Program Files.
• Full localization: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr). Import, Settings → Local models, the model form and the whole first-run wizard were English-only in de/fr/es/zh/tr before this release.

Fixed

• Download progress jumped straight to 100% and then hung: with -L, curl prints a complete 0→100% progress bar for the redirect hop before the transfer starts, and that bar was being parsed as the download's progress. Progress is now derived from bytes on disk against the size reported by the HuggingFace tree (or a HEAD request), so the bar and the reported done / total bytes track the real transfer.
• Re-downloading over an existing file reported 100% instantly: the previous copy is now removed before the transfer starts, so the size poller cannot read stale bytes.
• Downloaded models were not recognised: the admin UI derived the output file name independently of the gateway, so a finished download did not mark its card as downloaded and the delete button never appeared. The gateway now returns the exact output stem (local_name) and the UI sends it back verbatim for download, progress and delete.
• Registering a local model produced an unusable pool entry: with manage_server = false the model was added to the static pool pointing at cortiq_server_url, where nothing listens — every call failed with upstream unavailable. Registration now always creates a managed server the gateway runs itself, and repairs a stale static entry for the same file.
• Cancel did nothing during a sharded download: cancellation was only checked between part files, so a multi-GB part had to finish first. The running transfer is now aborted immediately and the partial output removed.
• Deleting a model left the config broken: removing a .cmf now also unregisters the model and stops its managed server instead of leaving a pool entry pointing at a missing file.
• Two variants of one repo collapsed into a single job row; jobs are now keyed by repo + variant.
• Curl's stderr is captured into the job log, so a failed download explains itself instead of showing a bare exit code.

[0.2.36] - 2026-07-25

Fixed

• Docker image failed to start when built fresh (GLIBC_2.39 not found): the floating rust:1-slim builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
• Docker model downloads (curl exit code 23): the runtime image was missing curl, and the default models dir ($HOME/.cortiq-gateway/models) was not writable by the container user, so importing a ready .cmf model failed with a write error. The image now bundles curl, keeps all gateway state under /app/data (CORTIQ_GATEWAY_HOME), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass --user 10001 to harden when not using host bind mounts.
• Docker Compose persistence: added a cortiq-data volume at /app/data so downloaded models and stats survive docker compose up --force-recreate.
• Clear import errors: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.

[0.2.35] - 2026-07-23

Fixed

• Streaming Pipeline System Prompt Merging: Fixed system prompt injection in run_stream so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
• CMF Reasoning Mode Compatibility: Fully integrated with cortiq-engine v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.

[0.2.34] - 2026-07-23

Added

• Responsive mobile admin console: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
• Accessible mobile interactions: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.

Fixed

• Add model dialog: Restored the missing opt helper import that prevented the Add model button from opening its form.
• Local model settings on phones: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.

[0.2.31] - 2026-07-22

Added

• Structured O(1) Nyström Attention & Reasoning Control: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (o1_m, o1_window, o1_sink), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to think_budget = 0 for Qwen3/3.5 models), and Runtime (skip_mtp via CMF_MTP=0).
• Persistent Admin Token Configuration: Support for explicit token field in config/gateway.toml under [admin] block to prevent authorization invalidation across restarts.

Fixed

• JavaScript UI Helper Restoration: Restored opt(...) helper in web/js/ui.js and web/js/views/import.js to ensure HF model import modal options render smoothly.
• Engine Command Flag Accuracy: Fixed CMF_MTP=0 environment flag pass-through for cortiq serve processes.

[0.2.30] - 2026-07-22

Added

• Per-Model Generation & Reasoning Parameters: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (o1), MTP (Multi-Token Prediction toggle skip_mtp), and default System Prompt.

Fixed

• Duplicate Local Model Registration: Fixed duplicate managed model id conflict in register_import by cleanly routing managed CMF models directly into [[cmf.servers]] with automatic free port allocation without pushing duplicate entries to [[models]].

Fixed

• Quantization Badge & Title for Pre-Converted Models: Automatically detect quantization (Q1, Q1T) for ready-to-run .cmf models (e.g. infosave/Bonsai-27Bcmf) from repo name and downloaded filename, resolving default Q8_2F badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").

Fixed

• Automatic CMF Local Server Lifecycle: Registering an imported .cmf model automatically adds it to [[cmf.servers]] and reloads the runtime (cmf_runtime::manage), automatically spawning cortiq serve on port 8090 so chat completion requests route seamlessly without upstream unavailable errors.

Fixed

• Multi-GB Ready CMF Download Stream: Replaced HTTP stream decoder with a streaming process downloader (curl) for pre-converted .cmf models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
• Ready CMF Quantization & UI Labels: Set correct quantization badges (Q1, Q1T) for featured ready-to-run .cmf models and removed misleading "converting" log messages for direct downloads.

Fixed

• Direct Ready CMF Download: Importing pre-converted .cmf model repositories (e.g. infosave/Bonsai-8B_2bit_cmf) directly streams the ready .cmf file from Hugging Face instead of attempting a safetensors conversion without config.json.

Added

• New Quantization Schemes: Added support for Q4_TILED (q4t), Q1T (1.58-bit ternary), Q1S (1-bit sparse), Q1P (1-bit PTQ error diffusion), and Q1 (pure binary) in converter backend and web admin UI.
• Featured Ready-to-Run .cmf Models: Added 1-tap download cards for pre-converted .cmf models (infosave/Bonsai-1.7Bcmf, infosave/Bonsai-8B_2bit_cmf, infosave/Bonsai-27Bcmf).

[Unreleased]

Added

• Docker Compose quickstart (docker-compose.yml + config/gateway.compose.toml): gateway + a local Ollama model in one command. GHCR badge in the READMEs.

[0.2.2] - 2026-07-01

Added

• Latency benchmark vs LiteLLM and Portkey: a reproducible harness (bench/, instant mock backend + Apache Bench) and BENCHMARKS.md. Measured gateway overhead: Cortiq ~57k req/s / p99 1 ms vs Portkey ~5.8k / p99 9 ms vs LiteLLM ~1.2k / p99 59 ms.
• Accuracy benchmark: task-type routing on natural-language prompts (bench/accuracy.py, bench/tasks.jsonl) — allaigate semantic router 100% (37/37) vs a keyword heuristic 32% (competitors have no semantic task router).
• Router docs (docs/ROUTER.md + .ru.md): what the allaigate/cortiq-router does, how to point the gateway at it, API contract, pricing, and the current LAUNCH1 $1 promo.
• Docker image: multi-stage Dockerfile (non-root, ~160 MB) + docker.yml workflow publishing to ghcr.io/infosave2007/cortiq-gateway (:latest, :X.Y.Z, :edge; linux/amd64 + arm64 on release tags). docker run works out of the box.

[0.2.1] - 2026-06-30

Added

• OpenAI Models (GET /v1/models): lists the pool + the virtual cortiq-auto.
• OpenAI Completions (legacy) (POST /v1/completions): non-streaming and streaming (text_completion), routed through the same pipeline.
• Native passthrough (POST /route): returns the routing decision (task, complexity, candidate models) without calling a model.

All inbound protocol adapters are now implemented.

[0.2.0] - 2026-06-30

Added

• Streaming (SSE) end-to-end: POST /v1/chat/completions with "stream": true forwards provider SSE chunks verbatim, with X-Cortiq-* routing headers and token usage tapped from the final chunk for statistics. Failover applies before the first byte. Live streaming output in the admin Playground (/admin/api/test/stream).
• Cross-platform Windows binaries in releases (x86_64-pc-windows-msvc) and a cross-platform CI test matrix (ubuntu/windows/macos); secure token RNG via getrandom.
• Anthropic provider (outbound): POST /v1/messages with system extraction, x-api-key + anthropic-version headers, and streaming that translates Anthropic SSE events into the unified OpenAI wire format.
• Anthropic inbound adapter: POST /v1/messages (non-streaming JSON and streaming SSE), so Claude-native clients can target the gateway.
• Embeddings: POST /v1/embeddings inbound + provider embed(); embedding models are now resolvable from the pool.
• Semantic cache: embedding-based cache that returns a stored answer for prompts semantically near a previous one (cosine threshold, TTL, ring buffer), skipping the model call. Partitioned by routing signature; hit-rate and cost saved are tracked and shown on the dashboard; configurable from Settings ([cache]).
• MCP server (POST /mcp, JSON-RPC 2.0): exposes routing to MCP-native orchestrators via two tools — cortiq_route (classify → decision) and cortiq_chat (route → answer).

[0.1.0] - 2026-06-30

Added

• Embedded admin web console at /admin — a build-free single-page app compiled into the binary via rust-embed. Manage models, routing, protocols, virtual keys and secrets from the browser; changes are hot-reloaded (no restart).
• Multilingual UI in 7 languages (en, ru, de, fr, es, zh, tr) with browser auto-detection, plus light/dark themes.
• Admin API under /admin/api/* (Bearer admin token): config, models (CRUD + probe), routing, protocols, settings, keys, secrets, stats, requests, and a playground test endpoint.
• Hot config reload — config-derived state lives behind arc-swap; writes validate → atomically rewrite the TOML → swap, with no restart.
• Request statistics — in-memory aggregates, per-minute time buckets, and a ring buffer of recent requests, appended to a JSONL file and replayed on startup.
• Prometheus GET /metrics endpoint.
• Secret store (config/secrets.toml, mode 0600) — provider keys can be set from the UI without writing them into gateway.toml; values are never returned by the API.
• English documentation as the primary language, with Russian translations (README.ru.md, docs/*.ru.md).

Changed

• The Anthropic provider now returns a clear error instead of panicking when called (the adapter is still planned for v0.2).

[0.0.1]

• Initial skeleton: OpenAI Chat Completions inbound adapter, OpenAI-compatible outbound provider, routing table with fallback and graceful degradation, and cost/token accounting.