cortiq-gateway/CHANGELOG.md

# Changelog

All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).

## [0.2.43] - 2026-08-08

### Added
- **API keys actually gate the gateway.** Configured `[[api_keys]]` now enforce auth on
  `/v1`, `/route` and `/mcp` (Bearer or `x-api-key`), attribute every request to the key's
  account in stats, apply per-key rate limits (`rate_per_min` → 429 with `Retry-After`) and
  per-key monthly budgets (`monthly_budget_usd` → 429 `insufficient_quota`). No keys
  configured = open mode, as before.
- **Auto-detected tool support for managed local servers**: `cortiq serve` ≥ 0.5.63
  advertises the loaded model's capabilities in `/healthz`; the gateway reads the handshake
  when the server becomes healthy, so a local model whose chat template can drive tool
  calling serves agent (tools) traffic with zero manual configuration. Manual override:
  `caps = ["tools"]` on the `[[cmf.servers]]` entry (also the fallback for older serves).
- **OpenRouter balance on the dashboard** (when an OpenRouter key is configured): the
  remaining credit is a KPI card, red under $5.
- **Harness snippets for every OS** on the API-keys page: Linux/macOS and Windows variants
  per harness, an install command chip when the harness is missing, and a paste-your-key
  field that substitutes the real key into every snippet.
- Deploy guide (`docs/DEPLOY.md`, Russian mirror): systemd/launchd/Windows service, nginx and
  Caddy TLS fronting, security checklist, backup list.

### Fixed
- Capability filtering widens the tier chain instead of killing the request: when an AUTO
  route's chain has no model with the required capabilities (tools), the gateway escalates
  through medium→high chains and then the rest of the pool instead of returning
  "No healthy model available". Explicit pins still fail loudly.
- Semantic-cache threshold no longer renders as `0.9200000166893005` (f32→f64), the cache
  settings card shows live entries/hits/saved-$ and warns when no embedding model exists.
- Key-page snippets survive being pasted into a real interactive zsh (no `#` comment lines,
  no history-expansion `!`), and the install chips are readable in the light theme.
- Stats log growth is bounded: `[stats] max_file_mb` (default 16) folds old JSONL records
  into the sidecar totals file — per-account all-time and month spend survive compaction
  and restarts.

### Breaking
- If your config already lists `[[api_keys]]` but your clients never send keys, requests
  will now be rejected with 401 — either remove the keys from the config or start sending
  `Authorization: Bearer <key>`.

## [0.2.42] - 2026-08-08

### Added
- **The continuous complexity score is visible**: the benchmark's routing table and the
  dashboard's recent-requests list show the router's raw 0..1 score next to the tier badge —
  see what different prompts actually weigh and tune the boundary sliders from real traffic.

### Changed
- Dashboard tier badges render translated tier names; the Russian locale calls the dashboard
  «Рабочий стол».

## [0.2.41] - 2026-08-08

### Added
- **Linked complexity-boundary sliders** on the Routing page: one track, two draggable handles
  (the low|medium and medium|high boundaries) over green/amber/red zones. Handles push each
  other, the numbers ride on the handles and the legend updates live — no hand-typed score
  ranges. The gateway re-maps the router's continuous complexity score with YOUR boundaries
  (`[routing.bands]`, validated `0 < low < medium < 1`) before tier chains, escalation and
  stats; unchecked = the router's own bands, untouched behavior.

### Fixed
- Complexity tier names are translated everywhere they appear (bands legend, tier badges, pair
  selects, benchmark column, Playground decision): лёгкий/средний/сложный, leicht/mittel/schwer,
  facile/moyen/difficile, fácil/medio/difícil, 简单/中等/困难, kolay/orta/zor.
- Benchmark route-judged retries re-ask the SAME prompt after a pause before reaching for a
  fresh template — difficulty is judged in the background, so the second occurrence is the
  honest steady-state answer.

## [0.2.40] - 2026-08-08

### Added
- **Per-key spend accounting** on the API-keys page: requests, tokens in/out and cost per key
  for a selectable window (24h / 7d / 30d) plus a cumulative all-time cost column — "which
  department spends what" at a glance.
- **Bounded stats on disk**: once `stats.jsonl` outgrows `stats.max_file_mb` (default 16, 0
  disables), records older than the retention window fold into cumulative per-account totals in
  a small sidecar file (tmp+rename on both) and leave the log. All-time accounting survives
  compaction and restarts; nothing is ever counted twice.
- **Per-complexity models inside a task rule**: each task is edited as rows of
  "model + complexity" pairs — `[routing.tasks.code] low=["cheap"] high=["best"] any=[...]`.
  The tier-specific pick leads its own tier, `any` (and legacy plain arrays) leads everywhere,
  the global tier chain and `default` always remain as fallback, and high verdicts still
  escalate over any-tier pins.

### Changed
- Benchmark hard probes are generated from wider interdisciplinary pools (6 families each) and
  retry once with a fresh template — the router's novelty decay no longer flips them red.

### Fixed
- The `task-rule` badge is honest: it appears only when the rule actually chose the head of the
  candidate chain, not whenever a rule merely exists for the task.

## [0.2.39] - 2026-08-07

### Added
- **Benchmark, redesigned into the console** (⏱ sidebar entry): two explicit modes. **Smart
  routing** (default) sends every probe through `cortiq-auto` and judges the ROUTER — task label
  vs expected (✓/✗), complexity tier, chosen model with per-request cost, task-rule badge —
  with KPIs for answer quality, classification accuracy, TTFT, tok/s and money spent, plus a
  "requests went to" per-model distribution. **Individual models** compares pinned models
  head-to-head. Results render live row-by-row, runs land in a browser-side history (per-run ×
  and Clear-all), custom tests supported, ⚡ quick subset (7) vs full suite (17).
- **Hard probes that exercise the high tier**: two route-judged tests (science / ops pools)
  generated fresh each run — interdisciplinary prompts rate high reliably while fixed wordings
  decay with the router's novelty signal. Pass = the router said `high` and the request escalated;
  the tiny token budget keeps them at cents even on the priciest model.
- **One-paste harness setup** on the API-keys page: Claude Code (native Anthropic `/v1/messages`,
  now on by default, one-click enable for older configs), Codex, OpenCode, Continue and Cline —
  each pre-filled with the gateway URL and the freshly created key. The gateway detects harnesses
  already installed on its machine (✓ on the tab, "already configured for this gateway" status).
- **Stale-tab detection**: open consoles notice a gateway restart via the meta start marker and
  show a reload banner instead of silently running an old interface.
- Log viewer Clear button (`DELETE /admin/api/logs`).

### Changed
- **High complexity escalates over task rules**: at tier `high` the tier chain leads and the task
  rule demotes to fallback — a `code → cheap-model` rule no longer pins the hardest prompts to
  the cheap model. Low/medium keep task-rule-first semantics (documented in docs/ROUTING).
- **Honest stream costs**: the final cost rides a closing SSE event (`cortiq_cost_usd`) since the
  initial header is written before generation; money renders with 4 decimals (last two smaller)
  and grows precision below $0.0001 instead of collapsing to $0.0000.
- **Oracle-sized timeouts**: router default 800 ms → 15 s (hosted escalations take ~10 s);
  provider clients gained a 10 s connect timeout (dead upstreams fail over fast) with no overall
  cap so long generations and streams are never cut.
- Benchmark fairness: +256-token reasoning headroom (thinking models aren't cut mid-`<think>`),
  router degradation shown as "⚠ router degraded" and excluded from accuracy, "budget spent on
  reasoning" instead of a misleading "empty answer", creative-writing budget 96 → 512.
- New-key dialog simplified to account / rate limit / optional custom value.

### Fixed
- `cortiq bench` results showed "error" with newer cortiq releases that pretty-print `--json`:
  the parser now finds the last balanced JSON object anywhere in the output, and tok/s prefers
  steady decode speed (the old heuristic could report `allocs_per_token` as tok/s).

## [0.2.38] - 2026-08-07

### Added
- **Image & video generation in the Playground**: three tabs now — Chat, Image, Video. Image runs
  `cortiq imagine` over a local text-to-image `.cmf` (Lumina) with width/height/steps/CFG/seed,
  live step progress and an honest ETA; the P6 PPM result is decoded to a canvas in the browser
  with one-click PNG export. Video runs `cortiq animate` (MiniMax-H3) with frames/steps/quality
  and optional first/last keyframes (rasterized to PPM client-side); the MJPEG+PCM AVI is decoded
  in JS and played on a canvas synced to the wav — no ffmpeg on either side (an mp4 remux happens
  opportunistically when ffmpeg exists). Local `.cmf` files are classified chat/image/video via
  `cortiq info`; media packs are never registered as chat servers. Requires cortiq ≥ 0.5.56.
- **Task-based routing**: `[routing.tasks]` maps a router task label to an ordered model list.
  A matching rule's models lead the candidate chain and the complexity chain follows as fallback,
  so a task pin never weakens failover. The Routing page builds the editor from the account's
  live taxonomy (`GET /v1/taxonomies`), shows each task's share of 7-day traffic, and supports
  custom labels. Responses whose chain was headed by a rule report `route_source "router+task-rule"`.
- **Generation gallery** under the media tabs (lazy PPM thumbnails, reopen with seed/steps, delete),
  and generations **queue up** (with position shown) instead of being rejected.
- **Chat is a real dialogue**: history is resent, bubbles, Clear, Cmd/Ctrl+Enter — plus live
  metrics: token counts, tok/s and time-to-first-token.
- **Exact stream token counts** when the model server provides usage; the gateway fallback counts
  content chunks (exactly 1/token on cortiq serve) instead of chars/4, flagged as an estimate.
- **Jobs survive restarts**: import and media job lists persist to `<data>/jobs-*.json` and are
  reconciled on start — the Register button no longer vanishes because the gateway restarted.
- **Admin console, ops batch**: Logs viewer in the sidebar (500-line ring, filter/level/pause/copy);
  a System panel on the dashboard (CPU sparkline + load, RAM incl. the gateway's RSS, free disk at
  the models dir); "not enough disk / may not fit in RAM" badges on model cards before a download;
  a Restart-gateway button; an update-available notice (crates.io checked twice a day); config
  export as the real `gateway.toml` and TOML import (validated, hot-reloaded); "Connect your app"
  snippets (curl/Python/JS) on the keys page; one-click `cortiq bench --json` on the Models page.
- **Docker ships the cortiq binary** (built in the builder stage, plus `libvulkan1`): local `.cmf`
  chat models and media generation now work in containers, where auto-install never could.
- **Security**: an open admin console listening on 0.0.0.0 outside Docker warns in the log and
  shows a dismissible banner.

### Fixed
- The default config pointed at a placeholder `localhost:8080` router with a nonexistent taxonomy,
  leaving every fresh install with a permanently red "Router: offline" badge. A fresh install now
  targets the hosted allaigate router and degrades gracefully until a key is set.
- The wizard's Finish lands on the Playground instead of the Models list.

## [0.2.37] - 2026-08-06

### Added
- **Zero-argument startup**: `cortiq-gateway` now runs with no flags. The config is created on
  first launch (repo `config/gateway.toml` when run from a checkout, otherwise
  `~/.cortiq-gateway/gateway.toml`, or `$CORTIQ_GATEWAY_HOME`), the browser opens by itself,
  and a busy port is swapped for the next free one. `--no-browser` / `--no-port-search` opt out;
  `--config` still takes an explicit path.
- **No admin token required by default**: the console opens without auth. Set `--admin-token`,
  `[admin].token` or `[admin].token_env` to protect it.
- **First-run wizard** (`/admin#/onboarding`): pick a language, download a ready-made `.cmf`
  model, and finish — Finish connects every completed download as a local model, starts its
  `cortiq serve` and points `routing.default` at it. Nothing to edit by hand.
- **Ready-made models are read live** from the `infosave` HuggingFace account in both the wizard
  and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a
  delete button. Newly published models appear without a gateway update.
- **Windows-friendly data dir**: `%APPDATA%` / `%LOCALAPPDATA%` / `%USERPROFILE%` are used before
  falling back, so running the exe from `Downloads` never writes into `Program Files`.
- **Full localization**: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr).
  Import, Settings → Local models, the model form and the whole first-run wizard were English-only
  in de/fr/es/zh/tr before this release.

### Fixed
- **Download progress jumped straight to 100% and then hung**: with `-L`, curl prints a complete
  `0→100%` progress bar for the redirect hop before the transfer starts, and that bar was being
  parsed as the download's progress. Progress is now derived from bytes on disk against the size
  reported by the HuggingFace tree (or a `HEAD` request), so the bar and the reported
  `done / total` bytes track the real transfer.
- **Re-downloading over an existing file reported 100% instantly**: the previous copy is now
  removed before the transfer starts, so the size poller cannot read stale bytes.
- **Downloaded models were not recognised**: the admin UI derived the output file name
  independently of the gateway, so a finished download did not mark its card as downloaded and the
  delete button never appeared. The gateway now returns the exact output stem (`local_name`) and
  the UI sends it back verbatim for download, progress and delete.
- **Registering a local model produced an unusable pool entry**: with `manage_server = false` the
  model was added to the static pool pointing at `cortiq_server_url`, where nothing listens —
  every call failed with `upstream unavailable`. Registration now always creates a managed server
  the gateway runs itself, and repairs a stale static entry for the same file.
- **Cancel did nothing during a sharded download**: cancellation was only checked between part
  files, so a multi-GB part had to finish first. The running transfer is now aborted immediately
  and the partial output removed.
- **Deleting a model left the config broken**: removing a `.cmf` now also unregisters the model
  and stops its managed server instead of leaving a pool entry pointing at a missing file.
- **Two variants of one repo collapsed into a single job row**; jobs are now keyed by repo + variant.
- Curl's stderr is captured into the job log, so a failed download explains itself instead of
  showing a bare exit code.

## [0.2.36] - 2026-07-25

### Fixed
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.

## [0.2.35] - 2026-07-23

### Fixed
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.

## [0.2.34] - 2026-07-23

### Added
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.

### Fixed
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.

## [0.2.31] - 2026-07-22

### Added
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.

### Fixed
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.

## [0.2.30] - 2026-07-22

### Added
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.


### Fixed
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.


### Fixed
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").


### Fixed
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.


### Fixed
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.


### Fixed
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.


### Added
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).

## [Unreleased]


### Added
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
  gateway + a local Ollama model in one command. GHCR badge in the READMEs.

## [0.2.2] - 2026-07-01

### Added
- **Latency benchmark** vs LiteLLM and Portkey: a reproducible harness (`bench/`,
  instant mock backend + Apache Bench) and `BENCHMARKS.md`. Measured gateway overhead:
  Cortiq ~57k req/s / p99 1 ms vs Portkey ~5.8k / p99 9 ms vs LiteLLM ~1.2k / p99 59 ms.
- **Accuracy benchmark**: task-type routing on natural-language prompts (`bench/accuracy.py`,
  `bench/tasks.jsonl`) — allaigate semantic router 100% (37/37) vs a keyword heuristic 32%
  (competitors have no semantic task router).
- **Router docs** (`docs/ROUTER.md` + `.ru.md`): what the allaigate/cortiq-router does, how
  to point the gateway at it, API contract, pricing, and the current `LAUNCH1` $1 promo.
- **Docker image**: multi-stage `Dockerfile` (non-root, ~160 MB) + `docker.yml` workflow
  publishing to `ghcr.io/infosave2007/cortiq-gateway` (`:latest`, `:X.Y.Z`, `:edge`;
  linux/amd64 + arm64 on release tags). `docker run` works out of the box.

## [0.2.1] - 2026-06-30

### Added
- **OpenAI Models** (`GET /v1/models`): lists the pool + the virtual `cortiq-auto`.
- **OpenAI Completions (legacy)** (`POST /v1/completions`): non-streaming and streaming
  (`text_completion`), routed through the same pipeline.
- **Native passthrough** (`POST /route`): returns the routing decision (task, complexity,
  candidate models) without calling a model.

All inbound protocol adapters are now implemented.

## [0.2.0] - 2026-06-30

### Added
- **Streaming (SSE)** end-to-end: `POST /v1/chat/completions` with `"stream": true`
  forwards provider SSE chunks verbatim, with `X-Cortiq-*` routing headers and token
  usage tapped from the final chunk for statistics. Failover applies before the first
  byte. Live streaming output in the admin Playground (`/admin/api/test/stream`).
- Cross-platform Windows binaries in releases (`x86_64-pc-windows-msvc`) and a
  cross-platform CI test matrix (ubuntu/windows/macos); secure token RNG via `getrandom`.
- **Anthropic provider** (outbound): `POST /v1/messages` with `system` extraction,
  `x-api-key` + `anthropic-version` headers, and streaming that translates Anthropic
  SSE events into the unified OpenAI wire format.
- **Anthropic inbound** adapter: `POST /v1/messages` (non-streaming JSON and streaming
  SSE), so Claude-native clients can target the gateway.
- **Embeddings**: `POST /v1/embeddings` inbound + provider `embed()`; embedding models
  are now resolvable from the pool.
- **Semantic cache**: embedding-based cache that returns a stored answer for prompts
  semantically near a previous one (cosine threshold, TTL, ring buffer), skipping the
  model call. Partitioned by routing signature; hit-rate and cost saved are tracked and
  shown on the dashboard; configurable from Settings (`[cache]`).
- **MCP server** (`POST /mcp`, JSON-RPC 2.0): exposes routing to MCP-native orchestrators
  via two tools — `cortiq_route` (classify → decision) and `cortiq_chat` (route → answer).

## [0.1.0] - 2026-06-30

### Added
- **Embedded admin web console** at `/admin` — a build-free single-page app compiled
  into the binary via `rust-embed`. Manage models, routing, protocols, virtual keys
  and secrets from the browser; changes are hot-reloaded (no restart).
- **Multilingual UI** in 7 languages (en, ru, de, fr, es, zh, tr) with browser
  auto-detection, plus light/dark themes.
- **Admin API** under `/admin/api/*` (Bearer admin token): config, models (CRUD +
  probe), routing, protocols, settings, keys, secrets, stats, requests, and a
  playground `test` endpoint.
- **Hot config reload** — config-derived state lives behind `arc-swap`; writes
  validate → atomically rewrite the TOML → swap, with no restart.
- **Request statistics** — in-memory aggregates, per-minute time buckets, and a ring
  buffer of recent requests, appended to a JSONL file and replayed on startup.
- **Prometheus `GET /metrics`** endpoint.
- **Secret store** (`config/secrets.toml`, mode `0600`) — provider keys can be set
  from the UI without writing them into `gateway.toml`; values are never returned by
  the API.
- English documentation as the primary language, with Russian translations
  (`README.ru.md`, `docs/*.ru.md`).

### Changed
- The Anthropic provider now returns a clear error instead of panicking when called
  (the adapter is still planned for v0.2).

## [0.0.1]
- Initial skeleton: OpenAI Chat Completions inbound adapter, OpenAI-compatible
  outbound provider, routing table with fallback and graceful degradation, and
  cost/token accounting.