cortiq-gateway/CHANGELOG.md

# Changelog

All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).

## [0.2.37] - 2026-08-06

### Added
- **Zero-argument startup**: `cortiq-gateway` now runs with no flags. The config is created on
  first launch (repo `config/gateway.toml` when run from a checkout, otherwise
  `~/.cortiq-gateway/gateway.toml`, or `$CORTIQ_GATEWAY_HOME`), the browser opens by itself,
  and a busy port is swapped for the next free one. `--no-browser` / `--no-port-search` opt out;
  `--config` still takes an explicit path.
- **No admin token required by default**: the console opens without auth. Set `--admin-token`,
  `[admin].token` or `[admin].token_env` to protect it.
- **First-run wizard** (`/admin#/onboarding`): pick a language, download a ready-made `.cmf`
  model, and finish — Finish connects every completed download as a local model, starts its
  `cortiq serve` and points `routing.default` at it. Nothing to edit by hand.
- **Ready-made models are read live** from the `infosave` HuggingFace account in both the wizard
  and Import, with per-variant quantizations, real sizes, an "already downloaded" mark and a
  delete button. Newly published models appear without a gateway update.
- **Windows-friendly data dir**: `%APPDATA%` / `%LOCALAPPDATA%` / `%USERPROFILE%` are used before
  falling back, so running the exe from `Downloads` never writes into `Program Files`.
- **Full localization**: all 432 UI strings now exist in every language (en, ru, de, fr, es, zh, tr).
  Import, Settings → Local models, the model form and the whole first-run wizard were English-only
  in de/fr/es/zh/tr before this release.

### Fixed
- **Download progress jumped straight to 100% and then hung**: with `-L`, curl prints a complete
  `0→100%` progress bar for the redirect hop before the transfer starts, and that bar was being
  parsed as the download's progress. Progress is now derived from bytes on disk against the size
  reported by the HuggingFace tree (or a `HEAD` request), so the bar and the reported
  `done / total` bytes track the real transfer.
- **Re-downloading over an existing file reported 100% instantly**: the previous copy is now
  removed before the transfer starts, so the size poller cannot read stale bytes.
- **Downloaded models were not recognised**: the admin UI derived the output file name
  independently of the gateway, so a finished download did not mark its card as downloaded and the
  delete button never appeared. The gateway now returns the exact output stem (`local_name`) and
  the UI sends it back verbatim for download, progress and delete.
- **Registering a local model produced an unusable pool entry**: with `manage_server = false` the
  model was added to the static pool pointing at `cortiq_server_url`, where nothing listens —
  every call failed with `upstream unavailable`. Registration now always creates a managed server
  the gateway runs itself, and repairs a stale static entry for the same file.
- **Cancel did nothing during a sharded download**: cancellation was only checked between part
  files, so a multi-GB part had to finish first. The running transfer is now aborted immediately
  and the partial output removed.
- **Deleting a model left the config broken**: removing a `.cmf` now also unregisters the model
  and stops its managed server instead of leaving a pool entry pointing at a missing file.
- **Two variants of one repo collapsed into a single job row**; jobs are now keyed by repo + variant.
- Curl's stderr is captured into the job log, so a failed download explains itself instead of
  showing a bare exit code.

## [0.2.36] - 2026-07-25

### Fixed
- **Docker image failed to start when built fresh (`GLIBC_2.39 not found`)**: the floating `rust:1-slim` builder had drifted to Debian trixie while the runtime stayed on bookworm, so a locally built container crashed on startup. Builder and runtime are now pinned to the same Debian release (trixie).
- **Docker model downloads (curl exit code 23)**: the runtime image was missing `curl`, and the default models dir (`$HOME/.cortiq-gateway/models`) was not writable by the container user, so importing a ready `.cmf` model failed with a write error. The image now bundles `curl`, keeps all gateway state under `/app/data` (`CORTIQ_GATEWAY_HOME`), and runs as root so bind-mounted host folders (Docker Desktop on Windows/macOS) are writable; pass `--user 10001` to harden when not using host bind mounts.
- **Docker Compose persistence**: added a `cortiq-data` volume at `/app/data` so downloaded models and stats survive `docker compose up --force-recreate`.
- **Clear import errors**: a models dir that cannot be created now fails the import immediately with the path and reason instead of a cryptic curl exit code; curl exits 23 (write error) and 22 (HTTP error) are annotated with a human-readable hint in the job log.

## [0.2.35] - 2026-07-23

### Fixed
- **Streaming Pipeline System Prompt Merging**: Fixed system prompt injection in `run_stream` so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
- **CMF Reasoning Mode Compatibility**: Fully integrated with `cortiq-engine` v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.

## [0.2.34] - 2026-07-23

### Added
- **Responsive mobile admin console**: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
- **Accessible mobile interactions**: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.

### Fixed
- **Add model dialog**: Restored the missing `opt` helper import that prevented the Add model button from opening its form.
- **Local model settings on phones**: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.

## [0.2.31] - 2026-07-22

### Added
- **Structured O(1) Nyström Attention & Reasoning Control**: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (`o1_m`, `o1_window`, `o1_sink`), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to `think_budget = 0` for Qwen3/3.5 models), and Runtime (`skip_mtp` via `CMF_MTP=0`).
- **Persistent Admin Token Configuration**: Support for explicit `token` field in `config/gateway.toml` under `[admin]` block to prevent authorization invalidation across restarts.

### Fixed
- **JavaScript UI Helper Restoration**: Restored `opt(...)` helper in `web/js/ui.js` and `web/js/views/import.js` to ensure HF model import modal options render smoothly.
- **Engine Command Flag Accuracy**: Fixed `CMF_MTP=0` environment flag pass-through for `cortiq serve` processes.

## [0.2.30] - 2026-07-22

### Added
- **Per-Model Generation & Reasoning Parameters**: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (`o1`), MTP (Multi-Token Prediction toggle `skip_mtp`), and default System Prompt.


### Fixed
- **Duplicate Local Model Registration**: Fixed `duplicate managed model id` conflict in `register_import` by cleanly routing managed CMF models directly into `[[cmf.servers]]` with automatic free port allocation without pushing duplicate entries to `[[models]]`.


### Fixed
- **Quantization Badge & Title for Pre-Converted Models**: Automatically detect quantization (`Q1`, `Q1T`) for ready-to-run `.cmf` models (e.g. `infosave/Bonsai-27Bcmf`) from repo name and downloaded filename, resolving default `Q8_2F` badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").


### Fixed
- **Automatic CMF Local Server Lifecycle**: Registering an imported `.cmf` model automatically adds it to `[[cmf.servers]]` and reloads the runtime (`cmf_runtime::manage`), automatically spawning `cortiq serve` on port 8090 so chat completion requests route seamlessly without `upstream unavailable` errors.


### Fixed
- **Multi-GB Ready CMF Download Stream**: Replaced HTTP stream decoder with a streaming process downloader (`curl`) for pre-converted `.cmf` models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
- **Ready CMF Quantization & UI Labels**: Set correct quantization badges (`Q1`, `Q1T`) for featured ready-to-run `.cmf` models and removed misleading "converting" log messages for direct downloads.


### Fixed
- **Direct Ready CMF Download**: Importing pre-converted `.cmf` model repositories (e.g. `infosave/Bonsai-8B_2bit_cmf`) directly streams the ready `.cmf` file from Hugging Face instead of attempting a safetensors conversion without `config.json`.


### Added
- **New Quantization Schemes**: Added support for `Q4_TILED` (q4t), `Q1T` (1.58-bit ternary), `Q1S` (1-bit sparse), `Q1P` (1-bit PTQ error diffusion), and `Q1` (pure binary) in converter backend and web admin UI.
- **Featured Ready-to-Run .cmf Models**: Added 1-tap download cards for pre-converted `.cmf` models (`infosave/Bonsai-1.7Bcmf`, `infosave/Bonsai-8B_2bit_cmf`, `infosave/Bonsai-27Bcmf`).

## [Unreleased]


### Added
- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
  gateway + a local Ollama model in one command. GHCR badge in the READMEs.

## [0.2.2] - 2026-07-01

### Added
- **Latency benchmark** vs LiteLLM and Portkey: a reproducible harness (`bench/`,
  instant mock backend + Apache Bench) and `BENCHMARKS.md`. Measured gateway overhead:
  Cortiq ~57k req/s / p99 1 ms vs Portkey ~5.8k / p99 9 ms vs LiteLLM ~1.2k / p99 59 ms.
- **Accuracy benchmark**: task-type routing on natural-language prompts (`bench/accuracy.py`,
  `bench/tasks.jsonl`) — allaigate semantic router 100% (37/37) vs a keyword heuristic 32%
  (competitors have no semantic task router).
- **Router docs** (`docs/ROUTER.md` + `.ru.md`): what the allaigate/cortiq-router does, how
  to point the gateway at it, API contract, pricing, and the current `LAUNCH1` $1 promo.
- **Docker image**: multi-stage `Dockerfile` (non-root, ~160 MB) + `docker.yml` workflow
  publishing to `ghcr.io/infosave2007/cortiq-gateway` (`:latest`, `:X.Y.Z`, `:edge`;
  linux/amd64 + arm64 on release tags). `docker run` works out of the box.

## [0.2.1] - 2026-06-30

### Added
- **OpenAI Models** (`GET /v1/models`): lists the pool + the virtual `cortiq-auto`.
- **OpenAI Completions (legacy)** (`POST /v1/completions`): non-streaming and streaming
  (`text_completion`), routed through the same pipeline.
- **Native passthrough** (`POST /route`): returns the routing decision (task, complexity,
  candidate models) without calling a model.

All inbound protocol adapters are now implemented.

## [0.2.0] - 2026-06-30

### Added
- **Streaming (SSE)** end-to-end: `POST /v1/chat/completions` with `"stream": true`
  forwards provider SSE chunks verbatim, with `X-Cortiq-*` routing headers and token
  usage tapped from the final chunk for statistics. Failover applies before the first
  byte. Live streaming output in the admin Playground (`/admin/api/test/stream`).
- Cross-platform Windows binaries in releases (`x86_64-pc-windows-msvc`) and a
  cross-platform CI test matrix (ubuntu/windows/macos); secure token RNG via `getrandom`.
- **Anthropic provider** (outbound): `POST /v1/messages` with `system` extraction,
  `x-api-key` + `anthropic-version` headers, and streaming that translates Anthropic
  SSE events into the unified OpenAI wire format.
- **Anthropic inbound** adapter: `POST /v1/messages` (non-streaming JSON and streaming
  SSE), so Claude-native clients can target the gateway.
- **Embeddings**: `POST /v1/embeddings` inbound + provider `embed()`; embedding models
  are now resolvable from the pool.
- **Semantic cache**: embedding-based cache that returns a stored answer for prompts
  semantically near a previous one (cosine threshold, TTL, ring buffer), skipping the
  model call. Partitioned by routing signature; hit-rate and cost saved are tracked and
  shown on the dashboard; configurable from Settings (`[cache]`).
- **MCP server** (`POST /mcp`, JSON-RPC 2.0): exposes routing to MCP-native orchestrators
  via two tools — `cortiq_route` (classify → decision) and `cortiq_chat` (route → answer).

## [0.1.0] - 2026-06-30

### Added
- **Embedded admin web console** at `/admin` — a build-free single-page app compiled
  into the binary via `rust-embed`. Manage models, routing, protocols, virtual keys
  and secrets from the browser; changes are hot-reloaded (no restart).
- **Multilingual UI** in 7 languages (en, ru, de, fr, es, zh, tr) with browser
  auto-detection, plus light/dark themes.
- **Admin API** under `/admin/api/*` (Bearer admin token): config, models (CRUD +
  probe), routing, protocols, settings, keys, secrets, stats, requests, and a
  playground `test` endpoint.
- **Hot config reload** — config-derived state lives behind `arc-swap`; writes
  validate → atomically rewrite the TOML → swap, with no restart.
- **Request statistics** — in-memory aggregates, per-minute time buckets, and a ring
  buffer of recent requests, appended to a JSONL file and replayed on startup.
- **Prometheus `GET /metrics`** endpoint.
- **Secret store** (`config/secrets.toml`, mode `0600`) — provider keys can be set
  from the UI without writing them into `gateway.toml`; values are never returned by
  the API.
- English documentation as the primary language, with Russian translations
  (`README.ru.md`, `docs/*.ru.md`).

### Changed
- The Anthropic provider now returns a clear error instead of panicking when called
  (the adapter is still planned for v0.2).

## [0.0.1]
- Initial skeleton: OpenAI Chat Completions inbound adapter, OpenAI-compatible
  outbound provider, routing table with fallback and graceful degradation, and
  cost/token accounting.