cortiq-gateway/CHANGELOG.md

Changelog

All notable changes to this project are documented here. The format is based on Keep a Changelog.

[0.2.35] - 2026-07-23

Fixed

• Streaming Pipeline System Prompt Merging: Fixed system prompt injection in run_stream so configured model system prompts and reasoning suppression directives are properly passed into streaming requests.
• CMF Reasoning Mode Compatibility: Fully integrated with cortiq-engine v0.5.17 for sampler-level logit bias reasoning token suppression and zero-token thinking bypass.

[0.2.34] - 2026-07-23

Added

• Responsive mobile admin console: Added a hamburger navigation drawer, touch-friendly controls, phone-sized forms and cards, scrollable tables, compact toolbars, safe-area support, and mobile bottom-sheet dialogs across the admin interface.
• Accessible mobile interactions: Navigation and dialogs now support backdrop dismissal, Escape-key closing, reduced-motion preferences, and explicit accessibility attributes.

Fixed

• Add model dialog: Restored the missing opt helper import that prevented the Add model button from opening its form.
• Local model settings on phones: Reflowed managed CMF server controls to avoid clipped fields and actions on narrow screens.

[0.2.31] - 2026-07-22

Added

• Structured O(1) Nyström Attention & Reasoning Control: Enhanced settings drawer with structured parameter sections for O(1) Nyström Streaming Attention (o1_m, o1_window, o1_sink), Generation (Temp, Top-P, MaxTokens, System Prompt), Reasoning ("Disable reasoning" toggle mapping to think_budget = 0 for Qwen3/3.5 models), and Runtime (skip_mtp via CMF_MTP=0).
• Persistent Admin Token Configuration: Support for explicit token field in config/gateway.toml under [admin] block to prevent authorization invalidation across restarts.

Fixed

• JavaScript UI Helper Restoration: Restored opt(...) helper in web/js/ui.js and web/js/views/import.js to ensure HF model import modal options render smoothly.
• Engine Command Flag Accuracy: Fixed CMF_MTP=0 environment flag pass-through for cortiq serve processes.

[0.2.30] - 2026-07-22

Added

• Per-Model Generation & Reasoning Parameters: Full support in Web UI and Gateway core for per-model generation defaults: Temperature, Top-P, Max Tokens, Think Budget (Reasoning mode), O(1) Constant-Memory Streaming Attention (o1), MTP (Multi-Token Prediction toggle skip_mtp), and default System Prompt.

Fixed

• Duplicate Local Model Registration: Fixed duplicate managed model id conflict in register_import by cleanly routing managed CMF models directly into [[cmf.servers]] with automatic free port allocation without pushing duplicate entries to [[models]].

Fixed

• Quantization Badge & Title for Pre-Converted Models: Automatically detect quantization (Q1, Q1T) for ready-to-run .cmf models (e.g. infosave/Bonsai-27Bcmf) from repo name and downloaded filename, resolving default Q8_2F badge fallback. Updated section title to "Загрузки / Конвертации" ("Downloads & Conversions").

Fixed

• Automatic CMF Local Server Lifecycle: Registering an imported .cmf model automatically adds it to [[cmf.servers]] and reloads the runtime (cmf_runtime::manage), automatically spawning cortiq serve on port 8090 so chat completion requests route seamlessly without upstream unavailable errors.

Fixed

• Multi-GB Ready CMF Download Stream: Replaced HTTP stream decoder with a streaming process downloader (curl) for pre-converted .cmf models to support multi-gigabyte Hugging Face LFS repositories without 30s/60s socket timeouts.
• Ready CMF Quantization & UI Labels: Set correct quantization badges (Q1, Q1T) for featured ready-to-run .cmf models and removed misleading "converting" log messages for direct downloads.

Fixed

• Direct Ready CMF Download: Importing pre-converted .cmf model repositories (e.g. infosave/Bonsai-8B_2bit_cmf) directly streams the ready .cmf file from Hugging Face instead of attempting a safetensors conversion without config.json.

Added

• New Quantization Schemes: Added support for Q4_TILED (q4t), Q1T (1.58-bit ternary), Q1S (1-bit sparse), Q1P (1-bit PTQ error diffusion), and Q1 (pure binary) in converter backend and web admin UI.
• Featured Ready-to-Run .cmf Models: Added 1-tap download cards for pre-converted .cmf models (infosave/Bonsai-1.7Bcmf, infosave/Bonsai-8B_2bit_cmf, infosave/Bonsai-27Bcmf).

[Unreleased]

Added

• Docker Compose quickstart (docker-compose.yml + config/gateway.compose.toml): gateway + a local Ollama model in one command. GHCR badge in the READMEs.

[0.2.2] - 2026-07-01

Added

• Latency benchmark vs LiteLLM and Portkey: a reproducible harness (bench/, instant mock backend + Apache Bench) and BENCHMARKS.md. Measured gateway overhead: Cortiq ~57k req/s / p99 1 ms vs Portkey ~5.8k / p99 9 ms vs LiteLLM ~1.2k / p99 59 ms.
• Accuracy benchmark: task-type routing on natural-language prompts (bench/accuracy.py, bench/tasks.jsonl) — allaigate semantic router 100% (37/37) vs a keyword heuristic 32% (competitors have no semantic task router).
• Router docs (docs/ROUTER.md + .ru.md): what the allaigate/cortiq-router does, how to point the gateway at it, API contract, pricing, and the current LAUNCH1 $1 promo.
• Docker image: multi-stage Dockerfile (non-root, ~160 MB) + docker.yml workflow publishing to ghcr.io/infosave2007/cortiq-gateway (:latest, :X.Y.Z, :edge; linux/amd64 + arm64 on release tags). docker run works out of the box.

[0.2.1] - 2026-06-30

Added

• OpenAI Models (GET /v1/models): lists the pool + the virtual cortiq-auto.
• OpenAI Completions (legacy) (POST /v1/completions): non-streaming and streaming (text_completion), routed through the same pipeline.
• Native passthrough (POST /route): returns the routing decision (task, complexity, candidate models) without calling a model.

All inbound protocol adapters are now implemented.

[0.2.0] - 2026-06-30

Added

• Streaming (SSE) end-to-end: POST /v1/chat/completions with "stream": true forwards provider SSE chunks verbatim, with X-Cortiq-* routing headers and token usage tapped from the final chunk for statistics. Failover applies before the first byte. Live streaming output in the admin Playground (/admin/api/test/stream).
• Cross-platform Windows binaries in releases (x86_64-pc-windows-msvc) and a cross-platform CI test matrix (ubuntu/windows/macos); secure token RNG via getrandom.
• Anthropic provider (outbound): POST /v1/messages with system extraction, x-api-key + anthropic-version headers, and streaming that translates Anthropic SSE events into the unified OpenAI wire format.
• Anthropic inbound adapter: POST /v1/messages (non-streaming JSON and streaming SSE), so Claude-native clients can target the gateway.
• Embeddings: POST /v1/embeddings inbound + provider embed(); embedding models are now resolvable from the pool.
• Semantic cache: embedding-based cache that returns a stored answer for prompts semantically near a previous one (cosine threshold, TTL, ring buffer), skipping the model call. Partitioned by routing signature; hit-rate and cost saved are tracked and shown on the dashboard; configurable from Settings ([cache]).
• MCP server (POST /mcp, JSON-RPC 2.0): exposes routing to MCP-native orchestrators via two tools — cortiq_route (classify → decision) and cortiq_chat (route → answer).

[0.1.0] - 2026-06-30

Added

• Embedded admin web console at /admin — a build-free single-page app compiled into the binary via rust-embed. Manage models, routing, protocols, virtual keys and secrets from the browser; changes are hot-reloaded (no restart).
• Multilingual UI in 7 languages (en, ru, de, fr, es, zh, tr) with browser auto-detection, plus light/dark themes.
• Admin API under /admin/api/* (Bearer admin token): config, models (CRUD + probe), routing, protocols, settings, keys, secrets, stats, requests, and a playground test endpoint.
• Hot config reload — config-derived state lives behind arc-swap; writes validate → atomically rewrite the TOML → swap, with no restart.
• Request statistics — in-memory aggregates, per-minute time buckets, and a ring buffer of recent requests, appended to a JSONL file and replayed on startup.
• Prometheus GET /metrics endpoint.
• Secret store (config/secrets.toml, mode 0600) — provider keys can be set from the UI without writing them into gateway.toml; values are never returned by the API.
• English documentation as the primary language, with Russian translations (README.ru.md, docs/*.ru.md).

Changed

• The Anthropic provider now returns a clear error instead of panicking when called (the adapter is still planned for v0.2).

[0.0.1]

• Initial skeleton: OpenAI Chat Completions inbound adapter, OpenAI-compatible outbound provider, routing table with fallback and graceful degradation, and cost/token accounting.