chore(release): v0.2.4

feat: fresh install starts with an empty model pool
- config.rs: validate() no longer errors on zero models — an empty pool is allowed (warns instead), so the admin console comes up on a fresh install and the user adds models in /admin -> Models (or points [cmf] at a local .cmf). - gateway.example.toml: ships an EMPTY pool (no seed cmf-local/local-embed, empty routing), so `cp example -> gateway.toml && run` shows an empty list — what a new user expects.
2026-07-08 17:39:43 +03:00 · 2026-07-08 17:39:43 +03:00 · 2026-07-08 16:59:58 +03:00 · 2026-07-08 16:39:30 +03:00 · 2026-07-08 16:39:30 +03:00 · 2026-07-08 16:18:31 +03:00
53 changed files with 5216 additions and 233 deletions
@@ -0,0 +1,10 @@
+target
+.git
+.github
+docs
+bench
+config/gateway.toml
+config/secrets.toml
+config/stats.jsonl
+*.log
+.DS_Store
@@ -0,0 +1,47 @@
+name: Docker
+
+# Publishes container images to GitHub Container Registry (ghcr.io).
+#   master push  -> :edge, :sha  (amd64, fast)
+#   tag v*       -> :X.Y.Z, :X.Y, :latest, :sha  (linux/amd64 + linux/arm64)
+on:
+  push:
+    branches: [master]
+    tags: ["v*"]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  packages: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository }}
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
+            type=raw,value=edge,enable={{is_default_branch}}
+            type=sha,format=short
+      - uses: docker/build-push-action@v6
+        with:
+          context: .
+          # multi-arch only for release tags (arm64 under emulation is slow)
+          platforms: ${{ startsWith(github.ref, 'refs/tags/v') && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
@@ -0,0 +1,109 @@
+# Gateway latency benchmark
+
+How much latency does the **gateway itself** add? For an LLM proxy the model call
+dominates wall-clock time, but the gateway's own overhead is what differs between
+products — and it compounds across the many calls an agent makes. This benchmark
+isolates that overhead and compares Cortiq Gateway with **LiteLLM** and **Portkey**.
+
+> Reproduce on your own hardware: `bash bench/run.sh` (see [Methodology](#methodology)).
+
+## TL;DR
+
+Every gateway proxies the **same instant mock backend** in passthrough mode, under
+identical load (`ab -k -r -c 20 -n 5000`). Latency is in milliseconds.
+
+| Gateway | Throughput (req/s) | p50 | p90 | p99 | mean |
+|---|--:|--:|--:|--:|--:|
+| **Cortiq Gateway** (Rust) | **~57,300** | 0 ms | 1 ms | **1 ms** | 0.35 ms |
+| Portkey Gateway (Node) | ~5,800 | 3 ms | 5 ms | 9 ms | 3.5 ms |
+| LiteLLM (Python, 4 workers) | ~1,200 | 9 ms | 37 ms | 59 ms | 16.6 ms |
+| _mock-direct (no gateway)_ | _~29,000_ | _1 ms_ | _1 ms_ | _3 ms_ | _0.7 ms_ |
+
+On this machine Cortiq adds **sub-millisecond p99** overhead and sustains **~10×**
+the throughput of Portkey and **~47×** of LiteLLM. The gap is architectural: a Rust
+/ Tokio gateway uses every core in one process, with no interpreter or GC in the hot
+path.
+
+> Absolute numbers depend on hardware; the **ratios** between gateways are the point.
+> Run `bench/run.sh` to get numbers for your environment.
+
+## Accuracy: task-type routing
+
+Speed is only half the story. Cortiq's other edge is **routing by task type** — sending
+each prompt to the right model — via the allaigate semantic classifier. LiteLLM and
+Portkey have **no semantic task router**: you either pin a model, write brittle rules,
+or pay for an extra LLM classification call.
+
+On a 37-prompt, 7-task-type set of **natural-language prompts** (no explicit trigger
+words like "translate" / "summarize" — [`bench/tasks.jsonl`](bench/tasks.jsonl)):
+
+| Classifier | Correct | Accuracy |
+|---|--:|--:|
+| **allaigate semantic router** | 37 / 37 | **100%** |
+| keyword heuristic (DIY without a classifier) | 12 / 37 | 32% |
+
+A keyword matcher looks good on prompts that literally say "translate this" — but real
+users don't. On natural phrasing (`"'Guten Tag' — what does that mean in English?"`,
+`"boil this down to a few key points"`, `"which companies are named in this report?"`)
+surface matching collapses while the semantic router still routes correctly.
+
+- **Methodology.** [`bench/accuracy.py`](bench/accuracy.py) sends each prompt to the live
+  allaigate router (`taxonomy = data-assistant`) and compares the predicted `task_label`
+  to the ground truth; a keyword heuristic is scored on the same set as a lower bound for
+  "route without a classifier."
+- **Caveats.** Small, hand-labeled set; phrasing is deliberately keyword-light to test
+  semantic understanding vs surface matching. Expand `tasks.jsonl` and re-run. allaigate's
+  own published figure is **94% at separating close task types** (a harder eval than this one).
+- **Reproduce.** `CORTIQ_ROUTER_KEY=cortiq_… python3 bench/accuracy.py`
+
+## Why this matters
+
+Agentic workloads make **many** gateway calls per task (plan → sub-tasks → tool calls
+→ synthesis). Overhead that looks small per call (tens of ms) multiplies across a run
+and caps the throughput a single instance can serve. Lower, more predictable tail
+latency (p99) is the developer-facing differentiator when the model time is equal.
+
+## Methodology
+
+- **Isolate gateway overhead.** A tiny, dependency-free Node mock ([`bench/mock.js`](bench/mock.js))
+  returns a fixed OpenAI-compatible completion **instantly**. So measured latency is
+  the gateway's own cost, not model time. Every gateway proxies this same mock.
+- **Apples-to-apples.** Each gateway runs in **passthrough / pinned** mode (no routing
+  or model-selection logic) so we compare raw proxy cost. Cortiq's intelligent routing
+  (`cortiq-auto`) adds one classifier round-trip — that's a feature, measured separately,
+  not folded into this overhead number.
+- **Load.** Apache Bench, keep-alive, tolerant of length variance:
+  `ab -k -r -c 20 -n 5000 -p bench/body.json -T application/json`.
+- **Fair to each runtime.** Cortiq = one process (Tokio, multi-core). Portkey = one Node
+  process (event loop). LiteLLM = **4 uvicorn workers** (its recommended multi-process
+  setup; a single Python worker is GIL-bound to one core and scores ~860 req/s).
+- **Same box, localhost**, warm processes, results are representative of one run.
+
+### Caveats (read me)
+
+- This measures **gateway overhead only**. Real requests are dominated by model latency,
+  which is identical regardless of gateway — but overhead and tail latency are not.
+- Localhost + mock backend removes network/model variance on purpose. Your absolute
+  numbers will differ; reproduce with `bench/run.sh`.
+- `ab` reports LiteLLM "Failed requests" as a non-zero count — these are **response
+  content-length variations, not HTTP errors** (all responses are `200`, verified).
+- Tested versions: Cortiq Gateway (this repo, `--release`), LiteLLM proxy via
+  `uvicorn litellm.proxy.proxy_server:app --workers 4`, Portkey `@portkey-ai/gateway`,
+  Apache Bench. Hardware: Apple Silicon, macOS.
+
+## Reproduce
+
+```bash
+# prerequisites: node, ab (Apache Bench), a release build of the gateway
+cargo build --release
+
+# optional competitors (the harness auto-detects them):
+pip install 'litellm[proxy]' uvicorn
+npm install -g @portkey-ai/gateway
+
+# run (tune load via env)
+CONC=20 REQ=5000 bash bench/run.sh
+```
+
+The harness ([`bench/run.sh`](bench/run.sh)) starts the mock, then benchmarks the mock
+directly and each available gateway, printing a throughput / p50 / p90 / p99 table.
@@ -3,6 +3,61 @@
 All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/).

+## [Unreleased]
+
+### Added
+- **Docker Compose** quickstart (`docker-compose.yml` + `config/gateway.compose.toml`):
+  gateway + a local Ollama model in one command. GHCR badge in the READMEs.
+
+## [0.2.2] - 2026-07-01
+
+### Added
+- **Latency benchmark** vs LiteLLM and Portkey: a reproducible harness (`bench/`,
+  instant mock backend + Apache Bench) and `BENCHMARKS.md`. Measured gateway overhead:
+  Cortiq ~57k req/s / p99 1 ms vs Portkey ~5.8k / p99 9 ms vs LiteLLM ~1.2k / p99 59 ms.
+- **Accuracy benchmark**: task-type routing on natural-language prompts (`bench/accuracy.py`,
+  `bench/tasks.jsonl`) — allaigate semantic router 100% (37/37) vs a keyword heuristic 32%
+  (competitors have no semantic task router).
+- **Router docs** (`docs/ROUTER.md` + `.ru.md`): what the allaigate/cortiq-router does, how
+  to point the gateway at it, API contract, pricing, and the current `LAUNCH1` $1 promo.
+- **Docker image**: multi-stage `Dockerfile` (non-root, ~160 MB) + `docker.yml` workflow
+  publishing to `ghcr.io/infosave2007/cortiq-gateway` (`:latest`, `:X.Y.Z`, `:edge`;
+  linux/amd64 + arm64 on release tags). `docker run` works out of the box.
+
+## [0.2.1] - 2026-06-30
+
+### Added
+- **OpenAI Models** (`GET /v1/models`): lists the pool + the virtual `cortiq-auto`.
+- **OpenAI Completions (legacy)** (`POST /v1/completions`): non-streaming and streaming
+  (`text_completion`), routed through the same pipeline.
+- **Native passthrough** (`POST /route`): returns the routing decision (task, complexity,
+  candidate models) without calling a model.
+
+All inbound protocol adapters are now implemented.
+
+## [0.2.0] - 2026-06-30
+
+### Added
+- **Streaming (SSE)** end-to-end: `POST /v1/chat/completions` with `"stream": true`
+  forwards provider SSE chunks verbatim, with `X-Cortiq-*` routing headers and token
+  usage tapped from the final chunk for statistics. Failover applies before the first
+  byte. Live streaming output in the admin Playground (`/admin/api/test/stream`).
+- Cross-platform Windows binaries in releases (`x86_64-pc-windows-msvc`) and a
+  cross-platform CI test matrix (ubuntu/windows/macos); secure token RNG via `getrandom`.
+- **Anthropic provider** (outbound): `POST /v1/messages` with `system` extraction,
+  `x-api-key` + `anthropic-version` headers, and streaming that translates Anthropic
+  SSE events into the unified OpenAI wire format.
+- **Anthropic inbound** adapter: `POST /v1/messages` (non-streaming JSON and streaming
+  SSE), so Claude-native clients can target the gateway.
+- **Embeddings**: `POST /v1/embeddings` inbound + provider `embed()`; embedding models
+  are now resolvable from the pool.
+- **Semantic cache**: embedding-based cache that returns a stored answer for prompts
+  semantically near a previous one (cosine threshold, TTL, ring buffer), skipping the
+  model call. Partitioned by routing signature; hit-rate and cost saved are tracked and
+  shown on the dashboard; configurable from Settings (`[cache]`).
+- **MCP server** (`POST /mcp`, JSON-RPC 2.0): exposes routing to MCP-native orchestrators
+  via two tools — `cortiq_route` (classify → decision) and `cortiq_chat` (route → answer).
+
 ## [0.1.0] - 2026-06-30

 ### Added
@@ -76,6 +76,28 @@ dependencies = [
 "rustversion",
 ]

+[[package]]
+name = "async-stream"
+version = "0.3.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0b5a71a6f37880a80d1d7f19efd781e4b5de42c88f0722cc13bcb6cc2cfe8476"
+dependencies = [
+ "async-stream-impl",
+ "futures-core",
+ "pin-project-lite",
+]
+
+[[package]]
+name = "async-stream-impl"
+version = "0.3.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
 [[package]]
 name = "async-trait"
 version = "0.1.89"
@@ -263,10 +285,11 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"

 [[package]]
 name = "cortiq-gateway"
-version = "0.1.0"
+version = "0.2.4"
 dependencies = [
 "anyhow",
 "arc-swap",
+ "async-stream",
 "async-trait",
 "axum",
 "bytes",
@@ -1,6 +1,6 @@
 [package]
 name = "cortiq-gateway"
-version = "0.1.0"
+version = "0.2.4"
 edition = "2021"
 rust-version = "1.75"
 license = "Apache-2.0"
@@ -48,6 +48,7 @@ getrandom = "0.2"
 # streaming / async utils
 futures = "0.3"
 async-trait = "0.1"
+async-stream = "0.3"
 bytes = "1"
 tokio-stream = "0.1"

@@ -0,0 +1,25 @@
+# ---- build stage ------------------------------------------------------------
+FROM rust:1-slim AS builder
+WORKDIR /app
+# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
+COPY Cargo.toml Cargo.lock ./
+COPY src ./src
+COPY web ./web
+RUN cargo build --release
+
+# ---- runtime stage ----------------------------------------------------------
+FROM debian:bookworm-slim
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends ca-certificates \
+    && rm -rf /var/lib/apt/lists/* \
+    && useradd -r -u 10001 cortiq \
+    && mkdir -p /app/config
+COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
+# ship the example as the default config so `docker run` works out of the box
+COPY config/gateway.example.toml /app/config/gateway.toml
+RUN chown -R 10001:10001 /app
+WORKDIR /app
+USER cortiq
+EXPOSE 9000
+ENTRYPOINT ["cortiq-gateway"]
+CMD ["--config", "/app/config/gateway.toml"]
@@ -3,6 +3,7 @@
 **English** · [Русский](https://github.com/infosave2007/cortiq-gateway/blob/master/README.ru.md)

 [![Crates.io](https://img.shields.io/crates/v/cortiq-gateway.svg)](https://crates.io/crates/cortiq-gateway)
+[![Docker](https://img.shields.io/badge/ghcr.io-cortiq--gateway-2496ED?logo=docker&logoColor=white)](https://github.com/infosave2007/cortiq-gateway/pkgs/container/cortiq-gateway)
 [![CI](https://github.com/infosave2007/cortiq-gateway/actions/workflows/ci.yml/badge.svg)](https://github.com/infosave2007/cortiq-gateway/actions/workflows/ci.yml)
 [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://github.com/infosave2007/cortiq-gateway/blob/master/LICENSE)
 ![Rust](https://img.shields.io/badge/Rust-2021-orange.svg)
@@ -35,8 +36,11 @@ One OpenAI-compatible endpoint → automatic model selection from your pool

 ## ✨ Highlights

- **Drop-in OpenAI API.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`.
+- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
+- **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
+- **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
 - **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down.
+- **Semantic cache.** Optional embedding-based cache returns a stored answer for prompts semantically near a previous one — skipping the model call (hit-rate & savings on the dashboard).
 - **Built-in multilingual admin console.** Manage models, routing, protocols, keys and secrets from a web UI — **no hand-editing TOML, no restart** (changes are hot-reloaded). Available in **7 languages** (en, ru, de, fr, es, zh, tr) with light/dark themes.
 - **Live analytics.** Per-request stats (tokens, cost, latency, success rate, failovers), time-series charts, breakdowns by model/tier/task, and a Prometheus `GET /metrics` endpoint.
 - **Playground.** Send a prompt through the live pipeline and inspect the routing decision.
@@ -48,6 +52,11 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
 ## 📦 Install

 ```bash
+# Docker (GitHub Container Registry)
+docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
+  -v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
+# → admin console at http://localhost:9000/admin
+
 # from crates.io
 cargo install cortiq-gateway

@@ -57,6 +66,20 @@ cd cortiq-gateway
 cargo build --release   # ./target/release/cortiq-gateway
 ```

+The image ships a default config and runs out of the box; mount `/app/config` (with a
+`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
+
+**Docker Compose** — gateway + a local Ollama model in one command:
+
+```bash
+docker compose up -d
+docker compose exec ollama ollama pull qwen2.5:0.5b   # pull a model once
+# → OpenAI API at http://localhost:9000/v1 · admin at /admin (token: change-me)
+```
+
+See [`docker-compose.yml`](docker-compose.yml). For smart routing, set `CORTIQ_ROUTER_KEY`
+and add a `[router]` (see [docs/ROUTER.md](docs/ROUTER.md)).
+
 ---

 ## 🖥️ Admin console
@@ -128,6 +151,20 @@ print(resp.choices[0].message.content)

 ---

+## 🧭 The router (allaigate)
+
+`cortiq-auto` routing is powered by a **semantic router** — the classifier that reads a
+prompt and returns its task type + complexity. Use the hosted
+**[allaigate](https://api.allaigate.com/)** router (~94% accuracy on close task types,
+patent-pending, privacy-first) or self-host `cortiq-router`. Pinned models need no router.
+
+> 🎉 **Launch promo:** grab a **$1 Starter key** with code **`LAUNCH1`** at
+> **[api.allaigate.com](https://api.allaigate.com/)** (limited activations).
+
+Setup, pricing, API contract and how to verify accuracy: **[docs/ROUTER.md](docs/ROUTER.md)**.
+
+---
+
 ## Request flow

 1. Client → `POST /v1/chat/completions` (or Anthropic/MCP) with `model: "cortiq-auto"`.
@@ -147,20 +184,20 @@ Each adapter is toggled in the config — take only what you need.

 | Protocol | Endpoint | Status |
 |---|---|---|
-| OpenAI Chat Completions | `POST /v1/chat/completions` | ✅ implemented |
-| OpenAI Completions (legacy) | `POST /v1/completions` | planned |
-| OpenAI Embeddings | `POST /v1/embeddings` | planned |
-| OpenAI Models | `GET /v1/models` | planned |
-| Anthropic Messages | `POST /v1/messages` | planned |
-| MCP (Model Context Protocol) | JSON-RPC / SSE | planned |
-| Native passthrough | `POST /route` | planned |
+| OpenAI Chat Completions | `POST /v1/chat/completions` | ✅ implemented (+ streaming) |
+| OpenAI Embeddings | `POST /v1/embeddings` | ✅ implemented |
+| Anthropic Messages | `POST /v1/messages` | ✅ implemented (+ streaming) |
+| MCP (Model Context Protocol) | `POST /mcp` (JSON-RPC) | ✅ implemented |
+| OpenAI Completions (legacy) | `POST /v1/completions` | ✅ implemented (+ streaming) |
+| OpenAI Models | `GET /v1/models` | ✅ implemented |
+| Native passthrough | `POST /route` | ✅ implemented |

 ## Supported providers (outbound)

 | Provider | Covers |
 |---|---|
 | `openai` (OpenAI-compatible) | OpenAI, OpenRouter, Together, Groq, **vLLM, llama.cpp server, LM Studio, Ollama (`/v1`)** |
-| `anthropic` | Claude Messages API (planned) |
+| `anthropic` | Claude Messages API (+ streaming) |
 | `ollama` | native Ollama API |
 | `http` | arbitrary HTTP endpoint (custom adapter) |

@@ -217,6 +254,27 @@ Details — [docs/PROTOCOLS.md](https://github.com/infosave2007/cortiq-gateway/b

 ---

+## ⚡ Performance & accuracy
+
+**Latency** — gateway overhead proxying the same instant backend, `ab -k -r -c 20 -n 5000`:
+
+| Gateway | req/s | p50 | p99 |
+|---|--:|--:|--:|
+| **Cortiq Gateway** (Rust) | **~57,300** | 0 ms | **1 ms** |
+| Portkey (Node) | ~5,800 | 3 ms | 9 ms |
+| LiteLLM (Python, 4 workers) | ~1,200 | 9 ms | 59 ms |
+
+**Accuracy** — task-type routing on natural-language prompts (7 task types). LiteLLM/Portkey
+have no semantic task router; a keyword heuristic is the DIY stand-in:
+
+| Classifier | Accuracy |
+|---|--:|
+| **allaigate semantic router** | **100%** (37/37) |
+| keyword heuristic (no classifier) | 32% (12/37) |
+
+Methodology, caveats and a reproducible harness: **[BENCHMARKS.md](BENCHMARKS.md)**
+(`bash bench/run.sh`, `python3 bench/accuracy.py`). Numbers vary; the gaps are the point.
+
 ## Build & test

 ```bash
@@ -224,14 +282,32 @@ cargo build --release      # single self-contained binary (SPA embedded)
 cargo test                 # config round-trip, routing validation
 ```

+## ✅ Tested
+
+| Area | How it's verified | Status |
+|---|---|---|
+| Config load/save round-trip, routing validation | `cargo test` (unit) | ✅ |
+| Build · format · lint | CI: `cargo build` / `fmt --check` / `clippy -D warnings` on Linux · macOS · Windows | ✅ |
+| OpenAI Chat + SSE streaming | integration (mock upstream) | ✅ |
+| Anthropic Messages — in/out, streaming + non-streaming | integration (mock) | ✅ |
+| Embeddings (`POST /v1/embeddings`) | integration (mock) | ✅ |
+| Semantic cache — hit / miss / savings | integration (mock) | ✅ |
+| MCP — `initialize` / `tools/list` / `tools/call` | integration (mock) | ✅ |
+| Models · Completions · native `POST /route` | integration (mock) | ✅ |
+| Routing — auto / pinned / failover / degrade | integration (mock **and** live allaigate router) | ✅ |
+| Hot config reload (no restart) | integration | ✅ |
+| Latency benchmark vs LiteLLM / Portkey | `bench/run.sh` (Apache Bench) — see [BENCHMARKS.md](BENCHMARKS.md) | ✅ |
+| Task-type routing accuracy | `bench/accuracy.py` — live router 100% vs keyword heuristic 32% | ✅ |
+
 ---

 ## Status

-🚧 Implemented so far: OpenAI Chat Completions, routing with fallback & graceful
-degradation, cost/token accounting, the **embedded multilingual admin console with
-hot config reload**, statistics and Prometheus `GET /metrics`. Planned: streaming
-(SSE), Anthropic provider/inbound, embeddings, MCP. See the roadmap in
+Implemented: OpenAI Chat Completions **with SSE streaming**, **Anthropic** provider +
+inbound `/v1/messages` (streaming), **embeddings**, a **semantic cache**, **MCP** server,
+routing with fallback & graceful degradation, cost/token accounting, the **embedded
+multilingual admin console with hot config reload**, statistics and Prometheus
+`GET /metrics`. Planned: per-account routing tables, router feedback loop. See the roadmap in
 [docs/ARCHITECTURE.md](https://github.com/infosave2007/cortiq-gateway/blob/master/docs/ARCHITECTURE.md).
 Contributions welcome — see [CONTRIBUTING.md](https://github.com/infosave2007/cortiq-gateway/blob/master/CONTRIBUTING.md).

@@ -2,6 +2,8 @@

 [English](README.md) · **Русский**

+[![Crates.io](https://img.shields.io/crates/v/cortiq-gateway.svg)](https://crates.io/crates/cortiq-gateway)
+[![Docker](https://img.shields.io/badge/ghcr.io-cortiq--gateway-2496ED?logo=docker&logoColor=white)](https://github.com/infosave2007/cortiq-gateway/pkgs/container/cortiq-gateway)
 [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
 ![Rust](https://img.shields.io/badge/Rust-2021-orange.svg)
 ![Status](https://img.shields.io/badge/status-active-success.svg)
@@ -34,7 +36,8 @@

 ## ✨ Ключевое

- **Drop-in OpenAI API.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`.
+- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
+- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
 - **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
 - **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
 - **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
@@ -47,6 +50,11 @@
 ## 📦 Установка

 ```bash
+# Docker (GitHub Container Registry)
+docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
+  -v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
+# → панель на http://localhost:9000/admin
+
 # с crates.io
 cargo install cortiq-gateway

@@ -56,6 +64,21 @@ cd cortiq-gateway
 cargo build --release   # ./target/release/cortiq-gateway
 ```

+Образ идёт с дефолтным конфигом и работает из коробки; смонтируйте `/app/config`
+(с `gateway.toml`), чтобы использовать свой. Теги: `:latest`, `:X.Y.Z`, `:edge`
+(linux/amd64 + arm64).
+
+**Docker Compose** — шлюз + локальная модель Ollama одной командой:
+
+```bash
+docker compose up -d
+docker compose exec ollama ollama pull qwen2.5:0.5b   # один раз скачать модель
+# → OpenAI API на http://localhost:9000/v1 · панель на /admin (токен: change-me)
+```
+
+См. [`docker-compose.yml`](docker-compose.yml). Для умного роутинга задайте
+`CORTIQ_ROUTER_KEY` и добавьте `[router]` (см. [docs/ROUTER.ru.md](docs/ROUTER.ru.md)).
+
 ---

 ## 🖥️ Панель управления
@@ -124,6 +147,20 @@ print(resp.choices[0].message.content)

 ---

+## 🧭 Роутер (allaigate)
+
+Роутинг `cortiq-auto` обеспечивает **семантический роутер** — классификатор, который
+читает промпт и возвращает тип задачи + сложность. Используйте хостовый
+**[allaigate](https://api.allaigate.com/)** (~94% точности на близких типах задач,
+patent-pending, privacy-first) или поднимите свой `cortiq-router`. Pinned-моделям роутер не нужен.
+
+> 🎉 **Акция:** возьмите **Starter-ключ за $1** по промокоду **`LAUNCH1`** на
+> **[api.allaigate.com](https://api.allaigate.com/)** (число активаций ограничено).
+
+Настройка, тарифы, контракт API и как проверить точность: **[docs/ROUTER.ru.md](docs/ROUTER.ru.md)**.
+
+---
+
 ## Поток запроса

 1. Клиент → `POST /v1/chat/completions` (или Anthropic/MCP) с `model: "cortiq-auto"`.
@@ -141,20 +178,20 @@ print(resp.choices[0].message.content)

 | Протокол | Endpoint | Статус |
 |---|---|---|
-| OpenAI Chat Completions | `POST /v1/chat/completions` | ✅ реализовано |
-| OpenAI Completions (legacy) | `POST /v1/completions` | план |
-| OpenAI Embeddings | `POST /v1/embeddings` | план |
-| OpenAI Models | `GET /v1/models` | план |
-| Anthropic Messages | `POST /v1/messages` | план |
-| MCP (Model Context Protocol) | JSON-RPC / SSE | план |
-| Native passthrough | `POST /route` | план |
+| OpenAI Chat Completions | `POST /v1/chat/completions` | ✅ реализовано (+ стриминг) |
+| OpenAI Embeddings | `POST /v1/embeddings` | ✅ реализовано |
+| Anthropic Messages | `POST /v1/messages` | ✅ реализовано (+ стриминг) |
+| MCP (Model Context Protocol) | `POST /mcp` (JSON-RPC) | ✅ реализовано |
+| OpenAI Completions (legacy) | `POST /v1/completions` | ✅ реализовано (+ стриминг) |
+| OpenAI Models | `GET /v1/models` | ✅ реализовано |
+| Native passthrough | `POST /route` | ✅ реализовано |

 ## Поддерживаемые провайдеры (исходящие)

 | Провайдер | Покрывает |
 |---|---|
 | `openai` (OpenAI-совместимый) | OpenAI, OpenRouter, Together, Groq, **vLLM, llama.cpp server, LM Studio, Ollama (`/v1`)** |
-| `anthropic` | Claude Messages API (план) |
+| `anthropic` | Claude Messages API (+ стриминг) |
 | `ollama` | нативный Ollama API |
 | `http` | произвольный HTTP-эндпоинт (свой адаптер) |

@@ -168,6 +205,27 @@ print(resp.choices[0].message.content)

 ---

+## ⚡ Производительность и точность
+
+**Латентность** — overhead шлюза при проксировании одного мгновенного бэкенда, `ab -k -r -c 20 -n 5000`:
+
+| Шлюз | req/s | p50 | p99 |
+|---|--:|--:|--:|
+| **Cortiq Gateway** (Rust) | **~57 300** | 0 ms | **1 ms** |
+| Portkey (Node) | ~5 800 | 3 ms | 9 ms |
+| LiteLLM (Python, 4 воркера) | ~1 200 | 9 ms | 59 ms |
+
+**Точность** — определение типа задачи на естественных промптах (7 типов). У LiteLLM/Portkey
+семантического роутера нет; keyword-эвристика — это DIY-замена:
+
+| Классификатор | Точность |
+|---|--:|
+| **семантический роутер allaigate** | **100%** (37/37) |
+| keyword-эвристика (без классификатора) | 32% (12/37) |
+
+Методология, оговорки и воспроизводимый harness — **[BENCHMARKS.md](BENCHMARKS.md)**
+(`bash bench/run.sh`, `python3 bench/accuracy.py`). Числа зависят от железа; важны разрывы.
+
 ## Сборка и тесты

 ```bash
@@ -175,14 +233,33 @@ cargo build --release      # один самодостаточный бинар
 cargo test                 # round-trip конфига, валидация роутинга
 ```

+## ✅ Проверено
+
+| Область | Как проверено | Статус |
+|---|---|---|
+| Round-trip загрузки/записи конфига, валидация роутинга | `cargo test` (юнит) | ✅ |
+| Сборка · формат · линт | CI: `cargo build` / `fmt --check` / `clippy -D warnings` на Linux · macOS · Windows | ✅ |
+| OpenAI Chat + SSE-стриминг | интеграционно (mock-бэкенд) | ✅ |
+| Anthropic Messages — in/out, стриминг + не-стриминг | интеграционно (mock) | ✅ |
+| Embeddings (`POST /v1/embeddings`) | интеграционно (mock) | ✅ |
+| Семантический кэш — hit / miss / экономия | интеграционно (mock) | ✅ |
+| MCP — `initialize` / `tools/list` / `tools/call` | интеграционно (mock) | ✅ |
+| Models · Completions · нативный `POST /route` | интеграционно (mock) | ✅ |
+| Роутинг — auto / pinned / failover / деградация | интеграционно (mock **и** живой allaigate-роутер) | ✅ |
+| Горячая перезагрузка конфига (без рестарта) | интеграционно | ✅ |
+| Бенчмарк latency vs LiteLLM / Portkey | `bench/run.sh` (Apache Bench) — см. [BENCHMARKS.md](BENCHMARKS.md) | ✅ |
+| Точность определения типа задачи | `bench/accuracy.py` — живой роутер 100% vs keyword-эвристика 32% | ✅ |
+
 ---

 ## Статус

-🚧 Уже работает: OpenAI Chat Completions, роутинг с fallback и мягкой деградацией,
-учёт стоимости/токенов, **встроенная многоязычная панель управления с горячей
-перезагрузкой конфига**, статистика и Prometheus `GET /metrics`. В планах: стриминг
-(SSE), провайдер/вход Anthropic, embeddings, MCP. Дорожная карта —
+Реализовано: OpenAI Chat Completions **со стримингом (SSE)**, провайдер и вход
+**Anthropic** `/v1/messages` (стриминг), **embeddings**, **семантический кэш**,
+**MCP**-сервер, роутинг с fallback и мягкой деградацией, учёт стоимости/токенов,
+**встроенная многоязычная панель управления с горячей перезагрузкой конфига**,
+статистика и Prometheus `GET /metrics`. В планах: per-account таблицы роутинга,
+feedback-loop в роутер. Дорожная карта —
 [docs/ARCHITECTURE.ru.md](docs/ARCHITECTURE.ru.md). Контрибьюции приветствуются —
 см. [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md).

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""Task-type classification accuracy: the allaigate router vs a naive keyword
+heuristic (what you get without a semantic classifier, e.g. with LiteLLM/Portkey).
+
+    CORTIQ_ROUTER_KEY=cortiq_... python3 bench/accuracy.py
+
+Env: CORTIQ_ROUTER_KEY (required), ROUTER_URL (default https://138.226.222.209),
+     TAXONOMY (default data-assistant).
+"""
+import json, os, ssl, sys, urllib.request
+
+ROUTER_URL = os.environ.get("ROUTER_URL", "https://138.226.222.209").rstrip("/")
+TAXONOMY = os.environ.get("TAXONOMY", "data-assistant")
+KEY = os.environ.get("CORTIQ_ROUTER_KEY", "")
+CTX = ssl._create_unverified_context()
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+DATA = os.path.join(HERE, "tasks.jsonl")
+
+
+def router_predict(text):
+    body = json.dumps({"input": {"text": text}, "taxonomy_id": TAXONOMY}).encode()
+    req = urllib.request.Request(
+        ROUTER_URL + "/v1/route",
+        data=body,
+        headers={"content-type": "application/json", "Authorization": "Bearer " + KEY},
+    )
+    with urllib.request.urlopen(req, context=CTX, timeout=30) as r:
+        d = json.load(r)
+    return d.get("decision", {}).get("task_label", "?")
+
+
+def keyword_predict(text):
+    """Naive DIY routing without a semantic classifier."""
+    t = text.lower()
+    def has(*words):
+        return any(w in t for w in words)
+    if has("translate", "say ", "in spanish", "in japanese", "in german", "in italian",
+           "to english", "to german", "from french", "render the phrase"):
+        return "translation"
+    if has("summarize", "summary", "tl;dr", "condense", "shorten", "bullet point"):
+        return "summarization"
+    if has("extract", "pull out", "list all", "find every", "find all"):
+        return "extraction"
+    if has("function", "python", "javascript", "rust", "sql", "query", "implement",
+           "bug", "linked list", "code"):
+        return "code"
+    if has("solve", "derivative", "integral", "compute", "sum of", "area of",
+           "limit of", "evaluate", "equation"):
+        return "math"
+    if t.strip().endswith("?") or t.split()[:1] and t.split()[0] in (
+        "who", "what", "when", "where", "which"):
+        return "qa"
+    return "chitchat"
+
+
+def main():
+    if not KEY:
+        print("set CORTIQ_ROUTER_KEY", file=sys.stderr)
+        sys.exit(2)
+    rows = [json.loads(l) for l in open(DATA) if l.strip()]
+    r_ok = k_ok = 0
+    misses = []
+    for row in rows:
+        gt = row["label"]
+        try:
+            rp = router_predict(row["text"])
+        except Exception as e:
+            rp = "ERR:" + str(e)[:30]
+        kp = keyword_predict(row["text"])
+        r_ok += rp == gt
+        k_ok += kp == gt
+        if rp != gt:
+            misses.append((gt, rp, row["text"][:45]))
+    n = len(rows)
+    print(f"dataset: {n} prompts, {len(set(r['label'] for r in rows))} task types")
+    print(f"allaigate router : {r_ok}/{n} = {100*r_ok/n:.1f}%")
+    print(f"keyword heuristic: {k_ok}/{n} = {100*k_ok/n:.1f}%")
+    if misses:
+        print("\nrouter misses (truth -> predicted | prompt):")
+        for gt, rp, txt in misses:
+            print(f"  {gt:>13} -> {rp:<13} | {txt}")
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1 @@
+{"model":"bench","messages":[{"role":"user","content":"ping"}],"max_tokens":1}
@@ -0,0 +1,26 @@
+# Benchmark config: one model pinned to the mock upstream. Requests use
+# model="bench" (a real id) → direct passthrough, no router call — so we measure
+# pure proxy overhead, comparable to LiteLLM/Portkey passthrough.
+listen = "0.0.0.0:9100"
+
+[router]
+url = "http://127.0.0.1:8100"
+
+[[models]]
+id = "bench"
+provider = "openai"
+base_url = "http://127.0.0.1:8100/v1"
+model = "bench-model"
+
+[routing]
+default = "bench"
+
+[protocols]
+openai_chat = true
+
+[admin]
+enabled = false
+
+[stats]
+enabled = true
+file = ""
@@ -0,0 +1,40 @@
+// Fast, dependency-free mock upstream for gateway benchmarks.
+// Responds instantly with a fixed OpenAI-compatible chat completion so the
+// measured latency reflects GATEWAY overhead, not model time.
+//
+//   node bench/mock.js [port]   # default 8100
+const http = require("http");
+const PORT = parseInt(process.argv[2] || "8100", 10);
+
+const COMPLETION = JSON.stringify({
+  id: "chatcmpl-bench",
+  object: "chat.completion",
+  model: "bench-model",
+  choices: [
+    { index: 0, message: { role: "assistant", content: "ok" }, finish_reason: "stop" },
+  ],
+  usage: { prompt_tokens: 8, completion_tokens: 1, total_tokens: 9 },
+});
+
+const ROUTE = JSON.stringify({
+  request_id: "bench",
+  decision: { task_label: "general", complexity: { score: 0.2, tier: "low" } },
+});
+
+const server = http.createServer((req, res) => {
+  if (req.method === "GET") {
+    res.writeHead(200, { "content-type": "text/plain" });
+    res.end("ok");
+    return;
+  }
+  // drain the request body, then reply
+  req.on("data", () => {});
+  req.on("end", () => {
+    const body = req.url.includes("/route") ? ROUTE : COMPLETION;
+    res.writeHead(200, { "content-type": "application/json" });
+    res.end(body);
+  });
+});
+
+server.keepAliveTimeout = 60000;
+server.listen(PORT, "127.0.0.1", () => console.log(`mock upstream on :${PORT}`));
@@ -0,0 +1,103 @@
+#!/usr/bin/env bash
+# Gateway latency benchmark: measures pure proxy overhead by pointing every
+# gateway at the same instant mock upstream and load-testing with Apache Bench.
+#
+#   CONC=20 REQ=10000 bash bench/run.sh
+#
+# Compares (when installed): mock-direct (baseline), cortiq-gateway, LiteLLM, Portkey.
+set -u
+HERE="$(cd "$(dirname "$0")" && pwd)"
+ROOT="$(cd "$HERE/.." && pwd)"
+CONC="${CONC:-20}"
+REQ="${REQ:-10000}"
+BODY="$HERE/body.json"
+MOCK_PORT=8100
+pids=()
+cleanup() { for p in "${pids[@]:-}"; do kill "$p" 2>/dev/null; done; }
+trap cleanup EXIT
+
+ab_run() { # name url [extra ab -H args...]
+  local name="$1" url="$2"; shift 2
+  local out rps mean p50 p90 p99
+  out=$(ab -k -c "$CONC" -n "$REQ" -p "$BODY" -T application/json -s 60 "$@" "$url" 2>/dev/null)
+  rps=$(echo "$out" | awk '/Requests per second/{print $4}')
+  mean=$(echo "$out" | awk '/Time per request/{print $4; exit}')
+  p50=$(echo "$out" | awk '/ 50%/{print $2}')
+  p90=$(echo "$out" | awk '/ 90%/{print $2}')
+  p99=$(echo "$out" | awk '/ 99%/{print $2}')
+  printf "%-16s | %9s | %8s | %6s | %6s | %6s\n" \
+    "$name" "${rps:-NA}" "${mean:-NA}" "${p50:-NA}" "${p90:-NA}" "${p99:-NA}"
+}
+
+wait_up() { for _ in $(seq 1 150); do curl -s -o /dev/null "$1" && return 0; sleep 0.2; done; return 1; }
+
+echo "concurrency=$CONC  requests=$REQ  (latency in ms; ab -k)"
+printf "%-16s | %9s | %8s | %6s | %6s | %6s\n" "gateway" "rps" "mean" "p50" "p90" "p99"
+printf -- "-----------------+-----------+----------+--------+--------+--------\n"
+
+# --- mock baseline -----------------------------------------------------------
+node "$HERE/mock.js" $MOCK_PORT >/tmp/bench_mock.log 2>&1 & pids+=($!)
+wait_up "http://127.0.0.1:$MOCK_PORT/healthz" || { echo "mock failed"; exit 1; }
+ab_run "mock-direct" "http://127.0.0.1:$MOCK_PORT/v1/chat/completions"
+
+# --- cortiq-gateway ----------------------------------------------------------
+BIN="$ROOT/target/release/cortiq-gateway"
+[ -x "$BIN" ] || BIN="$ROOT/target/debug/cortiq-gateway"
+"$BIN" --config "$HERE/cortiq.bench.toml" >/tmp/bench_cortiq.log 2>&1 & CG=$!; pids+=($CG)
+if wait_up "http://127.0.0.1:9100/healthz"; then
+  ab_run "cortiq-gateway" "http://127.0.0.1:9100/v1/chat/completions"
+fi
+kill $CG 2>/dev/null
+
+# --- LiteLLM -----------------------------------------------------------------
+if python3 -c "import litellm" 2>/dev/null; then
+  cat > /tmp/litellm.yaml <<YAML
+model_list:
+  - model_name: bench
+    litellm_params:
+      model: openai/bench-model
+      api_base: http://127.0.0.1:$MOCK_PORT/v1
+      api_key: sk-bench
+YAML
+  # the proxy CLI is brittle across versions; run the ASGI app via uvicorn.
+  # 4 workers = LiteLLM's recommended multi-process setup (fair vs a multi-core gateway).
+  CONFIG_FILE_PATH=/tmp/litellm.yaml python3 -m uvicorn litellm.proxy.proxy_server:app \
+    --host 127.0.0.1 --port 4000 --workers 4 >/tmp/bench_litellm.log 2>&1 & LL=$!; pids+=($LL)
+  if wait_up "http://127.0.0.1:4000/health/liveliness"; then
+    # warm up the workers before measuring (cold imports per worker)
+    for _ in $(seq 1 10); do curl -s -o /dev/null -X POST -H 'content-type: application/json' -d @"$BODY" "http://127.0.0.1:4000/v1/chat/completions"; done
+    sleep 2
+    ab_run "litellm" "http://127.0.0.1:4000/v1/chat/completions"
+  fi
+  kill $LL 2>/dev/null
+else
+  echo "litellm           | (not installed — pip install 'litellm[proxy]')"
+fi
+
+# --- Portkey -----------------------------------------------------------------
+# resolve relative to the active node (npm root -g may point at a different npm)
+PK_MAIN="$(dirname "$(command -v node 2>/dev/null)")/../lib/node_modules/@portkey-ai/gateway/build/start-server.js"
+if [ -f "$PK_MAIN" ]; then
+  PORT=8787 node "$PK_MAIN" >/tmp/bench_portkey.log 2>&1 & PK=$!; pids+=($PK)
+  if wait_up "http://127.0.0.1:8787/v1/health" || wait_up "http://127.0.0.1:8787/"; then
+    sleep 1
+    ab_run "portkey" "http://127.0.0.1:8787/v1/chat/completions" \
+      -H "x-portkey-provider: openai" \
+      -H "x-portkey-custom-host: http://127.0.0.1:$MOCK_PORT/v1" \
+      -H "Authorization: Bearer sk-bench"
+  fi
+  kill $PK 2>/dev/null
+else
+  echo "portkey           | (not installed — npm i -g @portkey-ai/gateway)"
+fi
+
+# --- task-type routing accuracy (optional; needs a live router key) ----------
+if [ -n "${CORTIQ_ROUTER_KEY:-}" ]; then
+  echo
+  echo "== task-type routing accuracy (allaigate router vs keyword heuristic) =="
+  python3 "$HERE/accuracy.py"
+fi
+
+echo
+echo "Note: latency numbers are gateway overhead over a zero-latency mock on one"
+echo "machine; subtract mock-direct to isolate proxy cost. Re-run on your hardware."
@@ -0,0 +1,37 @@
+{"text": "Write a Python function to reverse a string", "label": "code"}
+{"text": "Reverse a linked list in place", "label": "code"}
+{"text": "Given an array of integers, return the indices of the two that add up to a target", "label": "code"}
+{"text": "My loop prints one extra line at the end — what's wrong with it?", "label": "code"}
+{"text": "Merge two sorted arrays into a single sorted array", "label": "code"}
+{"text": "Make this faster: it compares every element with every other element", "label": "code"}
+{"text": "Solve x^2 - 5x + 6 = 0", "label": "math"}
+{"text": "If a train covers 60 miles in 1.5 hours, how fast is it going?", "label": "math"}
+{"text": "A rectangle is twice as long as it is wide and its area is 50 — what are its sides?", "label": "math"}
+{"text": "What's 15 percent of 240?", "label": "math"}
+{"text": "Two dice are rolled — what are the odds the total is seven?", "label": "math"}
+{"text": "The sequence 2, 6, 18, 54 — what comes next?", "label": "math"}
+{"text": "Translate 'good morning' into Spanish", "label": "translation"}
+{"text": "'Guten Tag' — what does that mean in English?", "label": "translation"}
+{"text": "What's the Spanish word for butterfly?", "label": "translation"}
+{"text": "Put 'I would like a coffee, please' into French", "label": "translation"}
+{"text": "How do Italians say 'see you tomorrow'?", "label": "translation"}
+{"text": "Give me the gist of the text below in one line", "label": "summarization"}
+{"text": "What are the main takeaways from this report?", "label": "summarization"}
+{"text": "Boil this down to a few key points", "label": "summarization"}
+{"text": "In a sentence, what is this article about?", "label": "summarization"}
+{"text": "Cut this down to just the essentials", "label": "summarization"}
+{"text": "Ugh, Mondays. How was your weekend?", "label": "chitchat"}
+{"text": "I'm a little bored, tell me something interesting", "label": "chitchat"}
+{"text": "Do you think pineapple belongs on pizza?", "label": "chitchat"}
+{"text": "Feeling great today, the sun is finally out!", "label": "chitchat"}
+{"text": "Any fun plans for the holidays?", "label": "chitchat"}
+{"text": "Which one is the tallest mountain on Earth?", "label": "qa"}
+{"text": "At what temperature does water freeze in Fahrenheit?", "label": "qa"}
+{"text": "Who is the author of Pride and Prejudice?", "label": "qa"}
+{"text": "How many continents are there?", "label": "qa"}
+{"text": "What is the currency used in Japan called?", "label": "qa"}
+{"text": "From the paragraph below, grab all the phone numbers", "label": "extraction"}
+{"text": "Which companies are named in this report?", "label": "extraction"}
+{"text": "Give me just the dates and amounts from this receipt", "label": "extraction"}
+{"text": "Pick out the email addresses in this message", "label": "extraction"}
+{"text": "What are the product SKUs mentioned here?", "label": "extraction"}
@@ -0,0 +1,27 @@
+# Config used by docker-compose.yml — routes to the Ollama service in the same
+# compose network. Send model="local" (pinned) or "cortiq-auto" (falls back to the
+# default model when no router is configured). Add a [router] key to enable smart routing.
+listen = "0.0.0.0:9000"
+
+[router]
+url        = "http://localhost:8080"   # optional; pinned/default works without a router
+timeout_ms = 800
+
+[[models]]
+id       = "local"
+provider = "openai"
+base_url = "http://ollama:11434/v1"    # the Ollama service (see docker-compose.yml)
+model    = "qwen2.5:0.5b"
+cost_tier = "cheap"
+
+[routing]
+default = "local"
+
+[protocols]
+openai_chat = true
+
+[admin]
+enabled = true
+
+[telemetry]
+metrics = true
@@ -28,60 +28,29 @@ cache_ttl     = "60s"
 profile       = "balanced"    # cost-saver | balanced | quality-first

 # ============================================================================
-# Model pool: mix cheap local models and expensive cloud ones.
-# provider = "openai" covers OpenAI, OpenRouter, Together, Groq, and also
-# LOCAL servers vLLM / llama.cpp / LM Studio / Ollama (/v1).
+# Model pool — EMPTY by default. A fresh gateway starts with no models and an
+# empty pool; add them in the admin console (/admin → Models), or run a local
+# .cmf via the [cmf] section below. To add one by hand, copy this shape:
+#
+#   [[models]]
+#   id        = "my-model"
+#   provider  = "openai"                    # openai | anthropic (openai also covers
+#   base_url  = "http://127.0.0.1:8081/v1"  #   OpenRouter/Together/Groq/vLLM/Ollama)
+#   model     = "my-model"
+#   # api_key_env = "OPENAI_API_KEY"        # for hosted providers (never inline a key)
+#   cost_tier = "local"
+#   caps      = ["tools"]
 # ============================================================================

-[[models]]
-id        = "local-qwen"
-provider  = "openai"
-base_url  = "http://localhost:8000/v1"
-model     = "qwen2.5-7b-instruct"
-cost_tier = "cheap"
-price_in  = 0.0
-price_out = 0.0
-caps      = ["tools"]
-
-[[models]]
-id          = "gpt-4o-mini"
-provider    = "openai"
-base_url    = "https://api.openai.com/v1"
-model       = "gpt-4o-mini"
-api_key_env = "OPENAI_API_KEY"
-cost_tier   = "mid"
-price_in    = 0.15
-price_out   = 0.60
-caps        = ["tools", "vision"]
-
-[[models]]
-id          = "claude-opus"
-provider    = "anthropic"
-base_url    = "https://api.anthropic.com"
-model       = "claude-opus-4"
-api_key_env = "ANTHROPIC_API_KEY"
-cost_tier   = "expensive"
-price_in    = 15.0
-price_out   = 75.0
-caps        = ["tools", "vision"]
-
-# --- embedding model (for /v1/embeddings in provider mode) ------------------
-[[models]]
-id          = "local-embed"
-provider    = "openai"
-base_url    = "http://localhost:8000/v1"
-model       = "nomic-embed-text"
-cost_tier   = "cheap"
-kind        = "embedding"
-
 # ============================================================================
-# Routing table: complexity tier → ordered pool (priority = fallback chain)
+# Routing table: complexity tier → ordered pool (priority = fallback chain).
+# Leave default empty for a fresh install; set it to a model id once you add one.
 # ============================================================================
 [routing]
-low     = ["local-qwen"]
-medium  = ["gpt-4o-mini", "local-qwen"]
-high    = ["claude-opus", "gpt-4o-mini"]
-default = "gpt-4o-mini"        # used when the router is unreachable
+low     = []
+medium  = []
+high    = []
+default = ""                 # set to a model id once you add one

 [routing.policy]
 mode = "fixed_table"           # fixed_table | cost_aware
@@ -118,7 +87,7 @@ echo = true     # add a "cortiq" field to the response body (X-Cortiq-* headers
 key          = "sk-gw-team-a"
 account      = "team-a"
 rate_per_min = 600
-# allow_models = ["cortiq-auto", "local-qwen"]   # restrict available models (optional)
+# allow_models = ["cortiq-auto", "cmf-local"]   # restrict available models (optional)

 # --- idempotency and logging ------------------------------------------------
 [idempotency]
@@ -150,3 +119,33 @@ enabled   = true
 file      = "config/stats.jsonl"   # empty = in-memory only
 ring_size = 500                     # how many recent requests to keep in the feed
 retention = "7d"                    # aggregate/series depth (replayed on startup)
+
+# ============================================================================
+# Semantic cache — returns a cached answer for prompts semantically near a
+# previous one, skipping the model call. Requires an embedding model in the pool.
+# Disabled by default. (Cache settings take effect on restart.)
+# ============================================================================
+[cache]
+enabled     = false
+threshold   = 0.92                  # cosine similarity for a hit (0..1)
+ttl         = "1h"
+max_entries = 1000
+# embed_model = "local-embed"       # defaults to the first kind = "embedding" model
+
+# ============================================================================
+# Local CMF model server. The gateway can install/run a local .cmf via cortiq
+# (github.com/infosave2007/cmf) and register it as the "cmf-local" model above,
+# so it works even when the router and cloud providers are unavailable.
+# Changes to manage_server / local_model / threads / gpu apply on restart.
+# ============================================================================
+[cmf]
+manage_server = false                # true → gateway spawns `cortiq serve` at startup
+local_model   = "models/model.cmf"   # path to the .cmf to serve
+local_host    = "127.0.0.1"
+local_port    = 8081                  # must match the cmf-local base_url above
+model_id      = "cmf-local"           # id it registers under (matches [[models]])
+threads       = 8                     # decode worker threads (CMF_THREADS); ~3x vs serial
+gpu           = false                 # offload matvecs to GPU (Metal/wgpu); mainly a prefill win
+cortiq_bin    = "cortiq"             # the cortiq CLI (cargo install cortiq-cli)
+auto_install  = false                # install cortiq-cli from crates.io if the binary is missing
+local_only    = false                # true → route every request to the local model
@@ -0,0 +1,33 @@
+# Quick start: Cortiq Gateway + a local model (Ollama).
+#
+#   docker compose up -d
+#   docker compose exec ollama ollama pull qwen2.5:0.5b   # pull the model once
+#   # → OpenAI-compatible API at http://localhost:9000/v1
+#   # → admin console at   http://localhost:9000/admin  (token: change-me)
+#
+# For smart routing, add a [router] key to config/gateway.compose.toml
+# (or set CORTIQ_ROUTER_KEY and point [router] at allaigate — see docs/ROUTER.md).
+services:
+  gateway:
+    image: ghcr.io/infosave2007/cortiq-gateway:latest
+    ports:
+      - "9000:9000"
+    environment:
+      CORTIQ_ADMIN_TOKEN: change-me
+      CORTIQ_ROUTER_KEY: ${CORTIQ_ROUTER_KEY:-}
+    volumes:
+      - ./config/gateway.compose.toml:/app/config/gateway.toml:ro
+    depends_on:
+      - ollama
+    restart: unless-stopped
+
+  ollama:
+    image: ollama/ollama
+    ports:
+      - "11434:11434"
+    volumes:
+      - ollama:/root/.ollama
+    restart: unless-stopped
+
+volumes:
+  ollama:
@@ -0,0 +1,83 @@
+*Read this in [Русский](ROUTER.ru.md).*
+
+# The router (allaigate)
+
+Cortiq Gateway is a thin, fast proxy — the **intelligence** that decides *which* model
+a prompt should go to lives in a separate **router**. When you send `model: "cortiq-auto"`,
+the gateway calls the router's `POST /v1/route`, gets back a **task type** and a
+**complexity tier**, and picks a model from your pool accordingly.
+
+> Routing is optional. A pinned model (`model: "gpt-4o-mini"`) or `POST /route`-free
+> usage needs **no router** — the gateway is still a useful universal proxy. The router
+> is what turns it into a *smart* one.
+
+## Two ways to get a router
+
+1. **Hosted — [allaigate](https://api.allaigate.com/)** (fastest to start). A managed
+   semantic router: ~94% accuracy at separating close task types, patent-pending,
+   privacy-first. Get a key on the site (see the promo below).
+2. **Self-hosted — `cortiq-router`.** Run your own classifier and point the gateway at it.
+
+## 🎉 Launch promo
+
+allaigate is currently running a launch offer:
+
+> **$1 Starter key** — use promo code **`LAUNCH1`** at **https://api.allaigate.com/**
+> (limited activations).
+
+Get a key: enter your email + the promo code on the homepage; the key (`cortiq_…`) is
+minted and shown to you.
+
+### Pricing (as of this writing — check the site for current terms)
+
+| Plan | Price | Included decisions | Rate limit | Overage |
+|---|---|---|---|---|
+| **Starter** | **$1/mo** (`LAUNCH1`) | unlimited | 60/min | — |
+| Developer | $9/mo | 100k | 120/min | $0.10 / 1k |
+| Pro | $49/mo | 1M | 600/min | $0.05 / 1k |
+| Scale | $299/mo | 10M | 3000/min | $0.02 / 1k |
+
+## Point the gateway at the hosted router
+
+```toml
+[router]
+url         = "https://138.226.222.209"   # allaigate routing API
+verify_tls  = false                        # cert is bound to an IP
+taxonomy_id = "data-assistant"
+api_key_env = "CORTIQ_ROUTER_KEY"          # your cortiq_… key
+timeout_ms  = 15000                        # allow for oracle escalation on hard prompts
+```
+
+```bash
+export CORTIQ_ROUTER_KEY=cortiq_your_key
+cortiq-gateway --config config/gateway.toml
+```
+
+## API contract
+
+`POST {router}/v1/route`, `Authorization: Bearer cortiq_…`:
+
+```jsonc
+// request
+{ "input": { "text": "Write a Python function to reverse a list" }, "taxonomy_id": "data-assistant" }
+
+// response
+{
+  "decision": {
+    "task_label": "code",
+    "confidence": 0.99,
+    "complexity": { "tier": "low", "score": 0.27 }
+  }
+}
+```
+
+The gateway maps `complexity.tier` (`low`/`medium`/`high`) to your `[routing]` table and
+selects a model. On a hard/ambiguous prompt the router may escalate to an oracle LLM
+(a few seconds) for a more accurate decision — size `timeout_ms` accordingly. See
+[ROUTING.md](ROUTING.md).
+
+## Verify accuracy yourself
+
+`bench/accuracy.py` classifies a labeled prompt set through the live router — see
+[BENCHMARKS.md](../BENCHMARKS.md). In our run the router scored **100% (37/37)** on
+natural-language prompts vs a keyword heuristic's 32%.
@@ -0,0 +1,82 @@
+*Read this in [English](ROUTER.md).*
+
+# Роутер (allaigate)
+
+Cortiq Gateway — тонкий и быстрый прокси; **интеллект**, решающий *какой* модели
+отдать запрос, живёт в отдельном **роутере**. Когда вы шлёте `model: "cortiq-auto"`,
+шлюз вызывает `POST /v1/route` роутера, получает **тип задачи** и **полосу сложности**
+и выбирает модель из вашего пула.
+
+> Роутинг опционален. Pinned-модель (`model: "gpt-4o-mini"`) роутер **не требует** —
+> шлюз всё равно полезен как универсальный прокси. Роутер делает его *умным*.
+
+## Два способа получить роутер
+
+1. **Хостовый — [allaigate](https://api.allaigate.com/)** (быстрее всего начать).
+   Управляемый семантический роутер: ~94% точности на разделении близких типов задач,
+   patent-pending, privacy-first. Ключ — на сайте (см. акцию ниже).
+2. **Свой — `cortiq-router`.** Поднимите собственный классификатор и укажите его в шлюзе.
+
+## 🎉 Акция (launch)
+
+Сейчас у allaigate идёт стартовая акция:
+
+> **Starter-ключ за $1** — промокод **`LAUNCH1`** на **https://api.allaigate.com/**
+> (число активаций ограничено).
+
+Как получить ключ: введите e-mail и промокод на главной — ключ (`cortiq_…`) сгенерируется
+и покажется вам.
+
+### Тарифы (на момент написания — актуальные условия смотрите на сайте)
+
+| План | Цена | Включено решений | Rate limit | Сверх лимита |
+|---|---|---|---|---|
+| **Starter** | **$1/мес** (`LAUNCH1`) | безлимит | 60/мин | — |
+| Developer | $9/мес | 100k | 120/мин | $0.10 / 1k |
+| Pro | $49/мес | 1M | 600/мин | $0.05 / 1k |
+| Scale | $299/мес | 10M | 3000/мин | $0.02 / 1k |
+
+## Подключение шлюза к хостовому роутеру
+
+```toml
+[router]
+url         = "https://138.226.222.209"   # routing API allaigate
+verify_tls  = false                        # серт привязан к IP
+taxonomy_id = "data-assistant"
+api_key_env = "CORTIQ_ROUTER_KEY"          # ваш ключ cortiq_…
+timeout_ms  = 15000                        # запас под oracle-эскалацию на сложных запросах
+```
+
+```bash
+export CORTIQ_ROUTER_KEY=cortiq_your_key
+cortiq-gateway --config config/gateway.toml
+```
+
+## Контракт API
+
+`POST {router}/v1/route`, `Authorization: Bearer cortiq_…`:
+
+```jsonc
+// запрос
+{ "input": { "text": "Write a Python function to reverse a list" }, "taxonomy_id": "data-assistant" }
+
+// ответ
+{
+  "decision": {
+    "task_label": "code",
+    "confidence": 0.99,
+    "complexity": { "tier": "low", "score": 0.27 }
+  }
+}
+```
+
+Шлюз сопоставляет `complexity.tier` (`low`/`medium`/`high`) с таблицей `[routing]` и
+выбирает модель. На сложном/неоднозначном запросе роутер может эскалировать к oracle-LLM
+(несколько секунд) для более точного решения — подберите `timeout_ms` соответственно.
+См. [ROUTING.ru.md](ROUTING.ru.md).
+
+## Проверьте точность сами
+
+`bench/accuracy.py` прогоняет размеченный набор промптов через живой роутер — см.
+[BENCHMARKS.md](../BENCHMARKS.md). В нашем прогоне роутер дал **100% (37/37)** на
+естественных промптах против 32% у keyword-эвристики.
@@ -8,8 +8,9 @@
 pub mod assets;

 use crate::config::{
-    AdminCfg, ApiKeyCfg, BreakerCfg, Config, CortiqCfg, LogCfg, ModelCfg, ProtocolsCfg, RouteCfg,
-    RouterCfg, RoutingCfg, RoutingPolicy, StatsCfg, TelemetryCfg, TierTargets,
+    AdminCfg, ApiKeyCfg, BreakerCfg, CacheCfg, CmfCfg, Config, CortiqCfg, LogCfg, ModelCfg,
+    ProtocolsCfg, RouteCfg, RouterCfg, RoutingCfg, RoutingPolicy, StatsCfg, TelemetryCfg,
+    TierTargets,
 };
 use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
 use crate::state::SharedState;
@@ -142,6 +143,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
 pub fn api_routes(admin_token: String) -> Router<SharedState> {
    Router::new()
        .route("/admin/api/health", get(health))
+        .route("/admin/api/router/probe", post(router_probe))
        .route("/admin/api/meta", get(meta))
        .route("/admin/api/config", get(get_config).put(put_config))
        .route("/admin/api/models", get(list_models).post(create_model))
@@ -163,8 +165,15 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
            get(list_secrets).put(set_secret).delete(clear_secret),
        )
        .route("/admin/api/stats", get(get_stats))
+        .route("/admin/api/shadow", get(get_shadow))
+        .route("/admin/api/hf/search", get(hf_search))
+        .route("/admin/api/import", get(list_imports).post(start_import))
+        .route("/admin/api/import/:job", get(import_status).delete(delete_import))
+        .route("/admin/api/import/:job/cancel", post(cancel_import))
+        .route("/admin/api/import/:job/register", post(register_import))
        .route("/admin/api/requests", get(get_requests))
        .route("/admin/api/test", post(run_test))
+        .route("/admin/api/test/stream", post(run_test_stream))
        .route_layer(middleware::from_fn(move |req: Request, next: Next| {
            let expected = admin_token.clone();
            async move { auth(expected, req, next).await }
@@ -224,12 +233,14 @@ async fn probe_router(url: &str) -> (bool, u64) {
        Err(_) => return (false, 0),
    };
    let started = Instant::now();
-    let target = format!("{}/healthz", url.trim_end_matches('/'));
+    // cortiq-router exposes health at /v1/healthz (auth-exempt); a bare /healthz
+    // hits the auth middleware and 401s, which would read as a false "reachable".
+    let target = format!("{}/v1/healthz", url.trim_end_matches('/'));
    let ok = client
        .get(&target)
        .send()
        .await
-        .map(|r| r.status().as_u16() < 500)
+        .map(|r| r.status().is_success())
        .unwrap_or(false);
    (ok, started.elapsed().as_millis() as u64)
 }
@@ -237,6 +248,18 @@ async fn probe_router(url: &str) -> (bool, u64) {
 async fn health(State(state): State<SharedState>) -> ApiResult {
    let live = state.live();
    let (router_ok, router_ms) = probe_router(&live.cfg.router.url).await;
+    let key_source = match &live.cfg.router.api_key_env {
+        Some(env) => state.secrets.source(env),
+        None => "none",
+    };
+    let last = state
+        .router_status
+        .load(std::sync::atomic::Ordering::Relaxed);
+    let last_error = if last == 0 {
+        Value::Null
+    } else {
+        json!({ "kind": crate::router_client::classify_status(last), "http": last })
+    };
    let models: Vec<Value> = live
        .cfg
        .models
@@ -261,16 +284,110 @@ async fn health(State(state): State<SharedState>) -> ApiResult {
        "ok": true,
        "version": env!("CARGO_PKG_VERSION"),
        "listen": live.cfg.listen,
-        "router": { "url": live.cfg.router.url, "reachable": router_ok, "latency_ms": router_ms },
+        "router": {
+            "url": live.cfg.router.url,
+            "reachable": router_ok,
+            "latency_ms": router_ms,
+            "key_env": live.cfg.router.api_key_env,
+            "key_source": key_source,
+            "last_error": last_error,
+        },
        "models": models,
    }))
 }

+/// `POST /admin/api/router/probe` — a real (authenticated) `/v1/route` call so the
+/// panel can tell "key missing / rejected / out of quota" apart from "router down".
+/// User-triggered only: it consumes one routing decision. Also fetches
+/// `GET /v1/usage` best-effort — not part of the documented contract yet; the
+/// panel shows balance/usage automatically once the router starts serving it.
+async fn router_probe(State(state): State<SharedState>) -> ApiResult {
+    let live = state.live();
+    let rcfg = live.cfg.router.clone();
+    let key = rcfg
+        .api_key_env
+        .as_deref()
+        .and_then(|n| state.secrets.resolve(n));
+    if rcfg.api_key_env.is_some() && key.is_none() {
+        return ok(json!({ "ok": false, "status": "no_key" }));
+    }
+
+    let mut builder = reqwest::Client::builder()
+        .timeout(std::time::Duration::from_millis(rcfg.timeout_ms.max(5000)));
+    if !rcfg.verify_tls {
+        builder = builder
+            .danger_accept_invalid_certs(true)
+            .danger_accept_invalid_hostnames(true);
+    }
+    let client = builder.build().map_err(|e| ApiError::bad(e.to_string()))?;
+    let url = rcfg.url.trim_end_matches('/').to_string();
+
+    let mut body = json!({
+        "input": { "text": "ping" },
+        "options": { "policy_profile": "balanced", "allow_oracle": false }
+    });
+    if let Some(tax) = &rcfg.taxonomy_id {
+        body["taxonomy_id"] = json!(tax);
+    }
+    let mut req = client.post(format!("{url}/v1/route")).json(&body);
+    if let Some(k) = &key {
+        req = req.bearer_auth(k);
+    }
+
+    let started = Instant::now();
+    let resp = match req.send().await {
+        Ok(r) => r,
+        Err(e) => {
+            let status = if e.is_timeout() {
+                "timeout"
+            } else {
+                "unreachable"
+            };
+            return ok(json!({
+                "ok": false,
+                "status": status,
+                "latency_ms": started.elapsed().as_millis() as u64,
+            }));
+        }
+    };
+    let latency_ms = started.elapsed().as_millis() as u64;
+    let code = resp.status().as_u16();
+    if !resp.status().is_success() {
+        let message = resp
+            .json::<Value>()
+            .await
+            .ok()
+            .and_then(|v| v["error"]["message"].as_str().map(|s| s.to_string()));
+        return ok(json!({
+            "ok": false,
+            "status": crate::router_client::classify_status(code),
+            "http": code,
+            "latency_ms": latency_ms,
+            "message": message,
+        }));
+    }
+
+    let mut usage = Value::Null;
+    if let Some(k) = &key {
+        if let Ok(r) = client
+            .get(format!("{url}/v1/usage"))
+            .bearer_auth(k)
+            .send()
+            .await
+        {
+            if r.status().is_success() {
+                usage = r.json::<Value>().await.unwrap_or(Value::Null);
+            }
+        }
+    }
+    ok(json!({ "ok": true, "status": "ok", "latency_ms": latency_ms, "usage": usage }))
+}
+
 async fn meta() -> ApiResult {
    ok(json!({
        "version": env!("CARGO_PKG_VERSION"),
        "providers": ["openai", "anthropic", "ollama", "http"],
-        "provider_impl": { "openai": true, "ollama": true, "http": true, "anthropic": false },
+        "provider_impl": { "openai": true, "ollama": true, "http": true, "anthropic": true },
        "cost_tiers": ["cheap", "mid", "expensive"],
        "kinds": ["chat", "embedding"],
        "profiles": ["cost-saver", "balanced", "quality-first"],
@@ -279,8 +396,8 @@ async fn meta() -> ApiResult {
        "tiers": ["low", "medium", "high"],
        "caps": ["tools", "vision"],
        "protocols_impl": {
-            "openai_chat": true, "openai_completions": false, "openai_embeddings": false,
-            "openai_models": false, "anthropic_messages": false, "mcp": false, "native_passthrough": false
+            "openai_chat": true, "openai_completions": true, "openai_embeddings": true,
+            "openai_models": true, "anthropic_messages": true, "mcp": true, "native_passthrough": true
        },
        "languages": ["en", "ru", "de", "fr", "es", "zh", "tr"],
    }))
@@ -476,6 +593,8 @@ async fn get_settings(State(state): State<SharedState>) -> ApiResult {
        "cortiq": c.cortiq,
        "stats": c.stats,
        "admin": c.admin,
+        "cache": c.cache,
+        "cmf": c.cmf,
    }))
 }

@@ -490,6 +609,8 @@ struct SettingsBody {
    cortiq: Option<CortiqCfg>,
    stats: Option<StatsCfg>,
    admin: Option<AdminCfg>,
+    cache: Option<CacheCfg>,
+    cmf: Option<CmfCfg>,
 }

 async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBody>) -> ApiResult {
@@ -528,6 +649,24 @@ async fn put_settings(State(state): State<SharedState>, Json(b): Json<SettingsBo
        }
        cfg.admin = v;
    }
+    if let Some(v) = b.cache {
+        // the cache is built at startup; persist now, apply on restart
+        needs_restart = true;
+        cfg.cache = v;
+    }
+    if let Some(v) = b.cmf {
+        // manage_server / local_model / threads / gpu apply on restart (the server
+        // is spawned once at startup with those env vars); local_only / router
+        // routing apply immediately.
+        if v.manage_server != cfg.cmf.manage_server
+            || v.local_model != cfg.cmf.local_model
+            || v.threads != cfg.cmf.threads
+            || v.gpu != cfg.cmf.gpu
+        {
+            needs_restart = true;
+        }
+        cfg.cmf = v;
+    }
    state.reload(cfg)?;
    ok(json!({ "ok": true, "needs_restart": needs_restart }))
 }
@@ -665,7 +804,140 @@ async fn get_stats(State(state): State<SharedState>, Query(q): Query<StatsQuery>
        .and_then(parse_duration_secs)
        .unwrap_or(24 * 3600);
    let groupby = q.groupby.as_deref().unwrap_or("model");
-    ok(state.stats.snapshot(range_secs, groupby))
+    let mut snap = state.stats.snapshot(range_secs, groupby);
+    snap["cache"] = state.cache.snapshot();
+    ok(snap)
+}
+
+/// Self-warming shadow loop: per-task-type promotion state, Wilson-LB of the
+/// judged pass-rate, and whether the local model is serving it yet.
+async fn get_shadow(State(state): State<SharedState>) -> ApiResult {
+    let labels: Vec<Value> = state
+        .promotion
+        .snapshot()
+        .into_iter()
+        .map(|(label, st, n, pass_rate, lb)| {
+            json!({
+                "label": label,
+                "state": format!("{st:?}"),
+                "n": n,
+                "pass_rate": pass_rate,
+                "wilson_lb": lb,
+                "serves_local": state.promotion.serves_local(&label),
+            })
+        })
+        .collect();
+    ok(json!({ "enabled": state.promotion.enabled(), "labels": labels }))
+}
+
+// ── CMF model factory: HF search → convert → register ──
+
+#[derive(Deserialize)]
+struct HfQuery {
+    q: Option<String>,
+    limit: Option<usize>,
+}
+
+/// Proxy HuggingFace model search (server-side: avoids CORS, adds token).
+async fn hf_search(State(state): State<SharedState>, Query(q): Query<HfQuery>) -> ApiResult {
+    let live = state.live();
+    let token = if live.cfg.cmf.hf_token_env.is_empty() {
+        None
+    } else {
+        state.secrets.resolve(&live.cfg.cmf.hf_token_env)
+    };
+    let query = q.q.unwrap_or_default();
+    let limit = q.limit.unwrap_or(24).min(50);
+    let models = crate::import::hf_search(&query, limit, token.as_deref())
+        .await
+        .map_err(ApiError::bad)?;
+    ok(json!({ "models": models }))
+}
+
+async fn list_imports(State(state): State<SharedState>) -> ApiResult {
+    ok(json!({ "jobs": state.imports.list() }))
+}
+
+/// Start a conversion job (returns the job id; progress via `import/:job`).
+async fn start_import(
+    State(state): State<SharedState>,
+    Json(p): Json<crate::import::ImportParams>,
+) -> ApiResult {
+    let live = state.live();
+    let id = crate::import::start_import(state.imports.clone(), &live.cfg.cmf, p)
+        .map_err(ApiError::bad)?;
+    ok(json!({ "job": id }))
+}
+
+async fn import_status(
+    State(state): State<SharedState>,
+    Path(job): Path<String>,
+) -> ApiResult {
+    match state.imports.get(&job) {
+        Some(j) => ok(serde_json::to_value(j).unwrap_or_else(|_| json!({}))),
+        None => Err(ApiError::not_found(format!("job '{job}' not found"))),
+    }
+}
+
+/// Cancel a running conversion: kills the converter and removes partial output.
+async fn cancel_import(
+    State(state): State<SharedState>,
+    Path(job): Path<String>,
+) -> ApiResult {
+    if state.imports.get(&job).is_none() {
+        return Err(ApiError::not_found(format!("job '{job}' not found")));
+    }
+    let cancelled = state.imports.cancel(&job);
+    ok(json!({ "ok": cancelled }))
+}
+
+/// Delete a finished conversion and its converted `.cmf` file(s) from disk.
+async fn delete_import(
+    State(state): State<SharedState>,
+    Path(job): Path<String>,
+) -> ApiResult {
+    if state.imports.get(&job).is_none() {
+        return Err(ApiError::not_found(format!("job '{job}' not found")));
+    }
+    let removed = state.imports.delete(&job).map_err(ApiError::bad)?;
+    ok(json!({ "ok": removed }))
+}
+
+/// Register a finished `.cmf` as a local model (OpenAI provider → cortiq-server).
+async fn register_import(
+    State(state): State<SharedState>,
+    Path(job): Path<String>,
+) -> ApiResult {
+    let j = state
+        .imports
+        .get(&job)
+        .ok_or_else(|| ApiError::not_found(format!("job '{job}' not found")))?;
+    if j.state != "done" {
+        return Err(ApiError::bad("conversion not finished"));
+    }
+    let base = std::path::Path::new(&j.output)
+        .file_stem()
+        .map(|s| s.to_string_lossy().to_string())
+        .unwrap_or_else(|| "cmf-local".into());
+    let id = format!("cmf-{base}");
+    let mut cfg = current_cfg(&state);
+    if cfg.models.iter().any(|m| m.id == id) {
+        return Err(ApiError::bad(format!("model '{id}' already registered")));
+    }
+    cfg.models.push(crate::config::ModelCfg {
+        id: id.clone(),
+        provider: "openai".into(),
+        base_url: cfg.cmf.cortiq_server_url.clone(),
+        model: base,
+        cost_tier: "local".into(),
+        price_in: 0.0,
+        price_out: 0.0,
+        kind: "chat".into(),
+        api_key_env: None,
+        caps: Vec::new(),
+    });
+    state.reload(cfg)?;
+    ok(json!({ "ok": true, "model_id": id }))
 }

 #[derive(Deserialize)]
@@ -753,3 +1025,38 @@ async fn run_test(State(state): State<SharedState>, Json(b): Json<TestBody>) ->
        Err(e) => ok(json!({ "ok": false, "latency_ms": latency_ms, "error": e.to_string() })),
    }
 }
+
+/// Streaming playground: forwards provider SSE chunks; routing decision is exposed
+/// via X-Cortiq-* response headers (the SPA reads them after the stream ends).
+async fn run_test_stream(State(state): State<SharedState>, Json(b): Json<TestBody>) -> Response {
+    if b.messages.is_empty() {
+        return ApiError::bad("messages must not be empty").into_response();
+    }
+    let req = ChatRequest {
+        routing: parse_routing(&b.model),
+        messages: b.messages,
+        tools: vec![],
+        params: GenParams {
+            temperature: b.temperature,
+            max_tokens: b.max_tokens,
+            ..Default::default()
+        },
+        stream: true,
+        meta: RequestMeta {
+            account: "playground".into(),
+            protocol: "playground".into(),
+            ..Default::default()
+        },
+    };
+    match state.pipeline.run_stream(req, &state).await {
+        Ok((info, stream)) => {
+            let mut headers = crate::protocols::openai_chat::cortiq_headers(&info);
+            headers.insert(
+                header::CONTENT_TYPE,
+                axum::http::HeaderValue::from_static("text/event-stream"),
+            );
+            (headers, axum::body::Body::from_stream(stream)).into_response()
+        }
+        Err(e) => ApiError::bad(e.to_string()).into_response(),
+    }
+}
@@ -0,0 +1,171 @@
+//! Semantic (embedding-based) response cache.
+//!
+//! Each entry stores an L2-normalized embedding of the request plus the response.
+//! A lookup returns a cached answer when the cosine similarity to a previous request
+//! (with the same routing signature, within TTL) meets the configured threshold —
+//! skipping the expensive model call. In-memory ring buffer; counters track hit
+//! rate and the model cost saved.
+
+use crate::config::CacheCfg;
+use crate::stats::parse_duration_secs;
+use std::collections::VecDeque;
+use std::sync::atomic::{AtomicU64, Ordering};
+use std::sync::{Arc, Mutex};
+use std::time::{SystemTime, UNIX_EPOCH};
+
+fn now_secs() -> u64 {
+    SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .unwrap_or_default()
+        .as_secs()
+}
+
+struct Entry {
+    sig: String,
+    vec: Vec<f32>, // L2-normalized
+    ts: u64,
+    model_used: String,
+    content: String,
+    prompt_tokens: u32,
+    completion_tokens: u32,
+    cost: f64,
+}
+
+pub struct CacheHit {
+    pub model_used: String,
+    pub content: String,
+    pub prompt_tokens: u32,
+    pub completion_tokens: u32,
+    pub similarity: f32,
+}
+
+pub struct SemanticCache {
+    enabled: bool,
+    threshold: f32,
+    ttl_secs: u64,
+    max_entries: usize,
+    embed_model: Option<String>,
+    entries: Mutex<VecDeque<Entry>>,
+    hits: AtomicU64,
+    misses: AtomicU64,
+    saved_micro_usd: AtomicU64,
+}
+
+impl SemanticCache {
+    pub fn new(cfg: &CacheCfg) -> Arc<Self> {
+        Arc::new(Self {
+            enabled: cfg.enabled,
+            threshold: cfg.threshold,
+            ttl_secs: parse_duration_secs(&cfg.ttl).unwrap_or(3600),
+            max_entries: cfg.max_entries.max(1),
+            embed_model: cfg.embed_model.clone(),
+            entries: Mutex::new(VecDeque::new()),
+            hits: AtomicU64::new(0),
+            misses: AtomicU64::new(0),
+            saved_micro_usd: AtomicU64::new(0),
+        })
+    }
+
+    pub fn enabled(&self) -> bool {
+        self.enabled
+    }
+    pub fn embed_model(&self) -> Option<&str> {
+        self.embed_model.as_deref()
+    }
+
+    /// Look up a cached response by routing signature + query embedding.
+    pub fn lookup(&self, sig: &str, query: &[f32]) -> Option<CacheHit> {
+        if !self.enabled {
+            return None;
+        }
+        let q = normalized(query);
+        let cutoff = now_secs().saturating_sub(self.ttl_secs);
+        let inner = self.entries.lock().unwrap();
+        let mut best: Option<(f32, &Entry)> = None;
+        for e in inner.iter() {
+            if e.sig != sig || e.ts < cutoff {
+                continue;
+            }
+            let sim = dot(&e.vec, &q);
+            if sim >= self.threshold && best.map(|(b, _)| sim > b).unwrap_or(true) {
+                best = Some((sim, e));
+            }
+        }
+        match best {
+            Some((sim, e)) => {
+                self.hits.fetch_add(1, Ordering::Relaxed);
+                self.saved_micro_usd
+                    .fetch_add((e.cost * 1_000_000.0) as u64, Ordering::Relaxed);
+                Some(CacheHit {
+                    model_used: e.model_used.clone(),
+                    content: e.content.clone(),
+                    prompt_tokens: e.prompt_tokens,
+                    completion_tokens: e.completion_tokens,
+                    similarity: sim,
+                })
+            }
+            None => {
+                self.misses.fetch_add(1, Ordering::Relaxed);
+                None
+            }
+        }
+    }
+
+    /// Store a response under its routing signature + query embedding.
+    #[allow(clippy::too_many_arguments)]
+    pub fn insert(
+        &self,
+        sig: String,
+        query: Vec<f32>,
+        model_used: String,
+        content: String,
+        prompt_tokens: u32,
+        completion_tokens: u32,
+        cost: f64,
+    ) {
+        if !self.enabled || query.is_empty() {
+            return;
+        }
+        let vec = normalized(&query);
+        let mut inner = self.entries.lock().unwrap();
+        inner.push_back(Entry {
+            sig,
+            vec,
+            ts: now_secs(),
+            model_used,
+            content,
+            prompt_tokens,
+            completion_tokens,
+            cost,
+        });
+        while inner.len() > self.max_entries {
+            inner.pop_front();
+        }
+    }
+
+    pub fn snapshot(&self) -> serde_json::Value {
+        let h = self.hits.load(Ordering::Relaxed);
+        let m = self.misses.load(Ordering::Relaxed);
+        serde_json::json!({
+            "enabled": self.enabled,
+            "hits": h,
+            "misses": m,
+            "hit_rate": if h + m > 0 { h as f64 / (h + m) as f64 } else { 0.0 },
+            "entries": self.entries.lock().unwrap().len(),
+            "saved_usd": self.saved_micro_usd.load(Ordering::Relaxed) as f64 / 1_000_000.0,
+        })
+    }
+}
+
+fn dot(a: &[f32], b: &[f32]) -> f32 {
+    a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
+}
+
+fn normalized(v: &[f32]) -> Vec<f32> {
+    let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
+    if n > 0.0 {
+        v.iter().map(|x| x / n).collect()
+    } else {
+        v.to_vec()
+    }
+}
@@ -0,0 +1,279 @@
+//! Local CMF model server lifecycle.
+//!
+//! Closes the loop with <https://github.com/infosave2007/cmf>: install the
+//! `cortiq` CLI from crates.io, keep it up to date, and run `cortiq serve` as a
+//! managed child process. The served model exposes an OpenAI-compatible API, so
+//! the registry registers it as an ordinary provider and the gateway routes to
+//! it like any other backend.
+
+use crate::config::CmfCfg;
+use std::process::Stdio;
+use std::sync::{Arc, Mutex};
+use std::time::Duration;
+use tokio::process::{Child, Command};
+
+/// Observable state of the managed local CMF server (surfaced to the admin API).
+#[derive(Clone, Debug, Default, serde::Serialize)]
+pub struct CmfStatus {
+    /// `manage_server` is on and a `local_model` is configured.
+    pub enabled: bool,
+    /// Version reported by `cortiq --version` (None = not installed).
+    pub installed_version: Option<String>,
+    /// Newest `cortiq-cli` on crates.io (populated when `auto_update` runs).
+    pub latest_version: Option<String>,
+    /// The child `cortiq serve` process has been spawned.
+    pub running: bool,
+    /// `/healthz` on the local server answered 200.
+    pub healthy: bool,
+    /// OpenAI base URL the local server is registered under.
+    pub base_url: String,
+    /// Path of the `.cmf` being served.
+    pub model: String,
+    /// Last fatal error, if any.
+    pub last_error: Option<String>,
+    /// Recent lifecycle log lines (bounded).
+    pub log: Vec<String>,
+}
+
+/// Holds the managed child process and its live status.
+pub struct CmfRuntime {
+    status: Mutex<CmfStatus>,
+    child: Mutex<Option<Child>>,
+}
+
+impl CmfRuntime {
+    pub fn new() -> Arc<Self> {
+        Arc::new(Self {
+            status: Mutex::new(CmfStatus::default()),
+            child: Mutex::new(None),
+        })
+    }
+
+    pub fn status(&self) -> CmfStatus {
+        self.status.lock().unwrap().clone()
+    }
+
+    fn log(&self, msg: impl Into<String>) {
+        let msg = msg.into();
+        tracing::info!(target: "cmf", "{msg}");
+        let mut s = self.status.lock().unwrap();
+        s.log.push(msg);
+        let n = s.log.len();
+        if n > 50 {
+            s.log.drain(0..n - 50);
+        }
+    }
+
+    fn fail(&self, msg: impl Into<String>) {
+        let msg = msg.into();
+        tracing::warn!(target: "cmf", "{msg}");
+        let mut s = self.status.lock().unwrap();
+        s.last_error = Some(msg.clone());
+        s.log.push(format!("✗ {msg}"));
+    }
+
+    fn set_child(&self, child: Child) {
+        *self.child.lock().unwrap() = Some(child);
+        self.status.lock().unwrap().running = true;
+    }
+
+    /// Stop the managed server (used on shutdown / reconfigure).
+    pub async fn stop(&self) {
+        if let Some(mut child) = self.child.lock().unwrap().take() {
+            let _ = child.start_kill();
+        }
+        let mut s = self.status.lock().unwrap();
+        s.running = false;
+        s.healthy = false;
+    }
+}
+
+impl Default for CmfRuntime {
+    fn default() -> Self {
+        Self {
+            status: Mutex::new(CmfStatus::default()),
+            child: Mutex::new(None),
+        }
+    }
+}
+
+/// crates.io `newest_version` for `cortiq-cli`.
+async fn latest_crates_version() -> Option<String> {
+    let client = reqwest::Client::builder()
+        .timeout(Duration::from_secs(10))
+        .build()
+        .ok()?;
+    let body = client
+        .get("https://crates.io/api/v1/crates/cortiq-cli")
+        .header("User-Agent", "cortiq-gateway")
+        .send()
+        .await
+        .ok()?
+        .text()
+        .await
+        .ok()?;
+    let v: serde_json::Value = serde_json::from_str(&body).ok()?;
+    v["crate"]["newest_version"]
+        .as_str()
+        .map(|s| s.to_string())
+}
+
+/// `<bin> --version` → the trailing version token (e.g. "cortiq 0.1.2" → "0.1.2").
+fn installed_version(bin: &str) -> Option<String> {
+    let out = std::process::Command::new(bin).arg("--version").output().ok()?;
+    if !out.status.success() {
+        return None;
+    }
+    let s = String::from_utf8_lossy(&out.stdout);
+    s.split_whitespace().last().map(|s| s.trim().to_string())
+}
+
+fn parse_ver(s: &str) -> (u32, u32, u32) {
+    let mut it = s.split('.').map(|x| x.trim().parse::<u32>().unwrap_or(0));
+    (it.next().unwrap_or(0), it.next().unwrap_or(0), it.next().unwrap_or(0))
+}
+
+fn version_lt(a: &str, b: &str) -> bool {
+    parse_ver(a) < parse_ver(b)
+}
+
+/// `cargo install cortiq-cli [--force]` — blocking compile, run off-thread.
+async fn cargo_install(force: bool) -> Result<(), String> {
+    let mut cmd = Command::new("cargo");
+    cmd.args(["install", "cortiq-cli", "--locked"]);
+    if force {
+        cmd.arg("--force");
+    }
+    let out = cmd.output().await.map_err(|e| e.to_string())?;
+    if out.status.success() {
+        Ok(())
+    } else {
+        Err(String::from_utf8_lossy(&out.stderr).trim().to_string())
+    }
+}
+
+/// Spawn `cortiq serve <model> --host <host> --port <port>`; the child is killed
+/// when its handle is dropped (i.e. when the gateway shuts down).
+fn spawn_serve(cfg: &CmfCfg) -> Result<Child, String> {
+    let mut cmd = Command::new(&cfg.cortiq_bin);
+    cmd.arg("serve")
+        .arg(&cfg.local_model)
+        .arg("--host")
+        .arg(&cfg.local_host)
+        .arg("--port")
+        .arg(cfg.local_port.to_string());
+    // Parallelise decode matvecs across cores (CMF_THREADS). 0 = leave the
+    // runtime's own default; N>1 gave ~2.8x on a 15B CPU model.
+    if cfg.threads > 1 {
+        cmd.env("CMF_THREADS", cfg.threads.to_string());
+    }
+    // Offload large matvecs to the GPU (Metal on macOS). Memory-bandwidth-bound
+    // on unified memory, so mostly a prefill win; opt-in.
+    if cfg.gpu {
+        cmd.env("CMF_GPU", "1");
+    }
+    cmd.stdout(Stdio::null())
+        .stderr(Stdio::null())
+        .kill_on_drop(true)
+        .spawn()
+        .map_err(|e| e.to_string())
+}
+
+/// Poll `/healthz` until the local server answers 200 (or the deadline passes).
+async fn wait_healthy(host: &str, port: u16, tries: u32) -> bool {
+    let url = format!("http://{host}:{port}/healthz");
+    let client = reqwest::Client::new();
+    for _ in 0..tries {
+        if let Ok(r) = client
+            .get(&url)
+            .timeout(Duration::from_secs(2))
+            .send()
+            .await
+        {
+            if r.status().is_success() {
+                return true;
+            }
+        }
+        tokio::time::sleep(Duration::from_millis(500)).await;
+    }
+    false
+}
+
+/// Orchestrate the whole lifecycle: ensure installed → maybe update → spawn →
+/// wait healthy. Safe to call in a background task; it never panics.
+pub async fn manage(rt: Arc<CmfRuntime>, cfg: CmfCfg) {
+    if !cfg.manage_server || cfg.local_model.trim().is_empty() {
+        return;
+    }
+    {
+        let mut s = rt.status.lock().unwrap();
+        s.enabled = true;
+        s.base_url = format!("http://{}:{}/v1", cfg.local_host, cfg.local_port);
+        s.model = cfg.local_model.clone();
+    }
+    if !std::path::Path::new(&cfg.local_model).exists() {
+        rt.fail(format!("local_model not found: {}", cfg.local_model));
+        return;
+    }
+
+    // 1. Ensure the `cortiq` binary is available.
+    let mut ver = installed_version(&cfg.cortiq_bin);
+    if ver.is_none() {
+        if cfg.auto_install {
+            rt.log("cortiq not found — installing cortiq-cli from crates.io (this compiles, may take a few minutes)…");
+            if let Err(e) = cargo_install(false).await {
+                rt.fail(format!("cargo install cortiq-cli failed: {e}"));
+                return;
+            }
+            ver = installed_version(&cfg.cortiq_bin);
+            if ver.is_some() {
+                rt.log("installed cortiq-cli");
+            }
+        } else {
+            rt.fail("cortiq binary not found (set [cmf].auto_install = true, or install cortiq-cli)");
+            return;
+        }
+    }
+    rt.status.lock().unwrap().installed_version = ver.clone();
+
+    // 2. Optionally update to the newest crates.io release.
+    if cfg.auto_update {
+        if let Some(latest) = latest_crates_version().await {
+            rt.status.lock().unwrap().latest_version = Some(latest.clone());
+            if ver.as_deref().map(|c| version_lt(c, &latest)).unwrap_or(false) {
+                rt.log(format!("updating cortiq-cli {} → {latest}…", ver.as_deref().unwrap_or("?")));
+                if let Err(e) = cargo_install(true).await {
+                    rt.log(format!("update failed (keeping current): {e}"));
+                } else {
+                    ver = installed_version(&cfg.cortiq_bin);
+                    rt.status.lock().unwrap().installed_version = ver.clone();
+                    rt.log(format!("updated to {}", ver.as_deref().unwrap_or("?")));
+                }
+            }
+        }
+    }
+
+    // 3. Spawn the local server.
+    rt.log(format!(
+        "starting: cortiq serve {} --host {} --port {}",
+        cfg.local_model, cfg.local_host, cfg.local_port
+    ));
+    match spawn_serve(&cfg) {
+        Ok(child) => rt.set_child(child),
+        Err(e) => {
+            rt.fail(format!("failed to spawn cortiq serve: {e}"));
+            return;
+        }
+    }
+
+    // 4. Wait until it is serving (model load can take a while for big models).
+    if wait_healthy(&cfg.local_host, cfg.local_port, 240).await {
+        rt.status.lock().unwrap().healthy = true;
+        rt.log(format!(
+            "local CMF server ready — registered as model '{}' at http://{}:{}/v1",
+            cfg.model_id, cfg.local_host, cfg.local_port
+        ));
+    } else {
+        rt.fail("local CMF server did not become healthy in time");
+    }
+}
@@ -3,6 +3,26 @@

 use serde::{Deserialize, Serialize};
 use std::collections::HashMap;
+use std::path::PathBuf;
+
+/// Base directory for gateway-managed data (models, logs). Honors
+/// `$CORTIQ_GATEWAY_HOME`, then `$XDG_DATA_HOME/cortiq-gateway`, then
+/// `~/.cortiq-gateway` — so defaults work regardless of the process CWD
+/// (important once the gateway is installed rather than run from its repo).
+pub fn data_dir() -> PathBuf {
+    for (var, sub) in [("CORTIQ_GATEWAY_HOME", ""), ("XDG_DATA_HOME", "cortiq-gateway"), ("HOME", ".cortiq-gateway")] {
+        if let Ok(v) = std::env::var(var) {
+            if !v.is_empty() {
+                return if sub.is_empty() { PathBuf::from(v) } else { PathBuf::from(v).join(sub) };
+            }
+        }
+    }
+    PathBuf::from("data")
+}
+
+fn data_path(name: &str) -> String {
+    data_dir().join(name).to_string_lossy().into_owned()
+}

 #[derive(Clone, Debug, Serialize, Deserialize)]
 pub struct Config {
@@ -31,10 +51,133 @@ pub struct Config {
    pub admin: AdminCfg,
    #[serde(default)]
    pub stats: StatsCfg,
+    #[serde(default)]
+    pub cache: CacheCfg,
+    #[serde(default)]
+    pub shadow: ShadowCfg,
+    #[serde(default)]
+    pub cmf: CmfCfg,
+}
+
+/// CMF-format model factory: import a HuggingFace model, convert it to a
+/// local `.cmf` (quantized), and serve it via cortiq-server. Points the
+/// gateway at the Python converter + runtime on the host.
+#[derive(Clone, Debug, Serialize, Deserialize)]
+pub struct CmfCfg {
+    /// Python interpreter that has torch + the converter deps.
+    #[serde(default = "default_cmf_python")]
+    pub python_bin: String,
+    /// Path to `convert_dtgma_to_cmf.py`.
+    #[serde(default = "default_cmf_converter")]
+    pub converter: String,
+    /// Directory where produced `.cmf` files are written.
+    #[serde(default = "default_cmf_models_dir")]
+    pub models_dir: String,
+    /// Base URL of the cortiq-server that serves the `.cmf` (OpenAI /v1),
+    /// used when registering a converted model as a provider.
+    #[serde(default = "default_cmf_server_url")]
+    pub cortiq_server_url: String,
+    /// Secret name of a HuggingFace token (higher search rate limits +
+    /// gated/private repos). Empty = anonymous public search.
+    #[serde(default)]
+    pub hf_token_env: String,
+    /// `cortiq` CLI binary for the local server (from `cargo install cortiq-cli`).
+    #[serde(default = "default_cortiq_bin")]
+    pub cortiq_bin: String,
+    /// Install cortiq-cli from crates.io automatically if the binary is missing.
+    #[serde(default)]
+    pub auto_install: bool,
+    /// On startup, check crates.io and reinstall if a newer cortiq-cli exists.
+    #[serde(default)]
+    pub auto_update: bool,
+    /// Spawn and manage a local `cortiq serve` for `local_model`, and register
+    /// it in the model pool — a local `.cmf` backend the gateway can use even
+    /// when no external provider (or the router) is available.
+    #[serde(default)]
+    pub manage_server: bool,
+    /// Path to the local `.cmf` to serve (empty = disabled).
+    #[serde(default)]
+    pub local_model: String,
+    /// Host the managed local server binds to.
+    #[serde(default = "default_cmf_local_host")]
+    pub local_host: String,
+    /// Port the managed local server binds to.
+    #[serde(default = "default_cmf_local_port")]
+    pub local_port: u16,
+    /// Model id the local server is registered under in the pool.
+    #[serde(default = "default_cmf_model_id")]
+    pub model_id: String,
+    /// Use ONLY local models: route every request to the local model, ignoring
+    /// the router and any external providers ("use only local models").
+    #[serde(default)]
+    pub local_only: bool,
+    /// Worker threads for the managed `cortiq serve` (exported as `CMF_THREADS`).
+    /// 0 = let the runtime decide; N>1 parallelises decode matvecs across N cores
+    /// (measured ~2.8x faster than serial on a 15B model). Default 8.
+    #[serde(default = "default_cmf_threads")]
+    pub threads: usize,
+    /// Offload large matvecs to the GPU (exports CMF_GPU=1). On Apple Silicon the
+    /// decode loop is memory-bandwidth bound, so this is ~neutral for generation
+    /// (helps prefill of long prompts). Off by default.
+    #[serde(default)]
+    pub gpu: bool,
+}
+impl Default for CmfCfg {
+    fn default() -> Self {
+        Self {
+            python_bin: default_cmf_python(),
+            converter: default_cmf_converter(),
+            models_dir: default_cmf_models_dir(),
+            cortiq_server_url: default_cmf_server_url(),
+            hf_token_env: String::new(),
+            cortiq_bin: default_cortiq_bin(),
+            auto_install: false,
+            auto_update: false,
+            manage_server: false,
+            local_model: String::new(),
+            local_host: default_cmf_local_host(),
+            local_port: default_cmf_local_port(),
+            model_id: default_cmf_model_id(),
+            local_only: false,
+            threads: default_cmf_threads(),
+            gpu: false,
+        }
+    }
+}
+fn default_cmf_threads() -> usize {
+    8
+}
+fn default_cortiq_bin() -> String {
+    "cortiq".into()
+}
+fn default_cmf_local_host() -> String {
+    "127.0.0.1".into()
+}
+fn default_cmf_local_port() -> u16 {
+    8081
+}
+fn default_cmf_model_id() -> String {
+    "cmf-local".into()
+}
+fn default_cmf_python() -> String {
+    "python3".into()
+}
+fn default_cmf_converter() -> String {
+    "../cmf/converter/convert_dtgma_to_cmf.py".into()
+}
+fn default_cmf_models_dir() -> String {
+    data_path("models")
+}
+fn default_cmf_server_url() -> String {
+    "http://127.0.0.1:8081/v1".into()
 }

 #[derive(Clone, Debug, Serialize, Deserialize)]
 pub struct RouterCfg {
+    /// Master switch — when false the router is never called; the gateway
+    /// serves from the local model pool (local `.cmf` / configured default).
+    #[serde(default = "default_router_enabled")]
+    pub enabled: bool,
    pub url: String,
    #[serde(default = "default_router_timeout")]
    pub timeout_ms: u64,
@@ -45,6 +188,9 @@ pub struct RouterCfg {
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub taxonomy_id: Option<String>,
 }
+fn default_router_enabled() -> bool {
+    true
+}
 fn default_router_timeout() -> u64 {
    800
 }
@@ -311,7 +457,7 @@ fn default_stats_enabled() -> bool {
    true
 }
 fn default_stats_file() -> String {
-    "config/stats.jsonl".into()
+    data_path("stats.jsonl")
 }
 fn default_stats_ring() -> usize {
    500
@@ -320,6 +466,122 @@ fn default_stats_retention() -> String {
    "7d".into()
 }

+/// Self-warming shadow loop (docs/SELF_WARMING_GATEWAY.md). While a task
+/// label is not yet promoted, the client is served the CLOUD answer; in
+/// parallel (sampled, non-blocking) the local `cmf-local` model answers and
+/// a cheap `judge` model scores it against the cloud answer. The per-label
+/// Wilson-LB of that pass stream gates promotion to local serving.
+#[derive(Clone, Debug, Serialize, Deserialize)]
+pub struct ShadowCfg {
+    /// Master switch. Off = zero overhead, classic gateway behavior.
+    #[serde(default)]
+    pub enabled: bool,
+    /// model_id (from `[[models]]`) of the local CMF model (OpenAI-compatible
+    /// cortiq-server). Empty disables the loop.
+    #[serde(default)]
+    pub local_model_id: String,
+    /// model_id of the cheap judge (different family than the answerer).
+    #[serde(default)]
+    pub judge_model_id: String,
+    /// Fraction of eligible requests to shadow+judge (0..1). Bounds cost.
+    #[serde(default = "default_shadow_sample")]
+    pub sample_rate: f64,
+    /// judge.jsonl append+replay path. Empty = in-memory only.
+    #[serde(default = "default_shadow_file")]
+    pub file: String,
+    /// Wilson-95 lower bound of pass-rate required to promote.
+    #[serde(default = "default_shadow_lb")]
+    pub promote_lb: f64,
+    /// Min judgments before promotion is considered.
+    #[serde(default = "default_shadow_nmin")]
+    pub n_min: usize,
+    /// Rolling window of judgments per label.
+    #[serde(default = "default_shadow_window")]
+    pub window: usize,
+    /// Extra canary judgments before full local serving.
+    #[serde(default = "default_shadow_soak")]
+    pub soak: usize,
+    /// GATE past shadow-only: actually SERVE promoted labels from the local
+    /// model (with failover to cloud on error + a complexity veto). Default
+    /// OFF — the memo's 3 risks (§8) should be retired by measurement first.
+    #[serde(default)]
+    pub serve_when_promoted: bool,
+}
+impl Default for ShadowCfg {
+    fn default() -> Self {
+        Self {
+            enabled: false,
+            local_model_id: String::new(),
+            judge_model_id: String::new(),
+            sample_rate: default_shadow_sample(),
+            file: default_shadow_file(),
+            promote_lb: default_shadow_lb(),
+            n_min: default_shadow_nmin(),
+            window: default_shadow_window(),
+            soak: default_shadow_soak(),
+            serve_when_promoted: false,
+        }
+    }
+}
+fn default_shadow_sample() -> f64 {
+    0.15
+}
+fn default_shadow_file() -> String {
+    data_path("judge.jsonl")
+}
+fn default_shadow_lb() -> f64 {
+    0.95
+}
+fn default_shadow_nmin() -> usize {
+    200
+}
+fn default_shadow_window() -> usize {
+    500
+}
+fn default_shadow_soak() -> usize {
+    100
+}
+
+/// Semantic (embedding-based) response cache — returns a cached answer for prompts
+/// that are semantically near a previous one, skipping the (expensive) model call.
+#[derive(Clone, Debug, Serialize, Deserialize)]
+pub struct CacheCfg {
+    #[serde(default)]
+    pub enabled: bool,
+    /// Cosine-similarity threshold for a hit (0..1).
+    #[serde(default = "default_semcache_threshold")]
+    pub threshold: f32,
+    /// Entry time-to-live, e.g. `1h`.
+    #[serde(default = "default_semcache_ttl")]
+    pub ttl: String,
+    /// Maximum number of cached entries (ring buffer).
+    #[serde(default = "default_semcache_max")]
+    pub max_entries: usize,
+    /// Embedding model id (defaults to the first `kind = "embedding"` model).
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub embed_model: Option<String>,
+}
+impl Default for CacheCfg {
+    fn default() -> Self {
+        Self {
+            enabled: false,
+            threshold: default_semcache_threshold(),
+            ttl: default_semcache_ttl(),
+            max_entries: default_semcache_max(),
+            embed_model: None,
+        }
+    }
+}
+fn default_semcache_threshold() -> f32 {
+    0.92
+}
+fn default_semcache_ttl() -> String {
+    "1h".into()
+}
+fn default_semcache_max() -> usize {
+    1000
+}
+
 impl Config {
    /// Load and validate the config from a file.
    pub fn load(path: &str) -> anyhow::Result<Self> {
@@ -332,8 +594,17 @@ impl Config {

    /// Integrity validation: non-empty model pool, valid routing targets.
    pub fn validate(&self) -> anyhow::Result<()> {
-        if self.models.is_empty() {
-            anyhow::bail!("config: at least one [[models]] entry is required");
+        // A managed local CMF model counts as an available model even though it
+        // is registered dynamically rather than via a [[models]] entry — this
+        // is what lets the gateway run on local models alone (no cloud, no router).
+        let has_local = self.cmf.manage_server && !self.cmf.local_model.trim().is_empty();
+        if self.models.is_empty() && !has_local {
+            // Fresh install: start with an EMPTY model pool so the admin console
+            // comes up — the user adds models in /admin → Models (or points [cmf]
+            // at a local .cmf). Routing simply has nothing to select until then.
+            tracing::warn!(
+                "no models configured — starting with an empty pool; add one in /admin → Models"
+            );
        }
        // uniqueness of model ids
        let mut seen = std::collections::HashSet::new();
@@ -343,9 +614,12 @@ impl Config {
            }
        }
        // every target in [routing] must exist in the model pool
-        let ids: std::collections::HashSet<&str> =
+        let mut ids: std::collections::HashSet<&str> =
            self.models.iter().map(|m| m.id.as_str()).collect();
-        if !ids.contains(self.routing.default.as_str()) {
+        if has_local {
+            ids.insert(self.cmf.model_id.as_str());
+        }
+        if !self.routing.default.is_empty() && !ids.contains(self.routing.default.as_str()) {
            anyhow::bail!(
                "config: routing.default '{}' is not a known model id",
                self.routing.default
@@ -0,0 +1,390 @@
+//! CMF model factory: search HuggingFace, then convert a chosen repo to a
+//! local quantized `.cmf` via the Python converter — as a tracked, streamed
+//! background job so the admin UI can show live progress.
+
+use crate::config::CmfCfg;
+use serde::{Deserialize, Serialize};
+use std::collections::HashMap;
+use std::process::Stdio;
+use std::sync::{Arc, Mutex};
+use std::time::{SystemTime, UNIX_EPOCH};
+use tokio::io::{AsyncBufReadExt, BufReader};
+use tokio::sync::Notify;
+
+/// Parse a `@PROGRESS <0..1> <phase>` marker emitted by the converter.
+fn parse_progress(line: &str) -> Option<(f32, String)> {
+    let rest = line.strip_prefix("@PROGRESS ")?;
+    let mut it = rest.splitn(2, ' ');
+    let frac: f32 = it.next()?.trim().parse().ok()?;
+    let phase = it.next().unwrap_or("").trim().to_string();
+    Some((frac.clamp(0.0, 1.0), phase))
+}
+
+/// Remove a partial/aborted output — `.cmf` may be a file or a tensor-dir.
+fn cleanup_output(path: &str) {
+    let p = std::path::Path::new(path);
+    if p.is_dir() {
+        let _ = std::fs::remove_dir_all(p);
+    } else if p.exists() {
+        let _ = std::fs::remove_file(p);
+    }
+}
+
+fn now() -> u64 {
+    SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs()
+}
+
+/// Job id from a hash of (repo, nanos) — unique without a uuid dependency.
+fn gen_id(repo: &str) -> String {
+    use std::hash::{Hash, Hasher};
+    let nanos = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_nanos();
+    let mut h = std::collections::hash_map::DefaultHasher::new();
+    repo.hash(&mut h);
+    nanos.hash(&mut h);
+    format!("{:016x}", h.finish())
+}
+
+/// Parameters chosen in the import wizard.
+#[derive(Clone, Debug, Deserialize)]
+pub struct ImportParams {
+    pub repo: String,
+    #[serde(default = "default_quant")]
+    pub quant: String,
+    #[serde(default)]
+    pub name: String, // output basename; empty → derived from repo
+    // ── advanced ──
+    #[serde(default)]
+    pub linear_core: Option<String>, // gated_delta_net | vmf_phase
+    #[serde(default)]
+    pub nphase: Option<u32>,
+    #[serde(default)]
+    pub vbit_shape: Option<String>, // log2 | cubic
+    #[serde(default)]
+    pub mean_bits: Option<f32>,
+    #[serde(default)]
+    pub shard_max_gb: Option<f32>,
+    #[serde(default)]
+    pub skip_mtp: bool,
+}
+fn default_quant() -> String {
+    "Q8_2F".into()
+}
+
+/// Live state of one conversion job.
+#[derive(Clone, Debug, Serialize)]
+pub struct Job {
+    pub id: String,
+    pub repo: String,
+    pub quant: String,
+    pub output: String,
+    pub state: String, // running | done | error | cancelled
+    pub log: Vec<String>,
+    pub started: u64,
+    pub finished: Option<u64>,
+    pub size_bytes: Option<u64>,
+    pub progress: Option<f32>,  // 0..1, parsed from converter @PROGRESS
+    pub phase: Option<String>,  // current phase label
+}
+
+#[derive(Default)]
+pub struct JobStore {
+    jobs: Mutex<HashMap<String, Job>>,
+    /// Cancel handles, keyed by job id (not serialized). Notifying aborts the
+    /// converter process and cleans up its partial output.
+    cancels: Mutex<HashMap<String, Arc<Notify>>>,
+}
+
+impl JobStore {
+    pub fn new() -> Arc<Self> {
+        Arc::new(Self::default())
+    }
+    fn insert(&self, job: Job) {
+        self.jobs.lock().unwrap().insert(job.id.clone(), job);
+    }
+    fn set_cancel(&self, id: &str, n: Arc<Notify>) {
+        self.cancels.lock().unwrap().insert(id.to_string(), n);
+    }
+    fn drop_cancel(&self, id: &str) {
+        self.cancels.lock().unwrap().remove(id);
+    }
+    /// Signal a running job to abort. Returns false if the job isn't cancellable.
+    pub fn cancel(&self, id: &str) -> bool {
+        let running = matches!(
+            self.jobs.lock().unwrap().get(id).map(|j| j.state.as_str()),
+            Some("running")
+        );
+        if !running {
+            return false;
+        }
+        if let Some(n) = self.cancels.lock().unwrap().get(id) {
+            n.notify_one();
+            true
+        } else {
+            false
+        }
+    }
+    fn push_line(&self, id: &str, line: String) {
+        if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
+            j.log.push(line);
+            let n = j.log.len();
+            if n > 200 {
+                j.log.drain(0..n - 200); // keep the tail
+            }
+        }
+    }
+    fn set_progress(&self, id: &str, frac: f32, phase: String) {
+        if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
+            j.progress = Some(frac);
+            if !phase.is_empty() {
+                j.phase = Some(phase);
+            }
+        }
+    }
+    fn set_state(&self, id: &str, state: &str) {
+        if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
+            j.state = state.into();
+            j.finished = Some(now());
+        }
+    }
+    fn finish(&self, id: &str, ok: bool) {
+        if let Some(j) = self.jobs.lock().unwrap().get_mut(id) {
+            if j.state == "cancelled" {
+                return; // a concurrent cancel already settled this job
+            }
+            j.finished = Some(now());
+            j.size_bytes = std::fs::metadata(&j.output).ok().map(|m| m.len());
+            let done = ok && j.size_bytes.unwrap_or(0) > 0;
+            j.state = if done { "done" } else { "error" }.into();
+            if done {
+                j.progress = Some(1.0);
+            }
+        }
+    }
+    pub fn get(&self, id: &str) -> Option<Job> {
+        self.jobs.lock().unwrap().get(id).cloned()
+    }
+    /// Delete a finished job together with its converted output file(s).
+    /// Refuses a running job (cancel it first). `Ok(false)` if unknown.
+    pub fn delete(&self, id: &str) -> Result<bool, String> {
+        let job = self.jobs.lock().unwrap().get(id).cloned();
+        let Some(job) = job else { return Ok(false) };
+        if job.state == "running" {
+            return Err("cancel the running job before deleting".into());
+        }
+        cleanup_output(&job.output);
+        self.cancels.lock().unwrap().remove(id);
+        self.jobs.lock().unwrap().remove(id);
+        Ok(true)
+    }
+    pub fn list(&self) -> Vec<Job> {
+        let mut v: Vec<Job> = self.jobs.lock().unwrap().values().cloned().collect();
+        v.sort_by(|a, b| b.started.cmp(&a.started));
+        v.truncate(20);
+        v
+    }
+}
+
+/// Proxy HuggingFace model search (avoids browser CORS + adds our token).
+pub async fn hf_search(query: &str, limit: usize, token: Option<&str>) -> Result<serde_json::Value, String> {
+    let q: String = url_escape(query);
+    let sort = if query.trim().is_empty() { "trendingScore" } else { "downloads" };
+    let url = format!(
+        "https://huggingface.co/api/models?search={q}&sort={sort}&direction=-1&limit={limit}&full=false"
+    );
+    let client = reqwest::Client::builder()
+        .timeout(std::time::Duration::from_secs(15))
+        .build()
+        .map_err(|e| e.to_string())?;
+    let mut req = client.get(&url).header("User-Agent", "cortiq-gateway");
+    if let Some(t) = token {
+        req = req.bearer_auth(t);
+    }
+    let resp = req.send().await.map_err(|e| e.to_string())?;
+    if !resp.status().is_success() {
+        return Err(format!("HuggingFace API {}", resp.status()));
+    }
+    resp.json().await.map_err(|e| e.to_string())
+}
+
+fn url_escape(s: &str) -> String {
+    s.chars()
+        .map(|c| match c {
+            'a'..='z' | 'A'..='Z' | '0'..='9' | '-' | '_' | '.' | '/' => c.to_string(),
+            ' ' => "+".to_string(),
+            _ => format!("%{:02X}", c as u32),
+        })
+        .collect()
+}
+
+fn sanitize(s: &str) -> String {
+    s.chars()
+        .map(|c| if c.is_ascii_alphanumeric() || c == '-' || c == '_' { c } else { '-' })
+        .collect()
+}
+
+/// Kick off a conversion. Returns the job id immediately; progress streams
+/// into the job log. Fails fast if the converter script is missing.
+pub fn start_import(store: Arc<JobStore>, cfg: &CmfCfg, p: ImportParams) -> Result<String, String> {
+    if p.repo.trim().is_empty() {
+        return Err("empty repo id".into());
+    }
+    // Native `cortiq convert` (from crates.io, no Python) handles standard + MoE
+    // models; the bundled Python converter is used only for advanced options it
+    // doesn't support yet (linear-attention folding, v-bit shaping).
+    let use_python = p.linear_core.is_some() || p.vbit_shape.is_some() || p.mean_bits.is_some();
+    if use_python && !std::path::Path::new(&cfg.converter).exists() {
+        return Err(format!("advanced options need the Python converter, not found: {}", cfg.converter));
+    }
+    let _ = std::fs::create_dir_all(&cfg.models_dir);
+    let base = if p.name.trim().is_empty() {
+        sanitize(p.repo.rsplit('/').next().unwrap_or(&p.repo))
+    } else {
+        sanitize(&p.name)
+    };
+    let output = std::path::Path::new(&cfg.models_dir)
+        .join(format!("{base}.cmf"))
+        .to_string_lossy()
+        .to_string();
+    let output_abs = std::fs::canonicalize(&cfg.models_dir)
+        .map(|d| d.join(format!("{base}.cmf")).to_string_lossy().to_string())
+        .unwrap_or_else(|_| output.clone());
+
+    let id = gen_id(&p.repo);
+    store.insert(Job {
+        id: id.clone(),
+        repo: p.repo.clone(),
+        quant: p.quant.clone(),
+        output: output_abs.clone(),
+        state: "running".into(),
+        log: vec![format!("→ converting {} to {} ({})", p.repo, base, p.quant)],
+        started: now(),
+        finished: None,
+        size_bytes: None,
+        progress: Some(0.0),
+        phase: Some("starting".into()),
+    });
+    let cancel = Arc::new(Notify::new());
+    store.set_cancel(&id, cancel.clone());
+
+    let hf_token = if cfg.hf_token_env.is_empty() {
+        None
+    } else {
+        std::env::var(&cfg.hf_token_env).ok()
+    };
+
+    // Program + args: `cortiq convert` by default, or the Python converter for
+    // advanced options. Both stream `@PROGRESS <frac>` markers into the log.
+    let (program, args, workdir): (String, Vec<String>, std::path::PathBuf) = if use_python {
+        let conv = std::path::Path::new(&cfg.converter);
+        let mut a = vec![
+            std::fs::canonicalize(conv).map(|p| p.to_string_lossy().to_string()).unwrap_or_else(|_| cfg.converter.clone()),
+            "--model".into(), p.repo.clone(),
+            "--quant".into(), p.quant.clone(),
+            "--output".into(), output_abs.clone(),
+        ];
+        if let Some(lc) = &p.linear_core { a.push("--linear-core".into()); a.push(lc.clone()); }
+        if let Some(n) = p.nphase { a.push("--nphase".into()); a.push(n.to_string()); }
+        if let Some(vs) = &p.vbit_shape { a.push("--vbit-shape".into()); a.push(vs.clone()); }
+        if let Some(mb) = p.mean_bits { a.push("--mean-bits".into()); a.push(mb.to_string()); }
+        if let Some(g) = p.shard_max_gb { a.push("--shard-max-gb".into()); a.push(g.to_string()); }
+        if p.skip_mtp { a.push("--skip-mtp".into()); }
+        let wd = conv.parent().map(|d| d.to_path_buf()).unwrap_or_else(|| std::path::PathBuf::from("."));
+        (cfg.python_bin.clone(), a, wd)
+    } else {
+        // native Rust converter — quant name maps to cortiq's scheme.
+        let quant = match p.quant.to_ascii_uppercase().as_str() {
+            s if s.starts_with("Q4") => "q4",
+            "Q8_2F" => "q8_2f",
+            "F16" | "FP16" => "f16",
+            _ => "q8",
+        };
+        let mut a = vec![
+            "convert".into(),
+            "--model".into(), p.repo.clone(),
+            "--quant".into(), quant.into(),
+            "--output".into(), output_abs.clone(),
+        ];
+        if let Some(t) = &hf_token { a.push("--hf-token".into()); a.push(t.clone()); }
+        (cfg.cortiq_bin.clone(), a, std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")))
+    };
+
+    let ret_id = id.clone();
+    let output_for_task = output_abs.clone();
+    tokio::spawn(async move {
+        let mut cmd = tokio::process::Command::new(&program);
+        cmd.args(&args)
+            .current_dir(&workdir)
+            .env("PYTHONUNBUFFERED", "1")
+            .stdout(Stdio::piped())
+            .stderr(Stdio::piped());
+        // HF token for the Python converter (native cortiq gets --hf-token).
+        if use_python {
+            if let Some(tok) = hf_token {
+                cmd.env("HF_TOKEN", &tok).env("HUGGING_FACE_HUB_TOKEN", tok);
+            }
+        }
+        let mut child = match cmd.spawn() {
+            Ok(c) => c,
+            Err(e) => {
+                store.push_line(&id, format!("✗ spawn failed: {e}"));
+                store.finish(&id, false);
+                store.drop_cancel(&id);
+                return;
+            }
+        };
+        let stdout = child.stdout.take();
+        let stderr = child.stderr.take();
+        // stdout: parse @PROGRESS markers into progress/phase; everything else logs.
+        let s1 = store.clone();
+        let id1 = id.clone();
+        let t_out = tokio::spawn(async move {
+            if let Some(o) = stdout {
+                let mut lines = BufReader::new(o).lines();
+                while let Ok(Some(l)) = lines.next_line().await {
+                    if let Some((frac, phase)) = parse_progress(&l) {
+                        s1.set_progress(&id1, frac, phase);
+                    } else {
+                        s1.push_line(&id1, l);
+                    }
+                }
+            }
+        });
+        let s2 = store.clone();
+        let id2 = id.clone();
+        let t_err = tokio::spawn(async move {
+            if let Some(e) = stderr {
+                let mut lines = BufReader::new(e).lines();
+                while let Ok(Some(l)) = lines.next_line().await {
+                    s2.push_line(&id2, l);
+                }
+            }
+        });
+
+        tokio::select! {
+            status = child.wait() => {
+                let _ = tokio::join!(t_out, t_err);
+                let ok = status.map(|s| s.success()).unwrap_or(false);
+                if !ok {
+                    cleanup_output(&output_for_task); // drop partial/corrupt output
+                }
+                store.push_line(
+                    &id,
+                    if ok { "✓ done".into() } else { "✗ converter exited with error".into() },
+                );
+                store.finish(&id, ok);
+            }
+            _ = cancel.notified() => {
+                let _ = child.start_kill();
+                let _ = child.wait().await;
+                t_out.abort();
+                t_err.abort();
+                cleanup_output(&output_for_task);
+                store.push_line(&id, "✗ cancelled by user — partial output removed".into());
+                store.set_state(&id, "cancelled");
+            }
+        }
+        store.drop_cancel(&id);
+    });
+
+    Ok(ret_id)
+}
@@ -9,10 +9,14 @@
 #![allow(dead_code)]

 mod admin;
+mod cache;
+mod cmf_runtime;
 mod config;
 mod error;
+mod import;
 mod model;
 mod pipeline;
+mod promotion;
 mod protocols;
 mod providers;
 mod registry;
@@ -78,6 +82,16 @@ async fn main() -> anyhow::Result<()> {

    let state = SharedState::build(cfg, args.config.clone())?;

+    // Close the loop with the CMF format (github.com/infosave2007/cmf): install
+    // / update cortiq-cli and run a local `cortiq serve` if configured. Runs in
+    // the background so startup is never blocked by a cargo install or a model
+    // load; the served model is registered in the pool and usable offline.
+    {
+        let cmf_rt = state.cmf.clone();
+        let cmf_cfg = state.live().cfg.cmf.clone();
+        tokio::spawn(async move { cmf_runtime::manage(cmf_rt, cmf_cfg).await });
+    }
+
    let mut app = Router::new()
        .route("/healthz", get(|| async { "ok" }))
        .route("/readyz", get(|| async { "ready" }))
@@ -4,9 +4,10 @@

 use crate::error::{GatewayError, Result};
 use crate::model::{
-    ChatRequest, ChatResponse, Message, RouteDecision, RouteInfo, RoutingDirective,
+    ChatRequest, ChatResponse, Choice, GenParams, Message, RequestMeta, RouteDecision, RouteInfo,
+    RoutingDirective, Usage,
 };
-use crate::state::SharedState;
+use crate::state::{Live, SharedState};
 use crate::stats::RequestRecord;
 use std::time::{Instant, SystemTime, UNIX_EPOCH};

@@ -28,6 +29,60 @@ impl Pipeline {
        }
        .to_string();

+        // --- semantic cache lookup (short-circuit before the model call) ---
+        let cache = &state.cache;
+        let mut cache_key: Option<(String, Vec<f32>)> = None;
+        if cache.enabled() {
+            let live = state.live();
+            let sig = cache_sig(&req);
+            let text = self.extract_text(&req.messages, "concat_all", live.cfg.route.max_chars);
+            let embed_model = cache
+                .embed_model()
+                .map(|s| s.to_string())
+                .or_else(|| first_embedding_model(&live));
+            if let Some(model_id) = embed_model {
+                if let Some(vec) = embed(&live, &model_id, &text).await {
+                    if let Some(hit) = cache.lookup(&sig, &vec) {
+                        state.stats.record(RequestRecord {
+                            ts: now_secs(),
+                            account,
+                            protocol,
+                            directive,
+                            task_label: "cache".into(),
+                            tier: "cache".into(),
+                            score: 0.0,
+                            model_id: hit.model_used.clone(),
+                            route_source: "cache".into(),
+                            prompt_tokens: hit.prompt_tokens,
+                            completion_tokens: hit.completion_tokens,
+                            cost_usd: 0.0,
+                            latency_ms: started.elapsed().as_millis() as u64,
+                            outcome: "ok".into(),
+                            failover: false,
+                            error: None,
+                        });
+                        return Ok(cached_response(&hit));
+                    }
+                    cache_key = Some((sig, vec));
+                }
+            }
+        }
+
+        // Self-warming shadow loop: capture the prompt BEFORE `req` is moved,
+        // so we can (non-blocking, sampled) judge a local answer after the
+        // cloud reply. Client latency is unaffected. Nothing is served locally
+        // here — this only feeds the promotion table.
+        let shadow_cfg = state.live().cfg.shadow.clone();
+        let shadow_prompt = if shadow_cfg.enabled
+            && !shadow_cfg.local_model_id.is_empty()
+            && !shadow_cfg.judge_model_id.is_empty()
+        {
+            let live = state.live();
+            Some(self.extract_text(&req.messages, "last_user", live.cfg.route.max_chars))
+        } else {
+            None
+        };
+
        let result = self.run_inner(req, state).await;

        // record statistics (for both success and error)
@@ -76,12 +131,59 @@ impl Pipeline {
        };
        state.stats.record(rec);

+        // --- store a successful response in the semantic cache ---
+        if let (Ok(resp), Some((sig, vec))) = (&result, cache_key) {
+            let content = resp
+                .choices
+                .first()
+                .map(|c| c.message.content.clone())
+                .unwrap_or_default();
+            if !content.is_empty() {
+                state.cache.insert(
+                    sig,
+                    vec,
+                    resp.cortiq.selected_model.clone(),
+                    content,
+                    resp.usage.prompt_tokens,
+                    resp.usage.completion_tokens,
+                    resp.cortiq.cost_usd,
+                );
+            }
+        }
+
+        // Fire the shadow judgment for a sampled subset (non-blocking).
+        if let (Some(prompt), Ok(resp)) = (shadow_prompt, &result) {
+            let cloud_answer = resp
+                .choices
+                .first()
+                .map(|c| c.message.content.clone())
+                .unwrap_or_default();
+            let label = resp.cortiq.task_label.clone();
+            // Only judge CLOUD-served answers — the reference must be the cloud
+            // answer, not a local one (a promoted label served locally would
+            // otherwise judge local-vs-local). Canary auditing of served-local
+            // answers (forcing a cloud reference for a fraction) is the next step.
+            if !cloud_answer.is_empty()
+                && !label.is_empty()
+                && label != "cache"
+                && resp.cortiq.selected_model != shadow_cfg.local_model_id
+                && should_shadow(&prompt, shadow_cfg.sample_rate)
+            {
+                let st = state.0.clone();
+                tokio::spawn(shadow_capture(st, shadow_cfg, prompt, cloud_answer, label));
+            }
+        }
+
        result
    }

-    async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
-        let live = state.live();
-
+    /// Stages 1–3: extract text → route → select & capability-filter candidates.
+    /// Shared by both the non-streaming and streaming paths.
+    async fn resolve(
+        &self,
+        req: &ChatRequest,
+        live: &crate::state::Live,
+    ) -> Result<(RouteDecision, String, Vec<String>)> {
        // 1. Extract text for classification
        let text = self.extract_text(
            &req.messages,
@@ -89,50 +191,48 @@ impl Pipeline {
            live.cfg.route.max_chars,
        );

-        // 2. Obtain the router decision (or use the pinned model)
-        let (decision, route_source) = match &req.routing {
-            RoutingDirective::Pinned { model_id: _ } => {
-                let dec = RouteDecision {
-                    task_label: "pinned".to_string(),
-                    complexity_score: 0.0,
-                    complexity_tier: "pinned".to_string(),
-                    router_request_id: None,
-                    source: "pinned".to_string(),
-                };
-                (dec, "pinned".to_string())
-            }
-            RoutingDirective::Auto { profile } => {
-                let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
-                match live.router.route(&text, prof).await {
-                    Ok(Some(dec)) => {
-                        let src = dec.source.clone();
-                        (dec, src)
-                    }
-                    Ok(None) | Err(_) => {
-                        tracing::warn!(
-                            "Router is unavailable. Gracefully degrading to default model."
-                        );
-                        let dec = RouteDecision {
-                            task_label: "degraded".to_string(),
-                            complexity_score: 0.5,
-                            complexity_tier: "degraded".to_string(),
-                            router_request_id: None,
-                            source: "fallback".to_string(),
-                        };
-                        (dec, "fallback".to_string())
-                    }
-                }
-            }
+        // 2–3. Decide the route and pick candidate models.
+        //
+        // Local-first rules, so CMF models work with no router at all:
+        //   • `[cmf].local_only`         → every request goes to the local pool,
+        //     ignoring the router, the pin, and external providers;
+        //   • `[router].enabled = false` → skip the router, use the local pool;
+        //   • router unavailable at runtime → graceful degrade to the local pool;
+        //   • an explicitly pinned model → used directly, no router call.
+        let mk = |label: &str, source: &str| RouteDecision {
+            task_label: label.to_string(),
+            complexity_score: 0.0,
+            complexity_tier: label.to_string(),
+            router_request_id: None,
+            source: source.to_string(),
        };

-        // 3. Select candidates by complexity tier / pinned model
-        let candidates = match &req.routing {
-            RoutingDirective::Pinned { model_id } => vec![model_id.clone()],
-            RoutingDirective::Auto { .. } => {
-                if route_source == "fallback" {
-                    vec![live.routing.default_model().to_string()]
-                } else {
-                    live.routing.candidates(&decision.complexity_tier)
+        let (decision, route_source, candidates) = if live.cfg.cmf.local_only {
+            (mk("local-only", "local"), "local".to_string(), live.local_candidates())
+        } else {
+            match &req.routing {
+                RoutingDirective::Pinned { model_id } => {
+                    (mk("pinned", "pinned"), "pinned".to_string(), vec![model_id.clone()])
+                }
+                RoutingDirective::Auto { profile } => {
+                    if !live.cfg.router.enabled {
+                        (mk("router-disabled", "local"), "local".to_string(), live.local_candidates())
+                    } else {
+                        let prof = profile.as_deref().unwrap_or(&live.cfg.route.profile);
+                        match live.router.route(&text, prof).await {
+                            Ok(Some(dec)) => {
+                                let src = dec.source.clone();
+                                let tier = dec.complexity_tier.clone();
+                                (dec, src, live.routing.candidates(&tier))
+                            }
+                            Ok(None) | Err(_) => {
+                                tracing::warn!(
+                                    "Router is unavailable. Gracefully degrading to the local/default model."
+                                );
+                                (mk("degraded", "fallback"), "fallback".to_string(), live.local_candidates())
+                            }
+                        }
+                    }
                }
            }
        };
@@ -153,12 +253,47 @@ impl Pipeline {
            }
        }

+        // Local-first safety net: if routing produced no usable candidate (e.g.
+        // an unknown pinned model while the router is off), fall back to the
+        // managed local model so a configured local backend still answers.
+        if final_candidates.is_empty() && live.cfg.cmf.manage_server {
+            for id in live.local_candidates() {
+                if let Some(prov) = live.registry.get(&id) {
+                    if needs_tools && !prov.caps().tools {
+                        continue;
+                    }
+                    final_candidates.push(id);
+                }
+            }
+        }
        if final_candidates.is_empty() {
            return Err(GatewayError::UpstreamUnavailable(
                "No healthy model available that satisfies required capabilities (e.g. tools)"
                    .to_string(),
            ));
        }
+        Ok((decision, route_source, final_candidates))
+    }
+
+    async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
+        let live = state.live();
+        let (decision, route_source, mut final_candidates) = self.resolve(&req, &live).await?;
+
+        // Serving-when-promoted (gated): a promoted task-label is served by the
+        // local model FIRST (failover to cloud on error). Per-request VETO uses
+        // the router's own complexity signal — a "high"-tier request escalates
+        // to cloud even in a promoted label (the local skill is for the common
+        // case, not the hard tail). Off by default.
+        let sh = &live.cfg.shadow;
+        if sh.serve_when_promoted
+            && !sh.local_model_id.is_empty()
+            && matches!(req.routing, RoutingDirective::Auto { .. })
+            && state.promotion.serves_local(&decision.task_label)
+            && !matches!(decision.complexity_tier.as_str(), "high" | "hard" | "complex")
+            && !final_candidates.iter().any(|m| m == &sh.local_model_id)
+        {
+            final_candidates.insert(0, sh.local_model_id.clone());
+        }

        // 4. Attempt provider call with failover (fall back to alternatives on error)
        let mut last_err = None;
@@ -210,6 +345,89 @@ impl Pipeline {
        }))
    }

+    /// Streaming variant: returns the routing metadata plus a stream of SSE byte
+    /// chunks (OpenAI wire format). Failover applies only before the first byte —
+    /// once the upstream stream has started it cannot be retried. Statistics are
+    /// recorded when the stream completes (tokens taken from the final usage chunk).
+    pub async fn run_stream(
+        &self,
+        req: ChatRequest,
+        state: &SharedState,
+    ) -> Result<(RouteInfo, crate::providers::ChatStream)> {
+        let started = Instant::now();
+        let account = req.meta.account.clone();
+        let protocol = req.meta.protocol.clone();
+        let directive = match &req.routing {
+            RoutingDirective::Pinned { .. } => "pinned",
+            RoutingDirective::Auto { .. } => "auto",
+        }
+        .to_string();
+
+        let live = state.live();
+        let (decision, route_source, final_candidates) = match self.resolve(&req, &live).await {
+            Ok(t) => t,
+            Err(e) => {
+                record_error(state, &account, &protocol, &directive, started, &e);
+                return Err(e);
+            }
+        };
+
+        let mut last_err = None;
+        let mut attempt = 0usize;
+        for model_id in final_candidates {
+            let provider = match live.registry.get(&model_id) {
+                Some(p) => p,
+                None => continue,
+            };
+            tracing::info!(
+                "Streaming request (label: {}, tier: {}) to model '{}' via source '{}'",
+                decision.task_label,
+                decision.complexity_tier,
+                model_id,
+                route_source
+            );
+            match provider.chat_stream(req.clone()).await {
+                Ok(stream) => {
+                    let info = RouteInfo {
+                        task_label: decision.task_label.clone(),
+                        complexity_score: decision.complexity_score,
+                        complexity_tier: decision.complexity_tier.clone(),
+                        selected_model: model_id,
+                        route_source: route_source.clone(),
+                        router_request_id: decision.router_request_id.clone(),
+                        cost_usd: 0.0,
+                        failover: attempt > 0,
+                    };
+                    let tapped = tap_stream(
+                        stream,
+                        state.stats.clone(),
+                        provider.clone(),
+                        info.clone(),
+                        account,
+                        protocol,
+                        directive,
+                        started,
+                    );
+                    return Ok((info, Box::pin(tapped)));
+                }
+                Err(err) => {
+                    tracing::warn!(
+                        "Provider '{}' stream failed: {:?}. Attempting failover...",
+                        model_id,
+                        err
+                    );
+                    last_err = Some(err);
+                    attempt += 1;
+                }
+            }
+        }
+        let e = last_err.unwrap_or_else(|| {
+            GatewayError::UpstreamUnavailable("All candidate models failed".to_string())
+        });
+        record_error(state, &account, &protocol, &directive, started, &e);
+        Err(e)
+    }
+
    fn extract_text(&self, messages: &[Message], strategy: &str, max_chars: usize) -> String {
        let raw_text = match strategy {
            "last_user" => messages
@@ -249,3 +467,332 @@ impl Pipeline {
        }
    }
 }
+
+fn now_secs() -> u64 {
+    SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .unwrap_or_default()
+        .as_secs()
+}
+
+/// Routing signature used to partition cache entries (so a pinned model's cached
+/// answer is never returned for a different model/profile).
+fn cache_sig(req: &ChatRequest) -> String {
+    match &req.routing {
+        RoutingDirective::Auto { profile } => {
+            format!("auto:{}", profile.as_deref().unwrap_or(""))
+        }
+        RoutingDirective::Pinned { model_id } => format!("pinned:{model_id}"),
+    }
+}
+
+fn first_embedding_model(live: &Live) -> Option<String> {
+    live.cfg
+        .models
+        .iter()
+        .find(|m| m.kind == "embedding")
+        .map(|m| m.id.clone())
+}
+
+/// Embed `text` with the given model; returns the embedding vector (or None on error).
+async fn embed(live: &Live, model_id: &str, text: &str) -> Option<Vec<f32>> {
+    let provider = live.registry.get(model_id)?;
+    let body = provider.embed(serde_json::json!(text)).await.ok()?;
+    let arr = body["data"][0]["embedding"].as_array()?;
+    Some(
+        arr.iter()
+            .filter_map(|x| x.as_f64())
+            .map(|x| x as f32)
+            .collect(),
+    )
+}
+
+fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
+    ChatResponse {
+        id: "cache".to_string(),
+        model_used: hit.model_used.clone(),
+        choices: vec![Choice {
+            index: 0,
+            message: Message {
+                role: "assistant".into(),
+                content: hit.content.clone(),
+                tool_calls: vec![],
+            },
+            finish_reason: "stop".into(),
+        }],
+        usage: Usage {
+            prompt_tokens: hit.prompt_tokens,
+            completion_tokens: hit.completion_tokens,
+            total_tokens: hit.prompt_tokens + hit.completion_tokens,
+        },
+        cortiq: RouteInfo {
+            task_label: "cache".into(),
+            complexity_score: 0.0,
+            complexity_tier: "cache".into(),
+            selected_model: hit.model_used.clone(),
+            route_source: "cache".into(),
+            router_request_id: None,
+            cost_usd: 0.0,
+            failover: false,
+        },
+    }
+}
+
+fn record_error(
+    state: &SharedState,
+    account: &str,
+    protocol: &str,
+    directive: &str,
+    started: Instant,
+    e: &GatewayError,
+) {
+    state.stats.record(RequestRecord {
+        ts: now_secs(),
+        account: account.to_string(),
+        protocol: protocol.to_string(),
+        directive: directive.to_string(),
+        task_label: String::new(),
+        tier: String::new(),
+        score: 0.0,
+        model_id: String::new(),
+        route_source: String::new(),
+        prompt_tokens: 0,
+        completion_tokens: 0,
+        cost_usd: 0.0,
+        latency_ms: started.elapsed().as_millis() as u64,
+        outcome: "error".to_string(),
+        failover: false,
+        error: Some(e.to_string()),
+    });
+}
+
+/// Forward a provider SSE stream to the client verbatim while tapping the final
+/// `usage` chunk; record statistics when the stream completes.
+#[allow(clippy::too_many_arguments)]
+fn tap_stream(
+    inner: crate::providers::ChatStream,
+    stats: std::sync::Arc<crate::stats::Stats>,
+    provider: std::sync::Arc<dyn crate::providers::Provider>,
+    info: RouteInfo,
+    account: String,
+    protocol: String,
+    directive: String,
+    started: Instant,
+) -> impl futures::Stream<Item = Result<bytes::Bytes>> {
+    use futures::StreamExt;
+    async_stream::stream! {
+        let mut inner = inner;
+        let mut buf = String::new();
+        let mut prompt_tokens = 0u32;
+        let mut completion_tokens = 0u32;
+        let mut got_usage = false;
+        let mut delta_chars = 0usize;
+        let mut errored: Option<String> = None;
+
+        while let Some(item) = inner.next().await {
+            match &item {
+                Ok(bytes) => {
+                    if let Ok(s) = std::str::from_utf8(bytes) {
+                        buf.push_str(s);
+                        while let Some(idx) = buf.find("\n\n") {
+                            let evt: String = buf.drain(..idx + 2).collect();
+                            for line in evt.lines() {
+                                if let Some(data) = line.trim_start().strip_prefix("data:") {
+                                    let data = data.trim();
+                                    if data.is_empty() || data == "[DONE]" {
+                                        continue;
+                                    }
+                                    if let Ok(v) =
+                                        serde_json::from_str::<serde_json::Value>(data)
+                                    {
+                                        let u = &v["usage"];
+                                        if !u.is_null() {
+                                            prompt_tokens = u["prompt_tokens"]
+                                                .as_u64()
+                                                .unwrap_or(prompt_tokens as u64)
+                                                as u32;
+                                            completion_tokens = u["completion_tokens"]
+                                                .as_u64()
+                                                .unwrap_or(completion_tokens as u64)
+                                                as u32;
+                                            got_usage = true;
+                                        }
+                                        if let Some(c) =
+                                            v["choices"][0]["delta"]["content"].as_str()
+                                        {
+                                            delta_chars += c.chars().count();
+                                        }
+                                    }
+                                }
+                            }
+                        }
+                    }
+                }
+                Err(e) => errored = Some(e.to_string()),
+            }
+            yield item;
+        }
+
+        if !got_usage && completion_tokens == 0 {
+            // estimate when the server didn't return usage (~4 chars per token)
+            completion_tokens = (delta_chars / 4) as u32;
+        }
+        let cost = provider.price(prompt_tokens, completion_tokens);
+        stats.record(RequestRecord {
+            ts: now_secs(),
+            account,
+            protocol,
+            directive,
+            task_label: info.task_label.clone(),
+            tier: info.complexity_tier.clone(),
+            score: info.complexity_score,
+            model_id: info.selected_model.clone(),
+            route_source: info.route_source.clone(),
+            prompt_tokens,
+            completion_tokens,
+            cost_usd: cost,
+            latency_ms: started.elapsed().as_millis() as u64,
+            outcome: if errored.is_some() { "error" } else { "ok" }.to_string(),
+            failover: info.failover,
+            error: errored,
+        });
+    }
+}
+
+// ─────────────────────────── self-warming shadow loop ───────────────────────────
+
+/// Deterministic sampling by prompt hash — no RNG dependency, and the same
+/// prompt is always (not) sampled, so repeated office templates don't skew
+/// the judged set request-by-request.
+fn should_shadow(prompt: &str, rate: f64) -> bool {
+    if rate <= 0.0 {
+        return false;
+    }
+    if rate >= 1.0 {
+        return true;
+    }
+    use std::hash::{Hash, Hasher};
+    let mut h = std::collections::hash_map::DefaultHasher::new();
+    prompt.hash(&mut h);
+    (h.finish() % 10_000) < (rate * 10_000.0) as u64
+}
+
+fn clip(s: &str, max: usize) -> String {
+    if s.len() <= max {
+        s.to_string()
+    } else {
+        s.chars().take(max).collect()
+    }
+}
+
+fn shadow_req(model_id: &str, content: String, max_tokens: u32) -> ChatRequest {
+    ChatRequest {
+        routing: RoutingDirective::Pinned {
+            model_id: model_id.to_string(),
+        },
+        messages: vec![Message {
+            role: "user".to_string(),
+            content,
+            tool_calls: Vec::new(),
+        }],
+        tools: Vec::new(),
+        params: GenParams {
+            temperature: Some(0.0),
+            max_tokens: Some(max_tokens),
+            ..Default::default()
+        },
+        stream: false,
+        meta: RequestMeta::default(),
+    }
+}
+
+/// Extract `{"score":N,"hard_fail":bool}` from a judge reply, tolerant of
+/// surrounding prose. Missing/garbled → (0, true) (fail-closed: an
+/// unparseable judgment must never count as a pass).
+fn parse_judge(text: &str) -> (u8, bool) {
+    let score = text
+        .split("score")
+        .nth(1)
+        .and_then(|t| t.chars().skip_while(|c| !c.is_ascii_digit()).next())
+        .and_then(|c| c.to_digit(10))
+        .map(|d| d.min(4) as u8);
+    let hard_fail = text
+        .split("hard_fail")
+        .nth(1)
+        .map(|t| t.contains("true"))
+        .unwrap_or(true);
+    match score {
+        Some(s) => (s, hard_fail),
+        None => (0, true),
+    }
+}
+
+/// Non-blocking: answer locally, have the cheap judge score it against the
+/// already-paid cloud answer, and feed the per-label promotion table. Any
+/// failure (missing provider, upstream error, unparseable judgment) is
+/// silently dropped — the client already got the cloud answer.
+async fn shadow_capture(
+    st: std::sync::Arc<crate::state::AppState>,
+    cfg: crate::config::ShadowCfg,
+    prompt: String,
+    cloud_answer: String,
+    label: String,
+) {
+    let live = st.live.load_full();
+    let (Some(local), Some(judge)) = (
+        live.registry.get(&cfg.local_model_id),
+        live.registry.get(&cfg.judge_model_id),
+    ) else {
+        return;
+    };
+
+    // 1. Local answer (free, CPU).
+    let a_local = match local.chat(shadow_req(&cfg.local_model_id, clip(&prompt, 4000), 512)).await {
+        Ok(r) => r
+            .choices
+            .into_iter()
+            .next()
+            .map(|c| c.message.content)
+            .unwrap_or_default(),
+        Err(_) => return,
+    };
+    if a_local.trim().is_empty() {
+        return;
+    }
+
+    // 2. Reference-based judgment (cheap model, different family).
+    let jp = format!(
+        "You grade a CANDIDATE answer against a REFERENCE answer for the same task.\n\
+         Task type: {label}\n\n[TASK]\n{task}\n\n[REFERENCE]\n{reference}\n\n[CANDIDATE]\n{candidate}\n\n\
+         Reply with ONLY compact JSON: {{\"score\":0-4,\"hard_fail\":true|false}}. \
+         score>=3 means the candidate is acceptable (correct, complete, right format). \
+         hard_fail=true for fabricated facts, broken format, or unsafe content.",
+        label = label,
+        task = clip(&prompt, 2000),
+        reference = clip(&cloud_answer, 3000),
+        candidate = clip(&a_local, 3000),
+    );
+    let Ok(jr) = judge.chat(shadow_req(&cfg.judge_model_id, jp, 32)).await else {
+        return;
+    };
+    let judge_tokens = jr.usage.total_tokens;
+    let jtext = jr
+        .choices
+        .into_iter()
+        .next()
+        .map(|c| c.message.content)
+        .unwrap_or_default();
+    let (score, hard_fail) = parse_judge(&jtext);
+
+    st.promotion.record(crate::promotion::JudgeRecord {
+        ts: now_secs(),
+        task_label: label,
+        pass: score >= 3 && !hard_fail,
+        hard_fail,
+        score,
+        source: "judge".to_string(),
+        recon_e: None,
+        born_conf: None,
+        judge_tokens,
+    });
+}
@@ -0,0 +1,381 @@
+//! Cloud-as-judge promotion table (self-warming local model, MVP).
+//!
+//! Per task-label we accumulate a judge/checker PASS stream and gate the
+//! per-label lifecycle on the **Wilson-95 LOWER bound** of the pass rate —
+//! never on the point estimate, so a lucky streak of easy requests can NOT
+//! promote a label (small n → wide interval → low bound → stays SHADOW).
+//! The full design + honest holes: docs/SELF_WARMING_GATEWAY.md.
+//!
+//! This module is pure bookkeeping (like `stats::Stats`): it decides the
+//! STATE; the pipeline consults the state to decide who serves. Nothing here
+//! serves a local answer — that is the caller's decision, guarded by these
+//! states plus a per-request veto.
+
+use serde::{Deserialize, Serialize};
+use std::collections::{HashMap, VecDeque};
+use std::io::Write;
+use std::sync::Mutex;
+use std::time::{SystemTime, UNIX_EPOCH};
+
+fn now_secs() -> u64 {
+    SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs()
+}
+
+/// One judgment of a local answer (JSONL line, replayed on startup).
+#[derive(Clone, Debug, Serialize, Deserialize)]
+pub struct JudgeRecord {
+    pub ts: u64,
+    pub task_label: String,
+    /// 1 = local answer accepted (checker pass OR judge score≥bar & !hard_fail).
+    pub pass: bool,
+    /// Fact hallucination / broken format / unsafe — blocks promotion hard.
+    #[serde(default)]
+    pub hard_fail: bool,
+    /// Raw judge score 0..4 (diagnostic only); 4 for a $0 programmatic checker.
+    #[serde(default)]
+    pub score: u8,
+    /// "checker" ($0 ground-truth) or "judge" (cloud LLM-as-judge).
+    #[serde(default)]
+    pub source: String,
+    /// Local recon-E (novelty) at judge time — diagnostic; NOT a correctness gate.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub recon_e: Option<f32>,
+    /// Local calibrated Born-mass confidence — for the negative-control study.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub born_conf: Option<f32>,
+    /// Cloud tokens spent judging (0 for a programmatic checker).
+    #[serde(default)]
+    pub judge_tokens: u32,
+}
+
+/// Per-label lifecycle. Serving-locally is only ever Canary/LocalServed.
+#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
+pub enum PromotionState {
+    CloudOnly,
+    ShadowJudged,
+    Canary,
+    LocalServed,
+    Demoted,
+}
+
+/// Tuning (defaults from the memo; refit to measured ECE/traffic).
+#[derive(Clone, Debug)]
+pub struct PromotionCfg {
+    pub enabled: bool,
+    pub window: usize,       // rolling judgments per label (W)
+    pub n_min: usize,        // min judgments before promotion is even considered
+    pub promote_lb: f64,     // Wilson-95 LB of pass must clear this
+    pub demote_pass: f64,    // point pass-rate below this → demote a served label
+    pub hardfail_ub_max: f64,// Wilson-95 UPPER bound of hard_fail must stay under this
+    pub soak: usize,         // extra Canary judgments before LocalServed
+    pub z: f64,              // 1.96 = 95%
+    pub file: Option<String>,// judge.jsonl path (append + replay)
+}
+
+impl Default for PromotionCfg {
+    fn default() -> Self {
+        Self {
+            enabled: false,
+            window: 500,
+            n_min: 200,
+            promote_lb: 0.95,
+            demote_pass: 0.90,
+            hardfail_ub_max: 0.015,
+            soak: 100,
+            z: 1.96,
+            file: None,
+        }
+    }
+}
+
+struct LabelState {
+    passes: VecDeque<bool>,     // rolling window of pass/fail
+    hard_fails: VecDeque<bool>, // rolling window of hard_fail flags
+    state: PromotionState,
+    soaked: usize,              // judgments accumulated while in Canary
+}
+
+impl LabelState {
+    fn new() -> Self {
+        Self {
+            passes: VecDeque::new(),
+            hard_fails: VecDeque::new(),
+            state: PromotionState::ShadowJudged,
+            soaked: 0,
+        }
+    }
+}
+
+pub struct Promotion {
+    cfg: PromotionCfg,
+    labels: Mutex<HashMap<String, LabelState>>,
+}
+
+/// Wilson score interval bound for a Bernoulli rate. `upper=false` → lower.
+/// s successes of n trials, z the normal quantile. Distribution-aware: small
+/// n gives a wide interval (low lower bound / high upper bound) by construction.
+pub fn wilson_bound(s: usize, n: usize, z: f64, upper: bool) -> f64 {
+    if n == 0 {
+        return if upper { 1.0 } else { 0.0 };
+    }
+    let n = n as f64;
+    let p = s as f64 / n;
+    let z2 = z * z;
+    let denom = 1.0 + z2 / n;
+    let centre = p + z2 / (2.0 * n);
+    let margin = z * ((p * (1.0 - p) / n) + z2 / (4.0 * n * n)).sqrt();
+    let v = if upper { centre + margin } else { centre - margin };
+    (v / denom).clamp(0.0, 1.0)
+}
+
+impl Promotion {
+    pub fn new(cfg: PromotionCfg) -> std::sync::Arc<Self> {
+        let p = std::sync::Arc::new(Self { cfg, labels: Mutex::new(HashMap::new()) });
+        p.replay();
+        p
+    }
+
+    fn replay(&self) {
+        let Some(path) = &self.cfg.file else { return };
+        let Ok(content) = std::fs::read_to_string(path) else { return };
+        for line in content.lines() {
+            if let Ok(rec) = serde_json::from_str::<JudgeRecord>(line) {
+                self.apply(&rec);
+            }
+        }
+    }
+
+    /// Record a judgment: update the rolling windows, advance the state
+    /// machine, append to judge.jsonl. Returns the (possibly new) state.
+    pub fn record(&self, rec: JudgeRecord) -> PromotionState {
+        let st = self.apply(&rec);
+        if let Some(path) = &self.cfg.file {
+            if let Ok(line) = serde_json::to_string(&rec) {
+                if let Ok(mut f) =
+                    std::fs::OpenOptions::new().create(true).append(true).open(path)
+                {
+                    let _ = writeln!(f, "{line}");
+                }
+            }
+        }
+        st
+    }
+
+    fn apply(&self, rec: &JudgeRecord) -> PromotionState {
+        let mut map = self.labels.lock().unwrap();
+        let ls = map.entry(rec.task_label.clone()).or_insert_with(LabelState::new);
+        push_cap(&mut ls.passes, rec.pass, self.cfg.window);
+        push_cap(&mut ls.hard_fails, rec.hard_fail, self.cfg.window);
+        self.advance(ls);
+        ls.state
+    }
+
+    fn advance(&self, ls: &mut LabelState) {
+        let n = ls.passes.len();
+        let s = ls.passes.iter().filter(|&&p| p).count();
+        let hf = ls.hard_fails.iter().filter(|&&h| h).count();
+        let lb = wilson_bound(s, n, self.cfg.z, false);
+        let point = if n > 0 { s as f64 / n as f64 } else { 0.0 };
+        let recent_hf = ls.hard_fails.iter().rev().take(50).filter(|&&h| h).count();
+
+        // Zero-tolerance on hard-fails in the window: a Wilson UPPER bound on
+        // hard-fail rate ≤ 0.015 is UNREACHABLE at n=200 even with 0 fails
+        // (rule-of-three: 0/200 → ~0.019) — the panel flagged this. Requiring
+        // ZERO recent hard-fails is achievable AND safer (a recent
+        // hallucination/unsafe answer → stay on cloud until it ages out).
+        let bar = n >= self.cfg.n_min && lb >= self.cfg.promote_lb && hf == 0;
+        let regressed = point < self.cfg.demote_pass || recent_hf >= 2;
+
+        ls.state = match ls.state {
+            PromotionState::CloudOnly | PromotionState::ShadowJudged => {
+                if bar { ls.soaked = 0; PromotionState::Canary } else { PromotionState::ShadowJudged }
+            }
+            PromotionState::Canary => {
+                if regressed {
+                    PromotionState::Demoted
+                } else {
+                    ls.soaked += 1;
+                    if ls.soaked >= self.cfg.soak && bar {
+                        PromotionState::LocalServed
+                    } else {
+                        PromotionState::Canary
+                    }
+                }
+            }
+            PromotionState::LocalServed => {
+                if regressed { PromotionState::Demoted } else { PromotionState::LocalServed }
+            }
+            // Demoted goes back to SHADOW to re-earn promotion (skill stays in .cmf).
+            PromotionState::Demoted => PromotionState::ShadowJudged,
+        };
+    }
+
+    /// Current lifecycle state of a label.
+    pub fn state(&self, label: &str) -> PromotionState {
+        self.labels
+            .lock()
+            .unwrap()
+            .get(label)
+            .map(|l| l.state)
+            .unwrap_or(PromotionState::CloudOnly)
+    }
+
+    /// Should a request of this label be SERVED by the local model?
+    /// (Canary/LocalServed; caller still applies the per-request veto.)
+    pub fn serves_local(&self, label: &str) -> bool {
+        matches!(self.state(label), PromotionState::Canary | PromotionState::LocalServed)
+    }
+
+    /// Snapshot for /admin: (state, n, pass-rate, Wilson-LB) per label.
+    pub fn snapshot(&self) -> Vec<(String, PromotionState, usize, f64, f64)> {
+        let map = self.labels.lock().unwrap();
+        let mut out: Vec<_> = map
+            .iter()
+            .map(|(k, ls)| {
+                let n = ls.passes.len();
+                let s = ls.passes.iter().filter(|&&p| p).count();
+                let p = if n > 0 { s as f64 / n as f64 } else { 0.0 };
+                (k.clone(), ls.state, n, p, wilson_bound(s, n, self.cfg.z, false))
+            })
+            .collect();
+        out.sort_by(|a, b| b.2.cmp(&a.2));
+        out
+    }
+
+    pub fn enabled(&self) -> bool {
+        self.cfg.enabled
+    }
+}
+
+fn push_cap<T>(q: &mut VecDeque<T>, v: T, cap: usize) {
+    q.push_back(v);
+    while q.len() > cap {
+        q.pop_front();
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn cfg() -> PromotionCfg {
+        PromotionCfg { enabled: true, n_min: 200, soak: 0, ..Default::default() }
+    }
+    fn rec(label: &str, pass: bool, hard_fail: bool) -> JudgeRecord {
+        JudgeRecord {
+            ts: 0, task_label: label.into(), pass, hard_fail, score: if pass { 4 } else { 0 },
+            source: "checker".into(), recon_e: None, born_conf: None, judge_tokens: 0,
+        }
+    }
+
+    #[test]
+    fn wilson_lb_penalizes_small_n() {
+        // 5/5 perfect but tiny n → lower bound far below 0.95.
+        assert!(wilson_bound(5, 5, 1.96, false) < 0.6);
+        // 195/200 at 97.5% → lower bound clears 0.95.
+        assert!(wilson_bound(195, 200, 1.96, false) >= 0.94);
+        // upper bound of 0 hard-fails over 200 ≈ 0.019 (rule-of-three-ish).
+        assert!(wilson_bound(0, 200, 1.96, true) < 0.025);
+    }
+
+    #[test]
+    fn no_promotion_on_a_lucky_streak() {
+        let p = Promotion::new(cfg());
+        // 50 straight passes: point-rate 100% but n < n_min AND LB < bar.
+        for _ in 0..50 {
+            p.record(rec("code", true, false));
+        }
+        assert_eq!(p.state("code"), PromotionState::ShadowJudged);
+        assert!(!p.serves_local("code"));
+    }
+
+    #[test]
+    fn promotes_only_with_enough_high_evidence() {
+        let p = Promotion::new(cfg());
+        // 198/200 (99%) + zero hard-fails → Wilson-LB ≈ 0.964 ≥ 0.95 → Canary.
+        // (195/200 = 97.5% gives LB ≈ 0.943 < 0.95 and would NOT promote —
+        // the bound is deliberately strict.)
+        for i in 0..200 {
+            p.record(rec("code", i % 100 != 0, false)); // 2 fails / 200
+        }
+        assert_eq!(p.state("code"), PromotionState::Canary); // soak=0 → straight to Canary
+        assert!(p.serves_local("code"));
+    }
+
+    /// Runnable narrative of the full loop:
+    ///   cargo test --release demo_full_lifecycle -- --nocapture
+    /// Two labels side by side — a mature skill that promotes to local
+    /// serving, and a task the local can't do (stays on cloud). Proves the
+    /// state machine both PROMOTES on real evidence and REFUSES safely.
+    #[test]
+    fn demo_full_lifecycle() {
+        let p = Promotion::new(PromotionCfg { soak: 50, ..cfg() });
+        let show = |label: &str| {
+            let snap = p.snapshot();
+            let row = snap.iter().find(|r| r.0 == label);
+            match row {
+                Some((_, st, n, pr, lb)) => println!(
+                    "    {label:<12} {:<13} n={n:<4} pass={:.0}% LB={:.3} serves_local={}",
+                    format!("{st:?}"), pr * 100.0, lb, p.serves_local(label)
+                ),
+                None => println!("    {label:<12} (cloud-only, no judgments)"),
+            }
+        };
+
+        println!("\n── self-warming loop, live ──");
+        // A mature skill (e.g. ru-tech): local answers pass the judge ~99%.
+        println!("  1) SHADOW→CANARY: судим локаль 'ru_tech' (сильный скилл ~99% pass), 210 судейств:");
+        for i in 0..210 {
+            p.record(rec("ru_tech", i % 100 != 0, false));
+        }
+        show("ru_tech");
+        assert_eq!(p.state("ru_tech"), PromotionState::Canary, "n≥200 & LB≥0.95 → CANARY");
+
+        println!("  2) CANARY soak (ещё 60 судейств держат планку) → LOCAL_SERVED:");
+        for i in 0..60 {
+            p.record(rec("ru_tech", i % 100 != 0, false));
+        }
+        show("ru_tech");
+        assert_eq!(p.state("ru_tech"), PromotionState::LocalServed, "soak → LOCAL_SERVED");
+        assert!(p.serves_local("ru_tech"), "теперь обслуживается локально");
+
+        // The code dataset: local fails the checker → never promotes.
+        println!("  3) параллельно 'code' (0.8B не тянет генерацию кода, 0% pass):");
+        for _ in 0..250 {
+            p.record(rec("code", false, false));
+        }
+        show("code");
+        assert_eq!(p.state("code"), PromotionState::ShadowJudged, "остаётся на облаке — безопасно");
+        assert!(!p.serves_local("code"));
+
+        // Drift on the promoted label → auto-demote back to cloud.
+        println!("  4) деградация 'ru_tech' (дрейф) → авто-DEMOTED, назад на облако:");
+        for _ in 0..120 {
+            p.record(rec("ru_tech", false, false));
+        }
+        show("ru_tech");
+        assert!(!p.serves_local("ru_tech"), "демоушен снял с локали");
+        println!("  ── петля: промоушен по доказательству, отказ и откат — безопасно ──\n");
+    }
+
+    #[test]
+    fn hard_fail_and_regression_block_or_demote() {
+        // A hard-fail spike keeps hf upper bound high → no promotion.
+        let p = Promotion::new(cfg());
+        for i in 0..200 {
+            p.record(rec("risky", true, i % 20 == 0)); // 5% hard_fail
+        }
+        assert_ne!(p.state("risky"), PromotionState::LocalServed);
+
+        // Promote a clean label, then feed a regression → demote.
+        let p2 = Promotion::new(PromotionCfg { soak: 0, ..cfg() });
+        for i in 0..200 {
+            p2.record(rec("t", i % 100 != 0, false)); // 198/200, 0 hard-fails
+        }
+        assert_eq!(p2.state("t"), PromotionState::Canary);
+        for _ in 0..300 {
+            p2.record(rec("t", false, false)); // quality collapses
+        }
+        assert!(matches!(p2.state("t"), PromotionState::Demoted | PromotionState::ShadowJudged));
+    }
+}
@@ -0,0 +1,209 @@
+//! Inbound Anthropic Messages adapter: `POST /v1/messages`.
+//! Translates the Anthropic request into the canonical [`ChatRequest`], runs the
+//! pipeline, and converts the result back to the Anthropic Messages format
+//! (non-streaming JSON or streaming SSE).
+
+use crate::error::{GatewayError, Result};
+use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
+use crate::protocols::openai_chat::cortiq_headers;
+use crate::state::SharedState;
+use axum::response::{IntoResponse, Response};
+use axum::{extract::State, routing::post, Json, Router};
+use serde::Deserialize;
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/v1/messages", post(handler))
+}
+
+#[derive(Deserialize)]
+struct MessagesRequest {
+    model: String,
+    #[serde(default)]
+    system: serde_json::Value,
+    #[serde(default)]
+    messages: Vec<serde_json::Value>,
+    #[serde(default)]
+    max_tokens: Option<u32>,
+    #[serde(default)]
+    temperature: Option<f32>,
+    #[serde(default)]
+    top_p: Option<f32>,
+    #[serde(default)]
+    stream: bool,
+    #[serde(default)]
+    tools: Vec<serde_json::Value>,
+}
+
+/// Anthropic content is a string or an array of blocks; flatten to text.
+fn content_to_string(v: &serde_json::Value) -> String {
+    if let Some(s) = v.as_str() {
+        return s.to_string();
+    }
+    if let Some(arr) = v.as_array() {
+        return arr
+            .iter()
+            .filter_map(|b| b["text"].as_str())
+            .collect::<Vec<_>>()
+            .join("");
+    }
+    String::new()
+}
+
+fn parse_routing(model: &str) -> RoutingDirective {
+    if let Some(rest) = model.strip_prefix("cortiq-auto") {
+        RoutingDirective::Auto {
+            profile: rest.strip_prefix(':').map(|p| p.to_string()),
+        }
+    } else {
+        RoutingDirective::Pinned {
+            model_id: model.to_string(),
+        }
+    }
+}
+
+async fn handler(
+    State(state): State<SharedState>,
+    Json(req): Json<MessagesRequest>,
+) -> Result<Response> {
+    if !state.live().cfg.protocols.anthropic_messages {
+        return Err(GatewayError::InvalidRequest(
+            "anthropic_messages protocol is disabled".into(),
+        ));
+    }
+    if req.messages.is_empty() {
+        return Err(GatewayError::InvalidRequest(
+            "messages must not be empty".into(),
+        ));
+    }
+
+    let mut messages = Vec::new();
+    let sys = content_to_string(&req.system);
+    if !sys.is_empty() {
+        messages.push(Message {
+            role: "system".into(),
+            content: sys,
+            tool_calls: vec![],
+        });
+    }
+    for m in &req.messages {
+        messages.push(Message {
+            role: m["role"].as_str().unwrap_or("user").to_string(),
+            content: content_to_string(&m["content"]),
+            tool_calls: vec![],
+        });
+    }
+
+    let canonical = ChatRequest {
+        routing: parse_routing(&req.model),
+        messages,
+        tools: req.tools,
+        params: GenParams {
+            temperature: req.temperature,
+            max_tokens: req.max_tokens,
+            top_p: req.top_p,
+            stop: Vec::new(),
+            passthrough: Default::default(),
+        },
+        stream: req.stream,
+        meta: RequestMeta {
+            protocol: "anthropic_messages".into(),
+            ..Default::default()
+        },
+    };
+
+    if req.stream {
+        let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
+        let mut headers = cortiq_headers(&info);
+        headers.insert(
+            axum::http::header::CONTENT_TYPE,
+            axum::http::HeaderValue::from_static("text/event-stream"),
+        );
+        let translated = openai_to_anthropic_sse(stream);
+        return Ok((headers, axum::body::Body::from_stream(translated)).into_response());
+    }
+
+    let resp = state.pipeline.run(canonical, &state).await?;
+    let answer = resp
+        .choices
+        .first()
+        .map(|c| c.message.content.clone())
+        .unwrap_or_default();
+    let headers = cortiq_headers(&resp.cortiq);
+    let body = serde_json::json!({
+        "id": resp.id,
+        "type": "message",
+        "role": "assistant",
+        "model": resp.model_used,
+        "content": [{ "type": "text", "text": answer }],
+        "stop_reason": "end_turn",
+        "stop_sequence": serde_json::Value::Null,
+        "usage": {
+            "input_tokens": resp.usage.prompt_tokens,
+            "output_tokens": resp.usage.completion_tokens,
+        },
+    });
+    Ok((headers, Json(body)).into_response())
+}
+
+/// Translate the gateway's internal OpenAI-format SSE into Anthropic Messages SSE
+/// events, so Anthropic-native clients can stream regardless of the upstream provider.
+fn openai_to_anthropic_sse(
+    stream: crate::providers::ChatStream,
+) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
+    use futures::StreamExt;
+    async_stream::stream! {
+        fn evt(event: &str, data: serde_json::Value) -> bytes::Bytes {
+            bytes::Bytes::from(format!("event: {event}\ndata: {data}\n\n"))
+        }
+
+        yield Ok(evt("message_start", serde_json::json!({
+            "type": "message_start",
+            "message": {"id": "msg_gw", "type": "message", "role": "assistant",
+                        "content": [], "model": "cortiq-auto", "stop_reason": null,
+                        "usage": {"input_tokens": 0, "output_tokens": 0}}
+        })));
+        yield Ok(evt("content_block_start", serde_json::json!({
+            "type": "content_block_start", "index": 0,
+            "content_block": {"type": "text", "text": ""}
+        })));
+
+        let mut buf = String::new();
+        let mut completion_tokens = 0u64;
+        futures::pin_mut!(stream);
+        while let Some(item) = stream.next().await {
+            let bytes = match item {
+                Ok(b) => b,
+                Err(e) => { yield Err(e); break; }
+            };
+            if let Ok(s) = std::str::from_utf8(&bytes) {
+                buf.push_str(s);
+            }
+            while let Some(idx) = buf.find("\n\n") {
+                let chunk: String = buf.drain(..idx + 2).collect();
+                for line in chunk.lines() {
+                    let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
+                    let data = data.trim();
+                    if data.is_empty() || data == "[DONE]" { continue; }
+                    let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
+                    if let Some(u) = v.get("usage").filter(|u| !u.is_null()) {
+                        completion_tokens = u["completion_tokens"].as_u64().unwrap_or(completion_tokens);
+                    }
+                    if let Some(t) = v["choices"][0]["delta"]["content"].as_str() {
+                        yield Ok(evt("content_block_delta", serde_json::json!({
+                            "type": "content_block_delta", "index": 0,
+                            "delta": {"type": "text_delta", "text": t}
+                        })));
+                    }
+                }
+            }
+        }
+
+        yield Ok(evt("content_block_stop", serde_json::json!({"type": "content_block_stop", "index": 0})));
+        yield Ok(evt("message_delta", serde_json::json!({
+            "type": "message_delta",
+            "delta": {"stop_reason": "end_turn", "stop_sequence": null},
+            "usage": {"output_tokens": completion_tokens}
+        })));
+        yield Ok(evt("message_stop", serde_json::json!({"type": "message_stop"})));
+    }
+}
@@ -0,0 +1,162 @@
+//! MCP (Model Context Protocol) server: `POST /mcp` (JSON-RPC 2.0).
+//!
+//! Exposes the gateway's routing to MCP-native orchestrators as two tools:
+//! `cortiq_route` (classify a prompt → routing decision) and `cortiq_chat`
+//! (run a prompt through intelligent routing → answer). Single JSON endpoint;
+//! notifications are accepted with `202`. Gated by `protocols.mcp`.
+
+use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
+use crate::state::SharedState;
+use axum::response::{IntoResponse, Response};
+use axum::{extract::State, http::StatusCode, routing::post, Json, Router};
+use serde_json::{json, Value};
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/mcp", post(handler))
+}
+
+fn parse_routing(model: &str) -> RoutingDirective {
+    if let Some(rest) = model.strip_prefix("cortiq-auto") {
+        RoutingDirective::Auto {
+            profile: rest.strip_prefix(':').map(|p| p.to_string()),
+        }
+    } else {
+        RoutingDirective::Pinned {
+            model_id: model.to_string(),
+        }
+    }
+}
+
+fn tools_list() -> Value {
+    json!([
+        {
+            "name": "cortiq_route",
+            "description": "Classify a prompt and return the routing decision (task type, complexity tier, score, and the model the gateway would select).",
+            "inputSchema": {
+                "type": "object",
+                "properties": { "text": { "type": "string", "description": "The prompt to classify." } },
+                "required": ["text"]
+            }
+        },
+        {
+            "name": "cortiq_chat",
+            "description": "Run a prompt through the gateway's intelligent routing and return the selected model's answer.",
+            "inputSchema": {
+                "type": "object",
+                "properties": {
+                    "prompt": { "type": "string", "description": "The user prompt." },
+                    "model": { "type": "string", "description": "Optional: cortiq-auto[:profile] or a model id (default cortiq-auto)." }
+                },
+                "required": ["prompt"]
+            }
+        }
+    ])
+}
+
+fn tool_text(text: String, is_error: bool) -> Value {
+    json!({ "content": [{ "type": "text", "text": text }], "isError": is_error })
+}
+
+async fn handle_tool_call(
+    state: &SharedState,
+    params: &Value,
+) -> std::result::Result<Value, (i64, String)> {
+    let name = params["name"].as_str().unwrap_or("");
+    let args = &params["arguments"];
+    let live = state.live();
+    match name {
+        "cortiq_route" => {
+            let text = args["text"].as_str().unwrap_or("");
+            let prof = live.cfg.route.profile.clone();
+            match live.router.route(text, &prof).await {
+                Ok(Some(d)) => Ok(tool_text(
+                    format!(
+                        "task_label={}\ncomplexity_tier={}\ncomplexity_score={:.3}\ncandidates={:?}",
+                        d.task_label,
+                        d.complexity_tier,
+                        d.complexity_score,
+                        live.routing.candidates(&d.complexity_tier)
+                    ),
+                    false,
+                )),
+                _ => Ok(tool_text(
+                    "router unavailable — would use the default model".to_string(),
+                    false,
+                )),
+            }
+        }
+        "cortiq_chat" => {
+            let prompt = args["prompt"].as_str().unwrap_or("");
+            let model = args["model"].as_str().unwrap_or("cortiq-auto");
+            let req = ChatRequest {
+                routing: parse_routing(model),
+                messages: vec![Message {
+                    role: "user".into(),
+                    content: prompt.to_string(),
+                    tool_calls: vec![],
+                }],
+                tools: vec![],
+                params: GenParams::default(),
+                stream: false,
+                meta: RequestMeta {
+                    account: "mcp".into(),
+                    protocol: "mcp".into(),
+                    ..Default::default()
+                },
+            };
+            match state.pipeline.run(req, state).await {
+                Ok(resp) => {
+                    let answer = resp
+                        .choices
+                        .first()
+                        .map(|c| c.message.content.clone())
+                        .unwrap_or_default();
+                    Ok(tool_text(answer, false))
+                }
+                Err(e) => Ok(tool_text(e.to_string(), true)),
+            }
+        }
+        _ => Err((-32602, format!("unknown tool '{name}'"))),
+    }
+}
+
+async fn handler(State(state): State<SharedState>, Json(req): Json<Value>) -> Response {
+    if !state.live().cfg.protocols.mcp {
+        return (StatusCode::NOT_FOUND, "mcp protocol is disabled").into_response();
+    }
+
+    let method = req["method"].as_str().unwrap_or("").to_string();
+    let id = req.get("id").cloned();
+
+    // JSON-RPC notifications carry no id and expect no response
+    if id.is_none() || method.starts_with("notifications/") {
+        return StatusCode::ACCEPTED.into_response();
+    }
+    let id = id.unwrap_or(Value::Null);
+
+    let result: std::result::Result<Value, (i64, String)> = match method.as_str() {
+        "initialize" => {
+            let pv = req["params"]["protocolVersion"]
+                .as_str()
+                .unwrap_or("2024-11-05")
+                .to_string();
+            Ok(json!({
+                "protocolVersion": pv,
+                "capabilities": { "tools": {} },
+                "serverInfo": { "name": "cortiq-gateway", "version": env!("CARGO_PKG_VERSION") }
+            }))
+        }
+        "ping" => Ok(json!({})),
+        "tools/list" => Ok(json!({ "tools": tools_list() })),
+        "tools/call" => handle_tool_call(&state, &req["params"]).await,
+        other => Err((-32601, format!("method not found: {other}"))),
+    };
+
+    let body = match result {
+        Ok(r) => json!({ "jsonrpc": "2.0", "id": id, "result": r }),
+        Err((code, message)) => {
+            json!({ "jsonrpc": "2.0", "id": id, "error": { "code": code, "message": message } })
+        }
+    };
+    Json(body).into_response()
+}
@@ -1,7 +1,13 @@
 //! Incoming protocol adapters. Each one builds its own axum routes and translates
 //! its protocol to/from the canonical model. Enabled via flags in `[protocols]`.

+pub mod anthropic_messages;
+pub mod mcp;
+pub mod native_passthrough;
 pub mod openai_chat;
+pub mod openai_completions;
+pub mod openai_embeddings;
+pub mod openai_models;

 use crate::state::SharedState;
 use axum::Router;
@@ -10,8 +16,12 @@ use axum::Router;
 /// each handler checks the live `protocols.*` flag from config itself, so
 /// toggles in the admin panel take effect without a restart (hot switching).
 pub fn build_router() -> Router<SharedState> {
-    Router::new().merge(openai_chat::routes())
-    // TODO: openai_completions, openai_embeddings, openai_models,
-    //       anthropic_messages, mcp, native_passthrough — add here as well,
-    //       with a check of the corresponding flag inside the handler.
+    Router::new()
+        .merge(openai_chat::routes())
+        .merge(openai_completions::routes())
+        .merge(openai_embeddings::routes())
+        .merge(openai_models::routes())
+        .merge(anthropic_messages::routes())
+        .merge(mcp::routes())
+        .merge(native_passthrough::routes())
 }
@@ -0,0 +1,86 @@
+//! Native passthrough adapter: `POST /route`.
+//! Returns the gateway's routing decision (task type, complexity, candidate models)
+//! without calling a model — direct access for clients that route themselves.
+
+use crate::error::{GatewayError, Result};
+use crate::state::SharedState;
+use axum::{extract::State, response::IntoResponse, routing::post, Json, Router};
+use serde::Deserialize;
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/route", post(handler))
+}
+
+#[derive(Deserialize)]
+struct RouteRequest {
+    /// `{ "input": { "text": "..." } }` (router-style) — optional.
+    #[serde(default)]
+    input: serde_json::Value,
+    /// `{ "text": "..." }` — convenience shorthand.
+    #[serde(default)]
+    text: Option<String>,
+    #[serde(default)]
+    profile: Option<String>,
+}
+
+async fn handler(
+    State(state): State<SharedState>,
+    Json(req): Json<RouteRequest>,
+) -> Result<impl IntoResponse> {
+    let live = state.live();
+    if !live.cfg.protocols.native_passthrough {
+        return Err(GatewayError::InvalidRequest(
+            "native_passthrough protocol is disabled".into(),
+        ));
+    }
+
+    let text = req
+        .text
+        .clone()
+        .or_else(|| req.input["text"].as_str().map(|s| s.to_string()))
+        .or_else(|| req.input.as_str().map(|s| s.to_string()))
+        .unwrap_or_default();
+    if text.is_empty() {
+        return Err(GatewayError::InvalidRequest(
+            "missing input text (use {\"text\": ...} or {\"input\": {\"text\": ...}})".into(),
+        ));
+    }
+
+    let profile = req
+        .profile
+        .clone()
+        .unwrap_or_else(|| live.cfg.route.profile.clone());
+
+    let (decision, source, candidates) = match live.router.route(&text, &profile).await {
+        Ok(Some(d)) => {
+            let cands = live.routing.candidates(&d.complexity_tier);
+            (Some(d), "router", cands)
+        }
+        _ => (
+            None,
+            "fallback",
+            vec![live.routing.default_model().to_string()],
+        ),
+    };
+
+    let selected = candidates.first().cloned().unwrap_or_default();
+    let decision_json = match &decision {
+        Some(d) => serde_json::json!({
+            "task_label": d.task_label,
+            "complexity": { "score": d.complexity_score, "tier": d.complexity_tier },
+            "router_request_id": d.router_request_id,
+        }),
+        None => serde_json::json!({
+            "task_label": "degraded",
+            "complexity": { "score": 0.5, "tier": "degraded" },
+            "router_request_id": serde_json::Value::Null,
+        }),
+    };
+
+    Ok(Json(serde_json::json!({
+        "decision": decision_json,
+        "source": source,
+        "candidates": candidates,
+        "selected_model": selected,
+    })))
+}
@@ -3,11 +3,37 @@
 //! to the pipeline, then converts the canonical [`ChatResponse`] back to OpenAI format.

 use crate::error::{GatewayError, Result};
-use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
+use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RouteInfo, RoutingDirective};
 use crate::state::SharedState;
+use axum::response::{IntoResponse, Response};
 use axum::{extract::State, routing::post, Json, Router};
 use serde::Deserialize;

+/// Build the `X-Cortiq-*` response headers from routing metadata.
+pub(crate) fn cortiq_headers(c: &RouteInfo) -> axum::http::HeaderMap {
+    use axum::http::{HeaderMap, HeaderValue};
+    fn put(h: &mut HeaderMap, k: &'static str, v: &str) {
+        if let Ok(val) = HeaderValue::from_str(v) {
+            h.insert(k, val);
+        }
+    }
+    let mut h = HeaderMap::new();
+    put(&mut h, "X-Cortiq-Task-Label", &c.task_label);
+    put(
+        &mut h,
+        "X-Cortiq-Complexity-Score",
+        &c.complexity_score.to_string(),
+    );
+    put(&mut h, "X-Cortiq-Complexity-Tier", &c.complexity_tier);
+    put(&mut h, "X-Cortiq-Selected-Model", &c.selected_model);
+    put(&mut h, "X-Cortiq-Route-Source", &c.route_source);
+    put(&mut h, "X-Cortiq-Cost-Usd", &c.cost_usd.to_string());
+    if let Some(id) = &c.router_request_id {
+        put(&mut h, "X-Cortiq-Request-Id", id);
+    }
+    h
+}
+
 pub fn routes() -> Router<SharedState> {
    Router::new().route("/v1/chat/completions", post(handler))
 }
@@ -46,7 +72,7 @@ fn parse_routing(model: &str) -> RoutingDirective {
 async fn handler(
    State(state): State<SharedState>,
    Json(req): Json<OpenAiChatRequest>,
-) -> Result<impl axum::response::IntoResponse> {
+) -> Result<Response> {
    // hot protocol toggle: if the adapter is disabled in config — return 404
    if !state.live().cfg.protocols.openai_chat {
        return Err(GatewayError::InvalidRequest(
@@ -77,52 +103,23 @@ async fn handler(
        },
    };

+    // streaming: forward provider SSE chunks verbatim with X-Cortiq-* headers
    if req.stream {
-        return Err(GatewayError::InvalidRequest(
-            "Streaming is not yet supported in this version".into(),
-        ));
+        let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
+        let mut headers = cortiq_headers(&info);
+        headers.insert(
+            axum::http::header::CONTENT_TYPE,
+            axum::http::HeaderValue::from_static("text/event-stream"),
+        );
+        headers.insert(
+            axum::http::header::CACHE_CONTROL,
+            axum::http::HeaderValue::from_static("no-cache"),
+        );
+        return Ok((headers, axum::body::Body::from_stream(stream)).into_response());
    }

    let resp = state.pipeline.run(canonical, &state).await?;
-
-    let mut headers = axum::http::HeaderMap::new();
-    headers.insert(
-        "X-Cortiq-Task-Label",
-        axum::http::HeaderValue::from_str(&resp.cortiq.task_label)
-            .unwrap_or(axum::http::HeaderValue::from_static("")),
-    );
-    headers.insert(
-        "X-Cortiq-Complexity-Score",
-        axum::http::HeaderValue::from_str(&resp.cortiq.complexity_score.to_string())
-            .unwrap_or(axum::http::HeaderValue::from_static("0.0")),
-    );
-    headers.insert(
-        "X-Cortiq-Complexity-Tier",
-        axum::http::HeaderValue::from_str(&resp.cortiq.complexity_tier)
-            .unwrap_or(axum::http::HeaderValue::from_static("")),
-    );
-    headers.insert(
-        "X-Cortiq-Selected-Model",
-        axum::http::HeaderValue::from_str(&resp.cortiq.selected_model)
-            .unwrap_or(axum::http::HeaderValue::from_static("")),
-    );
-    headers.insert(
-        "X-Cortiq-Route-Source",
-        axum::http::HeaderValue::from_str(&resp.cortiq.route_source)
-            .unwrap_or(axum::http::HeaderValue::from_static("")),
-    );
-    headers.insert(
-        "X-Cortiq-Cost-Usd",
-        axum::http::HeaderValue::from_str(&resp.cortiq.cost_usd.to_string())
-            .unwrap_or(axum::http::HeaderValue::from_static("0.0")),
-    );
-    if let Some(req_id) = &resp.cortiq.router_request_id {
-        headers.insert(
-            "X-Cortiq-Request-Id",
-            axum::http::HeaderValue::from_str(req_id)
-                .unwrap_or(axum::http::HeaderValue::from_static("")),
-        );
-    }
+    let headers = cortiq_headers(&resp.cortiq);

    let mut body = serde_json::json!({
        "id": resp.id,
@@ -164,5 +161,5 @@ async fn handler(
        });
    }

-    Ok((headers, Json(body)))
+    Ok((headers, Json(body)).into_response())
 }
@@ -0,0 +1,183 @@
+//! Inbound OpenAI Completions (legacy) adapter: `POST /v1/completions`.
+//! Converts the legacy `prompt` into a single user message, runs the pipeline, and
+//! returns the legacy `text_completion` shape (non-streaming JSON or streaming SSE).
+
+use crate::error::{GatewayError, Result};
+use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
+use crate::protocols::openai_chat::cortiq_headers;
+use crate::state::SharedState;
+use axum::response::{IntoResponse, Response};
+use axum::{extract::State, routing::post, Json, Router};
+use serde::Deserialize;
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/v1/completions", post(handler))
+}
+
+#[derive(Deserialize)]
+struct CompletionsRequest {
+    model: String,
+    prompt: serde_json::Value,
+    #[serde(default)]
+    temperature: Option<f32>,
+    #[serde(default)]
+    max_tokens: Option<u32>,
+    #[serde(default)]
+    top_p: Option<f32>,
+    #[serde(default)]
+    stream: bool,
+}
+
+fn parse_routing(model: &str) -> RoutingDirective {
+    if let Some(rest) = model.strip_prefix("cortiq-auto") {
+        RoutingDirective::Auto {
+            profile: rest.strip_prefix(':').map(|p| p.to_string()),
+        }
+    } else {
+        RoutingDirective::Pinned {
+            model_id: model.to_string(),
+        }
+    }
+}
+
+fn prompt_to_string(v: &serde_json::Value) -> String {
+    if let Some(s) = v.as_str() {
+        return s.to_string();
+    }
+    if let Some(arr) = v.as_array() {
+        return arr
+            .iter()
+            .filter_map(|x| x.as_str())
+            .collect::<Vec<_>>()
+            .join("\n");
+    }
+    String::new()
+}
+
+async fn handler(
+    State(state): State<SharedState>,
+    Json(req): Json<CompletionsRequest>,
+) -> Result<Response> {
+    if !state.live().cfg.protocols.openai_completions {
+        return Err(GatewayError::InvalidRequest(
+            "openai_completions protocol is disabled".into(),
+        ));
+    }
+    let prompt = prompt_to_string(&req.prompt);
+    if prompt.is_empty() {
+        return Err(GatewayError::InvalidRequest(
+            "prompt must not be empty".into(),
+        ));
+    }
+
+    let canonical = ChatRequest {
+        routing: parse_routing(&req.model),
+        messages: vec![Message {
+            role: "user".into(),
+            content: prompt,
+            tool_calls: vec![],
+        }],
+        tools: vec![],
+        params: GenParams {
+            temperature: req.temperature,
+            max_tokens: req.max_tokens,
+            top_p: req.top_p,
+            stop: Vec::new(),
+            passthrough: Default::default(),
+        },
+        stream: req.stream,
+        meta: RequestMeta {
+            protocol: "openai_completions".into(),
+            ..Default::default()
+        },
+    };
+
+    if req.stream {
+        let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
+        let mut headers = cortiq_headers(&info);
+        headers.insert(
+            axum::http::header::CONTENT_TYPE,
+            axum::http::HeaderValue::from_static("text/event-stream"),
+        );
+        return Ok((
+            headers,
+            axum::body::Body::from_stream(chat_to_completions_sse(stream)),
+        )
+            .into_response());
+    }
+
+    let resp = state.pipeline.run(canonical, &state).await?;
+    let answer = resp
+        .choices
+        .first()
+        .map(|c| c.message.content.clone())
+        .unwrap_or_default();
+    let finish = resp
+        .choices
+        .first()
+        .map(|c| c.finish_reason.clone())
+        .unwrap_or_else(|| "stop".into());
+    let headers = cortiq_headers(&resp.cortiq);
+    let body = serde_json::json!({
+        "id": resp.id,
+        "object": "text_completion",
+        "created": std::time::SystemTime::now()
+            .duration_since(std::time::UNIX_EPOCH)
+            .unwrap_or_default()
+            .as_secs(),
+        "model": resp.model_used,
+        "choices": [{ "text": answer, "index": 0, "logprobs": serde_json::Value::Null, "finish_reason": finish }],
+        "usage": {
+            "prompt_tokens": resp.usage.prompt_tokens,
+            "completion_tokens": resp.usage.completion_tokens,
+            "total_tokens": resp.usage.total_tokens,
+        },
+    });
+    Ok((headers, Json(body)).into_response())
+}
+
+/// Translate chat `chat.completion.chunk` SSE into legacy `text_completion` SSE
+/// (`delta.content` → `text`).
+fn chat_to_completions_sse(
+    stream: crate::providers::ChatStream,
+) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
+    use futures::StreamExt;
+    async_stream::stream! {
+        let mut buf = String::new();
+        futures::pin_mut!(stream);
+        while let Some(item) = stream.next().await {
+            let bytes = match item { Ok(b) => b, Err(e) => { yield Err(e); break; } };
+            if let Ok(s) = std::str::from_utf8(&bytes) {
+                buf.push_str(s);
+            }
+            while let Some(idx) = buf.find("\n\n") {
+                let chunk: String = buf.drain(..idx + 2).collect();
+                for line in chunk.lines() {
+                    let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
+                    let data = data.trim();
+                    if data.is_empty() { continue; }
+                    if data == "[DONE]" {
+                        yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
+                        continue;
+                    }
+                    let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
+                    if let Some(u) = v.get("usage").filter(|u| !u.is_null()) {
+                        yield Ok(bytes::Bytes::from(format!(
+                            "data: {}\n\n",
+                            serde_json::json!({"object": "text_completion", "choices": [], "usage": u})
+                        )));
+                    }
+                    if let Some(t) = v["choices"][0]["delta"]["content"].as_str() {
+                        yield Ok(bytes::Bytes::from(format!(
+                            "data: {}\n\n",
+                            serde_json::json!({
+                                "object": "text_completion",
+                                "choices": [{"text": t, "index": 0, "finish_reason": null}]
+                            })
+                        )));
+                    }
+                }
+            }
+        }
+    }
+}
@@ -0,0 +1,52 @@
+//! Inbound OpenAI Embeddings adapter: `POST /v1/embeddings`.
+//! Resolves an embedding model (explicit id, or the first `kind = "embedding"`),
+//! calls the provider, and forwards the OpenAI-style response body.
+
+use crate::error::{GatewayError, Result};
+use crate::state::SharedState;
+use axum::{extract::State, response::IntoResponse, routing::post, Json, Router};
+use serde::Deserialize;
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/v1/embeddings", post(handler))
+}
+
+#[derive(Deserialize)]
+struct EmbeddingsRequest {
+    #[serde(default)]
+    model: String,
+    input: serde_json::Value,
+}
+
+async fn handler(
+    State(state): State<SharedState>,
+    Json(req): Json<EmbeddingsRequest>,
+) -> Result<impl IntoResponse> {
+    let live = state.live();
+    if !live.cfg.protocols.openai_embeddings {
+        return Err(GatewayError::InvalidRequest(
+            "openai_embeddings protocol is disabled".into(),
+        ));
+    }
+
+    // resolve the embedding model: explicit real id, else first kind == "embedding"
+    let model_id = if !req.model.is_empty()
+        && req.model != "cortiq-auto"
+        && live.registry.get(&req.model).is_some()
+    {
+        req.model.clone()
+    } else {
+        live.cfg
+            .models
+            .iter()
+            .find(|m| m.kind == "embedding")
+            .map(|m| m.id.clone())
+            .ok_or_else(|| GatewayError::InvalidRequest("no embedding model configured".into()))?
+    };
+
+    let provider = live.registry.get(&model_id).ok_or_else(|| {
+        GatewayError::UpstreamUnavailable(format!("embedding model '{model_id}' is not available"))
+    })?;
+    let body = provider.embed(req.input).await?;
+    Ok(Json(body))
+}
@@ -0,0 +1,32 @@
+//! Inbound OpenAI Models adapter: `GET /v1/models`.
+//! Lists the configured pool (plus the virtual `cortiq-auto`) for client discovery.
+
+use crate::error::{GatewayError, Result};
+use crate::state::SharedState;
+use axum::{extract::State, response::IntoResponse, routing::get, Json, Router};
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/v1/models", get(handler))
+}
+
+async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse> {
+    let live = state.live();
+    if !live.cfg.protocols.openai_models {
+        return Err(GatewayError::InvalidRequest(
+            "openai_models protocol is disabled".into(),
+        ));
+    }
+    let mut data = vec![serde_json::json!({
+        "id": "cortiq-auto",
+        "object": "model",
+        "owned_by": "cortiq",
+    })];
+    for m in &live.cfg.models {
+        data.push(serde_json::json!({
+            "id": m.id,
+            "object": "model",
+            "owned_by": m.provider,
+        }));
+    }
+    Ok(Json(serde_json::json!({ "object": "list", "data": data })))
+}
@@ -1,23 +1,25 @@
-//! Anthropic Messages provider (v0.2). Translates the canonical ChatRequest to
-//! `POST /v1/messages` and back. The `system` message is extracted into a separate field.
+//! Anthropic Messages provider. Translates the canonical [`ChatRequest`] to
+//! `POST {base_url}/v1/messages` and back. `system` messages are lifted into the
+//! top-level `system` field. Streaming translates Anthropic SSE events into the
+//! OpenAI wire format, so the gateway's unified SSE passthrough works uniformly.

 use super::{Caps, Provider};
 use crate::config::ModelCfg;
 use crate::error::{GatewayError, Result};
-use crate::model::{ChatRequest, ChatResponse};
+use crate::model::{ChatRequest, ChatResponse, Choice, Message, RouteInfo, Usage};
 use async_trait::async_trait;

+const ANTHROPIC_VERSION: &str = "2023-06-01";
+
 pub struct AnthropicProvider {
    id: String,
-    #[allow(dead_code)]
    base_url: String,
-    #[allow(dead_code)]
    model: String,
-    #[allow(dead_code)]
    api_key: Option<String>,
    caps: Caps,
    price_in: f64,
    price_out: f64,
+    http: reqwest::Client,
 }

 impl AnthropicProvider {
@@ -30,8 +32,61 @@ impl AnthropicProvider {
            caps: Caps::from_list(&cfg.caps),
            price_in: cfg.price_in,
            price_out: cfg.price_out,
+            http: reqwest::Client::new(),
        })
    }
+
+    /// Canonical request → Anthropic Messages request body.
+    fn to_wire(&self, req: &ChatRequest, stream: bool) -> serde_json::Value {
+        let mut system = String::new();
+        let mut messages = Vec::new();
+        for m in &req.messages {
+            if m.role == "system" {
+                if !system.is_empty() {
+                    system.push('\n');
+                }
+                system.push_str(&m.content);
+            } else {
+                let role = if m.role == "assistant" {
+                    "assistant"
+                } else {
+                    "user"
+                };
+                messages.push(serde_json::json!({ "role": role, "content": m.content }));
+            }
+        }
+        let mut body = serde_json::json!({
+            "model": self.model,
+            "messages": messages,
+            "max_tokens": req.params.max_tokens.unwrap_or(1024),
+            "stream": stream,
+        });
+        if !system.is_empty() {
+            body["system"] = serde_json::Value::String(system);
+        }
+        if let Some(t) = req.params.temperature {
+            body["temperature"] = t.into();
+        }
+        if let Some(p) = req.params.top_p {
+            body["top_p"] = p.into();
+        }
+        if !req.tools.is_empty() {
+            body["tools"] = serde_json::Value::Array(req.tools.clone());
+        }
+        body
+    }
+
+    fn request(&self, body: &serde_json::Value) -> reqwest::RequestBuilder {
+        let mut r = self
+            .http
+            .post(format!("{}/v1/messages", self.base_url))
+            .header("anthropic-version", ANTHROPIC_VERSION)
+            .json(body);
+        if let Some(key) = &self.api_key {
+            r = r.header("x-api-key", key);
+        }
+        r
+    }
 }

 #[async_trait]
@@ -46,12 +101,159 @@ impl Provider for AnthropicProvider {
        (prompt_tokens as f64 * self.price_in + completion_tokens as f64 * self.price_out)
            / 1_000_000.0
    }
-    async fn chat(&self, _req: ChatRequest) -> Result<ChatResponse> {
-        // TODO(v0.2): extract system from messages → "system" field; messages → content blocks;
-        // add x-api-key + anthropic-version headers; map response → canonical form.
-        // Returns an error (not a panic) so that failover/playground continue to work normally.
-        Err(GatewayError::UpstreamUnavailable(
-            "anthropic provider not yet implemented (planned v0.2)".to_string(),
-        ))
+
+    async fn chat(&self, req: ChatRequest) -> Result<ChatResponse> {
+        let resp = self
+            .request(&self.to_wire(&req, false))
+            .send()
+            .await
+            .map_err(|e| GatewayError::UpstreamUnavailable(e.to_string()))?;
+        if !resp.status().is_success() {
+            let status = resp.status();
+            let b = resp.text().await.unwrap_or_default();
+            return Err(GatewayError::UpstreamUnavailable(format!(
+                "provider {} returned HTTP {}: {}",
+                self.id, status, b
+            )));
+        }
+        let v: serde_json::Value = resp.json().await.map_err(|e| {
+            GatewayError::Internal(format!("failed to parse anthropic response: {e}"))
+        })?;
+        let content = v["content"]
+            .as_array()
+            .map(|blocks| {
+                blocks
+                    .iter()
+                    .filter_map(|b| b["text"].as_str())
+                    .collect::<Vec<_>>()
+                    .join("")
+            })
+            .unwrap_or_default();
+        let prompt_tokens = v["usage"]["input_tokens"].as_u64().unwrap_or(0) as u32;
+        let completion_tokens = v["usage"]["output_tokens"].as_u64().unwrap_or(0) as u32;
+        Ok(ChatResponse {
+            id: v["id"].as_str().unwrap_or("").to_string(),
+            model_used: v["model"].as_str().unwrap_or(&self.model).to_string(),
+            choices: vec![Choice {
+                index: 0,
+                message: Message {
+                    role: "assistant".into(),
+                    content,
+                    tool_calls: vec![],
+                },
+                finish_reason: v["stop_reason"].as_str().unwrap_or("stop").to_string(),
+            }],
+            usage: Usage {
+                prompt_tokens,
+                completion_tokens,
+                total_tokens: prompt_tokens + completion_tokens,
+            },
+            cortiq: RouteInfo {
+                task_label: String::new(),
+                complexity_score: 0.0,
+                complexity_tier: String::new(),
+                selected_model: self.id.clone(),
+                route_source: String::new(),
+                router_request_id: None,
+                cost_usd: 0.0,
+                failover: false,
+            },
+        })
+    }
+
+    async fn chat_stream(&self, req: ChatRequest) -> Result<super::ChatStream> {
+        let resp = self
+            .request(&self.to_wire(&req, true))
+            .send()
+            .await
+            .map_err(|e| GatewayError::UpstreamUnavailable(e.to_string()))?;
+        if !resp.status().is_success() {
+            let status = resp.status();
+            let b = resp.text().await.unwrap_or_default();
+            return Err(GatewayError::UpstreamUnavailable(format!(
+                "provider {} returned HTTP {}: {}",
+                self.id, status, b
+            )));
+        }
+        Ok(Box::pin(anthropic_to_openai_sse(resp.bytes_stream())))
+    }
+}
+
+/// Translate an Anthropic Messages SSE stream into OpenAI `chat.completion.chunk`
+/// SSE so the gateway's unified streaming path works for Anthropic too.
+fn anthropic_to_openai_sse(
+    upstream: impl futures::Stream<Item = reqwest::Result<bytes::Bytes>> + Send + 'static,
+) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
+    use futures::StreamExt;
+    async_stream::stream! {
+        futures::pin_mut!(upstream);
+        let mut buf = String::new();
+        let mut input_tokens = 0u64;
+        let mut role_sent = false;
+        let emit = |obj: serde_json::Value| bytes::Bytes::from(format!("data: {obj}\n\n"));
+
+        while let Some(chunk) = upstream.next().await {
+            let chunk = match chunk {
+                Ok(c) => c,
+                Err(e) => {
+                    yield Err(GatewayError::UpstreamUnavailable(e.to_string()));
+                    return;
+                }
+            };
+            if let Ok(s) = std::str::from_utf8(&chunk) {
+                buf.push_str(s);
+            }
+            while let Some(idx) = buf.find("\n\n") {
+                let evt: String = buf.drain(..idx + 2).collect();
+                for line in evt.lines() {
+                    let Some(data) = line.trim_start().strip_prefix("data:") else {
+                        continue;
+                    };
+                    let data = data.trim();
+                    if data.is_empty() {
+                        continue;
+                    }
+                    let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else {
+                        continue;
+                    };
+                    match v["type"].as_str().unwrap_or("") {
+                        "message_start" => {
+                            input_tokens =
+                                v["message"]["usage"]["input_tokens"].as_u64().unwrap_or(0);
+                            if !role_sent {
+                                role_sent = true;
+                                yield Ok(emit(serde_json::json!({
+                                    "choices": [{"index": 0, "delta": {"role": "assistant"}, "finish_reason": null}]
+                                })));
+                            }
+                        }
+                        "content_block_delta" => {
+                            if let Some(t) = v["delta"]["text"].as_str() {
+                                yield Ok(emit(serde_json::json!({
+                                    "choices": [{"index": 0, "delta": {"content": t}, "finish_reason": null}]
+                                })));
+                            }
+                        }
+                        "message_delta" => {
+                            let out = v["usage"]["output_tokens"].as_u64().unwrap_or(0);
+                            let stop = v["delta"]["stop_reason"].as_str().unwrap_or("stop").to_string();
+                            yield Ok(emit(serde_json::json!({
+                                "choices": [{"index": 0, "delta": {}, "finish_reason": stop}]
+                            })));
+                            yield Ok(emit(serde_json::json!({
+                                "choices": [],
+                                "usage": {
+                                    "prompt_tokens": input_tokens,
+                                    "completion_tokens": out,
+                                    "total_tokens": input_tokens + out
+                                }
+                            })));
+                        }
+                        _ => {}
+                    }
+                }
+            }
+        }
+        yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
    }
 }
@@ -7,8 +7,16 @@
 pub mod anthropic;
 pub mod openai;

+use crate::error::GatewayError;
 use crate::model::{ChatRequest, ChatResponse};
 use async_trait::async_trait;
+use bytes::Bytes;
+use futures::stream::BoxStream;
+
+/// A streaming chat response: raw Server-Sent-Events byte chunks in OpenAI wire
+/// format (`data: {json}\n\n` … `data: [DONE]\n\n`). The gateway forwards these
+/// to the client verbatim while tapping the final `usage` for statistics.
+pub type ChatStream = BoxStream<'static, crate::error::Result<Bytes>>;

 /// Model capabilities — checked at selection time (whether tools/vision are needed).
 #[derive(Clone, Debug, Default)]
@@ -41,6 +49,21 @@ pub trait Provider: Send + Sync {
    /// Non-streaming chat call.
    async fn chat(&self, req: ChatRequest) -> crate::error::Result<ChatResponse>;

-    // TODO(v0.1): async fn chat_stream(...) -> Result<ChatStream> (SSE).
-    // TODO(v0.1): async fn embed(...) -> Result<EmbedResponse>.
+    /// Streaming chat call (SSE). Default: not supported (override per provider).
+    async fn chat_stream(&self, _req: ChatRequest) -> crate::error::Result<ChatStream> {
+        Err(GatewayError::UpstreamUnavailable(format!(
+            "streaming is not supported by provider '{}'",
+            self.id()
+        )))
+    }
+
+    /// Embeddings call. `input` is the OpenAI `input` field (string or array of
+    /// strings); returns the provider's OpenAI-style embeddings response body.
+    /// Default: not supported (override per provider).
+    async fn embed(&self, _input: serde_json::Value) -> crate::error::Result<serde_json::Value> {
+        Err(GatewayError::UpstreamUnavailable(format!(
+            "embeddings are not supported by provider '{}'",
+            self.id()
+        )))
+    }
 }
@@ -154,4 +154,61 @@ impl Provider for OpenAiProvider {
            },
        })
    }
+
+    async fn chat_stream(&self, req: ChatRequest) -> Result<super::ChatStream> {
+        use futures::StreamExt;
+        let mut body = self.to_wire(&req);
+        body["stream"] = serde_json::Value::Bool(true);
+        // ask OpenAI-compatible servers to include token usage in the final chunk
+        body["stream_options"] = serde_json::json!({ "include_usage": true });
+        let mut r = self
+            .http
+            .post(format!("{}/chat/completions", self.base_url))
+            .json(&body);
+        if let Some(key) = &self.api_key {
+            r = r.bearer_auth(key);
+        }
+        let resp = r
+            .send()
+            .await
+            .map_err(|e| GatewayError::UpstreamUnavailable(e.to_string()))?;
+        if !resp.status().is_success() {
+            let status = resp.status();
+            let err_body = resp.text().await.unwrap_or_default();
+            return Err(GatewayError::UpstreamUnavailable(format!(
+                "provider {} returned HTTP {}: {}",
+                self.id, status, err_body
+            )));
+        }
+        let stream = resp
+            .bytes_stream()
+            .map(|res| res.map_err(|e| GatewayError::UpstreamUnavailable(e.to_string())));
+        Ok(Box::pin(stream))
+    }
+
+    async fn embed(&self, input: serde_json::Value) -> Result<serde_json::Value> {
+        let body = serde_json::json!({ "model": self.model, "input": input });
+        let mut r = self
+            .http
+            .post(format!("{}/embeddings", self.base_url))
+            .json(&body);
+        if let Some(key) = &self.api_key {
+            r = r.bearer_auth(key);
+        }
+        let resp = r
+            .send()
+            .await
+            .map_err(|e| GatewayError::UpstreamUnavailable(e.to_string()))?;
+        if !resp.status().is_success() {
+            let status = resp.status();
+            let err_body = resp.text().await.unwrap_or_default();
+            return Err(GatewayError::UpstreamUnavailable(format!(
+                "provider {} returned HTTP {}: {}",
+                self.id, status, err_body
+            )));
+        }
+        resp.json().await.map_err(|e| {
+            GatewayError::Internal(format!("failed to parse embeddings response: {e}"))
+        })
+    }
 }
@@ -1,7 +1,7 @@
 //! Model pool registry: builds providers from `[[models]]` and resolves
 //! `model_id → Arc<dyn Provider>`.

-use crate::config::Config;
+use crate::config::{Config, ModelCfg};
 use crate::providers::{anthropic::AnthropicProvider, openai::OpenAiProvider, Provider};
 use crate::secrets::SecretStore;
 use std::collections::HashMap;
@@ -15,9 +15,6 @@ impl Registry {
    pub fn from_config(cfg: &Config, secrets: &SecretStore) -> anyhow::Result<Self> {
        let mut by_id: HashMap<String, Arc<dyn Provider>> = HashMap::new();
        for m in &cfg.models {
-            if m.kind == "embedding" {
-                continue; // embedding models are handled by a separate branch (v0.1)
-            }
            // provider key: secret store → environment variable
            let api_key = m
                .api_key_env
@@ -30,9 +27,37 @@ impl Registry {
            };
            by_id.insert(m.id.clone(), provider);
        }
+
+        // Managed local CMF server: register it as an OpenAI-compatible provider
+        // so it can be routed to like any other backend — including when the
+        // router is unavailable or the gateway is in local-only mode.
+        if cfg.cmf.manage_server
+            && !cfg.cmf.local_model.trim().is_empty()
+            && !by_id.contains_key(&cfg.cmf.model_id)
+        {
+            let m = ModelCfg {
+                id: cfg.cmf.model_id.clone(),
+                provider: "openai".into(),
+                base_url: format!("http://{}:{}/v1", cfg.cmf.local_host, cfg.cmf.local_port),
+                model: "cortiq".into(),
+                cost_tier: "local".into(),
+                price_in: 0.0,
+                price_out: 0.0,
+                kind: "chat".into(),
+                api_key_env: None,
+                caps: Vec::new(),
+            };
+            by_id.insert(m.id.clone(), Arc::new(OpenAiProvider::new(&m, None)?));
+        }
        Ok(Self { by_id })
    }

+    /// Any model id, preferring a real registered one (used as a local-first
+    /// fallback target when routing cannot pick a specific model).
+    pub fn any_id(&self) -> Option<String> {
+        self.by_id.keys().next().cloned()
+    }
+
    pub fn get(&self, model_id: &str) -> Option<Arc<dyn Provider>> {
        self.by_id.get(model_id).cloned()
    }
@@ -4,16 +4,48 @@

 use crate::config::RouterCfg;
 use crate::model::RouteDecision;
+use crate::secrets::SecretStore;
+use std::sync::atomic::{AtomicU16, Ordering};
+use std::sync::Arc;
+
+/// Outcome of the most recent real `/v1/route` call, shared with the admin health
+/// endpoint so the panel can tell "no money / bad key" apart from "router down".
+/// 0 = ok (or no calls yet); [`STATUS_NETWORK`] / [`STATUS_TIMEOUT`] for transport
+/// failures; otherwise the upstream HTTP status code.
+pub type RouterLastStatus = Arc<AtomicU16>;
+
+pub const STATUS_NETWORK: u16 = 1;
+pub const STATUS_TIMEOUT: u16 = 2;
+
+/// Error kind for the admin panel.
+pub fn classify_status(code: u16) -> &'static str {
+    match code {
+        STATUS_NETWORK => "unreachable",
+        STATUS_TIMEOUT => "timeout",
+        401 | 403 => "auth",
+        402 => "payment",
+        429 => "quota",
+        _ => "error",
+    }
+}

 pub struct RouterClient {
    http: reqwest::Client,
    url: String,
    api_key: Option<String>,
    taxonomy_id: Option<String>,
+    last_status: RouterLastStatus,
 }

 impl RouterClient {
-    pub fn new(cfg: &RouterCfg) -> anyhow::Result<Self> {
+    /// The key is resolved through the secret store (admin panel value first,
+    /// then the environment variable) so a key pasted in the console works
+    /// without a restart.
+    pub fn new(
+        cfg: &RouterCfg,
+        secrets: &SecretStore,
+        last_status: RouterLastStatus,
+    ) -> anyhow::Result<Self> {
        let mut builder =
            reqwest::Client::builder().timeout(std::time::Duration::from_millis(cfg.timeout_ms));

@@ -27,12 +59,13 @@ impl RouterClient {
        let api_key = cfg
            .api_key_env
            .as_ref()
-            .and_then(|name| std::env::var(name).ok());
+            .and_then(|name| secrets.resolve(name));
        Ok(Self {
            http,
            url: cfg.url.trim_end_matches('/').to_string(),
            api_key,
            taxonomy_id: cfg.taxonomy_id.clone(),
+            last_status,
        })
    }

@@ -54,11 +87,22 @@ impl RouterClient {

        let resp = match req.send().await {
            Ok(r) => r,
-            Err(_) => return Ok(None), // degradation
+            Err(e) => {
+                let code = if e.is_timeout() {
+                    STATUS_TIMEOUT
+                } else {
+                    STATUS_NETWORK
+                };
+                self.last_status.store(code, Ordering::Relaxed);
+                return Ok(None); // degradation
+            }
        };
        if !resp.status().is_success() {
+            self.last_status
+                .store(resp.status().as_u16(), Ordering::Relaxed);
            return Ok(None);
        }
+        self.last_status.store(0, Ordering::Relaxed);
        let v: serde_json::Value = resp.json().await?;
        let d = &v["decision"];
        Ok(Some(RouteDecision {
@@ -5,9 +5,10 @@
 //! a restart: the admin API validates the new config, builds a new `Live`,
 //! atomically writes the TOML, and swaps the pointer.

+use crate::cache::SemanticCache;
 use crate::config::Config;
 use crate::registry::Registry;
-use crate::router_client::RouterClient;
+use crate::router_client::{RouterClient, RouterLastStatus};
 use crate::routing::RoutingTable;
 use crate::secrets::SecretStore;
 use crate::stats::Stats;
@@ -26,8 +27,12 @@ pub struct Live {
 }

 impl Live {
-    pub fn build(cfg: Config, secrets: &SecretStore) -> anyhow::Result<Self> {
-        let router = RouterClient::new(&cfg.router)?;
+    pub fn build(
+        cfg: Config,
+        secrets: &SecretStore,
+        router_status: RouterLastStatus,
+    ) -> anyhow::Result<Self> {
+        let router = RouterClient::new(&cfg.router, secrets, router_status)?;
        let registry = Registry::from_config(&cfg, secrets)?;
        let routing = RoutingTable::from_config(&cfg);
        Ok(Self {
@@ -37,14 +42,57 @@ impl Live {
            router,
        })
    }
+
+    /// Local-first fallback targets, in order: the managed local CMF model (if
+    /// configured), then the routing default, then any registered model — only
+    /// registered ids are returned. Used whenever the router is bypassed
+    /// (local-only / disabled) or unavailable, so CMF models serve with no router.
+    pub fn local_candidates(&self) -> Vec<String> {
+        let mut v = Vec::new();
+        if self.cfg.cmf.manage_server && !self.cfg.cmf.local_model.trim().is_empty() {
+            v.push(self.cfg.cmf.model_id.clone());
+        }
+        let def = self.routing.default_model().to_string();
+        if !def.is_empty() && !v.contains(&def) {
+            v.push(def);
+        }
+        v.retain(|id| self.registry.get(id).is_some());
+        if v.is_empty() {
+            if let Some(a) = self.registry.any_id() {
+                v.push(a);
+            }
+        }
+        v
+    }
 }

 pub struct AppState {
    pub live: ArcSwap<Live>,
    pub stats: Arc<Stats>,
+    pub promotion: Arc<crate::promotion::Promotion>,
+    pub imports: Arc<crate::import::JobStore>,
+    /// Managed local CMF model server (install/update/spawn lifecycle + status).
+    pub cmf: Arc<crate::cmf_runtime::CmfRuntime>,
+    pub cache: Arc<SemanticCache>,
    pub secrets: SecretStore,
    pub config_path: String,
    pub pipeline: crate::pipeline::Pipeline,
+    /// Outcome of the most recent router call (survives config reloads) —
+    /// lets the admin panel distinguish a bad/expired key from a down router.
+    pub router_status: RouterLastStatus,
+}
+
+/// Map the user-facing `[shadow]` config to the promotion-table tuning.
+fn promotion_cfg(s: &crate::config::ShadowCfg) -> crate::promotion::PromotionCfg {
+    crate::promotion::PromotionCfg {
+        enabled: s.enabled && !s.local_model_id.is_empty(),
+        window: s.window,
+        n_min: s.n_min,
+        promote_lb: s.promote_lb,
+        soak: s.soak,
+        file: if s.file.is_empty() { None } else { Some(s.file.clone()) },
+        ..Default::default()
+    }
 }

 impl std::ops::Deref for SharedState {
@@ -59,13 +107,23 @@ impl SharedState {
        let secrets_path = sibling(&config_path, "secrets.toml");
        let secrets = SecretStore::load(&secrets_path);
        let stats = Stats::new(&cfg.stats);
-        let live = Live::build(cfg, &secrets)?;
+        let promotion = crate::promotion::Promotion::new(promotion_cfg(&cfg.shadow));
+        let imports = crate::import::JobStore::new();
+        let cmf = crate::cmf_runtime::CmfRuntime::new();
+        let cache = SemanticCache::new(&cfg.cache);
+        let router_status = RouterLastStatus::default();
+        let live = Live::build(cfg, &secrets, router_status.clone())?;
        Ok(SharedState(Arc::new(AppState {
            live: ArcSwap::from_pointee(live),
            stats,
+            promotion,
+            imports,
+            cmf,
+            cache,
            secrets,
            config_path,
            pipeline: crate::pipeline::Pipeline::new(),
+            router_status,
        })))
    }

@@ -80,7 +138,7 @@ impl AppState {
    /// If any step fails, the active state and file are left unchanged.
    pub fn reload(&self, new_cfg: Config) -> anyhow::Result<()> {
        new_cfg.validate()?;
-        let live = Live::build(new_cfg.clone(), &self.secrets)?;
+        let live = Live::build(new_cfg.clone(), &self.secrets, self.router_status.clone())?;
        new_cfg.save(&self.config_path)?;
        self.live.store(Arc::new(live));
        Ok(())
@@ -90,7 +148,7 @@ impl AppState {
    /// (e.g. after a secret change — only provider keys changed).
    pub fn rebuild(&self) -> anyhow::Result<()> {
        let cfg = self.live.load_full().cfg.clone();
-        let live = Live::build(cfg, &self.secrets)?;
+        let live = Live::build(cfg, &self.secrets, self.router_status.clone())?;
        self.live.store(Arc::new(live));
        Ok(())
    }
@@ -91,3 +91,51 @@ export const api = {
  // playground
  test: (b) => j("POST", "/test", b),
 };
+
+// Streaming playground: POST /test/stream, calls onDelta(text) per content chunk,
+// returns the routing decision read from X-Cortiq-* response headers.
+export async function testStream(body, onDelta) {
+  const r = await fetch(BASE + "/test/stream", {
+    method: "POST",
+    headers: { Authorization: "Bearer " + getToken(), "content-type": "application/json" },
+    body: JSON.stringify(body),
+  });
+  if (r.status === 401) {
+    if (onUnauthorized) onUnauthorized();
+    throw new Error("unauthorized");
+  }
+  if (!r.ok || !r.body) {
+    throw new Error((await r.text()) || r.statusText);
+  }
+  const reader = r.body.getReader();
+  const dec = new TextDecoder();
+  let buf = "";
+  for (;;) {
+    const { done, value } = await reader.read();
+    if (done) break;
+    buf += dec.decode(value, { stream: true });
+    let idx;
+    while ((idx = buf.indexOf("\n\n")) >= 0) {
+      const line = buf.slice(0, idx).split("\n").find((l) => l.startsWith("data:"));
+      buf = buf.slice(idx + 2);
+      if (!line) continue;
+      const data = line.slice(5).trim();
+      if (!data || data === "[DONE]") continue;
+      try {
+        const d = JSON.parse(data).choices?.[0]?.delta?.content;
+        if (d) onDelta(d);
+      } catch {
+        /* ignore malformed chunk */
+      }
+    }
+  }
+  const h = r.headers;
+  return {
+    task_label: h.get("x-cortiq-task-label") || "",
+    tier: h.get("x-cortiq-complexity-tier") || "",
+    score: parseFloat(h.get("x-cortiq-complexity-score") || "0"),
+    selected_model: h.get("x-cortiq-selected-model") || "",
+    route_source: h.get("x-cortiq-route-source") || "",
+    cost_usd: parseFloat(h.get("x-cortiq-cost-usd") || "0"),
+  };
+}
@@ -200,6 +200,7 @@ export default {
  "pg.temp": "Temperatur",
  "pg.maxTokens": "Max. Tokens",
  "pg.send": "Senden",
+  "pg.stream": "Streamen",
  "pg.sending": "Wird gesendet…",
  "pg.answer": "Antwort",
  "pg.empty": "Antwort und Routing-Entscheidung erscheinen hier.",
@@ -253,4 +254,15 @@ export default {
  "toast.deleted": "Gelöscht",
  "toast.probing": "Prüfe…",
  "toast.imported": "Konfiguration importiert",
+  "dash.cache": "Semantischer Cache",
+  "dash.cache.hitRate": "Trefferquote",
+  "dash.cache.hits": "Treffer",
+  "dash.cache.saved": "Gespart",
+  "dash.cache.disabled": "Semantischer Cache ist deaktiviert — in den Einstellungen aktivieren.",
+  "settings.cache": "Semantischer Cache",
+  "settings.cache.enabled": "Semantischen Cache aktivieren",
+  "settings.cache.threshold": "Ähnlichkeitsschwelle",
+  "settings.cache.ttl": "Eintrags-TTL",
+  "settings.cache.embed": "Embedding-Modell",
+  "settings.cache.note": "Cache-Einstellungen werden nach einem Neustart wirksam.",
 };
@@ -28,12 +28,14 @@ export default {
  // nav
  "nav.dashboard": "Dashboard",
  "nav.models": "Models",
+  "nav.import": "Import model",
  "nav.routing": "Routing",
  "nav.protocols": "Protocols",
  "nav.keys": "API keys",
  "nav.playground": "Playground",
  "nav.settings": "Settings",
  "nav.foot": "Gateway · v{v}",
+  "nav.billing": "Router key & billing",

  // topbar / status
  "status.router": "Router",
@@ -41,6 +43,7 @@ export default {
  "status.online": "online",
  "status.offline": "offline",
  "status.checking": "checking…",
+  "status.noKey": "no key",
  "theme.toggle": "Theme",
  "logout": "Log out",

@@ -89,6 +92,7 @@ export default {
  "dash.health.title": "Health",
  "dash.health.reachable": "reachable",
  "dash.health.unreachable": "unreachable",
+  "dash.health.getKey": "Get a router key",

  // models
  "models.title": "Models",
@@ -201,6 +205,7 @@ export default {
  "pg.temp": "Temperature",
  "pg.maxTokens": "Max tokens",
  "pg.send": "Send",
+  "pg.stream": "Stream",
  "pg.sending": "Sending…",
  "pg.answer": "Answer",
  "pg.empty": "The answer and routing decision will appear here.",
@@ -224,7 +229,37 @@ export default {
  "settings.router.keyEnv": "Router key env name",
  "settings.router.timeout": "Timeout (ms)",
  "settings.router.verifyTls": "Verify TLS certificate",
+  "settings.router.enabled": "Use the router (off = local models only)",
+  "settings.cmf": "Local models (CMF)",
+  "settings.cmf.localOnly": "Use only local models",
+  "settings.cmf.manage": "Run a local model server",
+  "settings.cmf.autoInstall": "Auto-install cortiq (crates.io)",
+  "settings.cmf.autoUpdate": "Auto-update cortiq",
+  "settings.cmf.model": "Local .cmf model path",
+  "settings.cmf.port": "Local server port",
+  "settings.cmf.modelId": "Model id in the pool",
+  "settings.cmf.note": "Serves a local .cmf via cortiq (github.com/infosave2007/cmf). Works even when the router is unavailable.",
+  "settings.cmf.threads": "Worker threads",
+  "settings.cmf.gpu": "Use GPU (Metal / wgpu)",
+  "settings.cmf.speedNote": "Threads parallelise the decode matvecs (~3x faster; 0 = runtime default). GPU offload mainly helps prefill of long prompts — decode is memory-bandwidth-bound, so on Apple Silicon it is roughly neutral.",
  "settings.router.taxonomy": "Taxonomy id",
+  "settings.router.getKey": "Get or top up a router key at",
+  "settings.router.secret": "Router key (value)",
+  "settings.router.saveKey": "Save key",
+  "settings.router.keySaved": "Router key saved",
+  "settings.router.keyEmpty": "Paste the key first",
+  "settings.router.secretHelp": "Paste the key from your api.allaigate.com email, then Save key. This is NOT a “Keys” value — those are for clients calling your gateway.",
+  "settings.router.keyStoredMask": "•••••••••••• — key saved (paste again to replace)",
+  "settings.router.test": "Test router",
+  "router.probe.ok": "Router OK ({ms} ms)",
+  "router.probe.noKey": "Router key is not set",
+  "router.probe.auth": "Key rejected — invalid or revoked",
+  "router.probe.payment": "Payment required — plan expired or quota exhausted",
+  "router.probe.quota": "Rate limit exceeded",
+  "router.probe.timeout": "Router timed out",
+  "router.probe.unreachable": "Router unreachable",
+  "router.probe.error": "Router error",
+  "router.usage": "Usage",
  "settings.route": "Routing input",
  "settings.route.strategy": "Text strategy",
  "settings.route.maxChars": "Max chars",
@@ -254,4 +289,57 @@ export default {
  "toast.deleted": "Deleted",
  "toast.probing": "Probing…",
  "toast.imported": "Config imported",
+  "dash.cache": "Semantic cache",
+  "dash.cache.hitRate": "Hit rate",
+  "dash.cache.hits": "Hits",
+  "dash.cache.saved": "Saved",
+  "dash.cache.disabled": "Semantic cache is disabled — enable it in Settings.",
+  "settings.cache": "Semantic cache",
+  "settings.cache.enabled": "Enable semantic cache",
+  "settings.cache.threshold": "Similarity threshold",
+  "settings.cache.ttl": "Entry TTL",
+  "settings.cache.embed": "Embedding model",
+  "settings.cache.note": "Cache settings take effect after a restart.",
+
+  // import (HuggingFace → CMF)
+  "import.subtitle": "Search HuggingFace, convert to a local .cmf, and serve it.",
+  "import.searchPlaceholder": "Search HuggingFace models (e.g. qwen3, llama)…",
+  "import.searching": "Searching…",
+  "import.noResults": "No models found.",
+  "import.configure": "Configure conversion",
+  "import.outName": "Output name",
+  "import.outNameHint": "The .cmf file name (letters, digits, - and _).",
+  "import.quant": "Quantization",
+  "import.convert": "Convert & load",
+  "import.converting": "Converting",
+  "import.started": "Conversion started",
+  "import.failed": "Conversion failed — see the log.",
+  "import.register": "Register as local model",
+  "import.registered": "Registered as '{id}'. Run cortiq-server to serve it.",
+  "import.jobs": "Conversions",
+  "import.cancel": "Cancel",
+  "import.cancelling": "Cancelling — cleaning up partial output…",
+  "import.delete": "Delete",
+  "import.deleteConfirm": "Delete this model and its converted .cmf file from disk?",
+  "import.deleted": "Deleted with its files",
+  "import.state.running": "running",
+  "import.state.done": "done",
+  "import.state.error": "error",
+  "import.state.cancelled": "cancelled",
+  "import.q.q8_2f": "8-bit two-field (𝒲×θ) — recommended: best quality/size.",
+  "import.q.q8_row": "8-bit per-row — simple, robust.",
+  "import.q.q4": "4-bit block — smallest, lower quality.",
+  "import.q.vbit": "Variable 3–8 bit — per-expert bit budgets.",
+  "import.q.f16": "16-bit — no quantization.",
+  "import.q.f32": "32-bit — full precision (large file).",
+  "import.adv.title": "Advanced settings",
+  "import.adv.auto": "auto",
+  "import.adv.linearCore": "Linear core",
+  "import.adv.linearHint": "How linear-attention layers are carried/folded.",
+  "import.adv.nphase": "vmf_phase phases/head",
+  "import.adv.vbitShape": "VBIT budget shape",
+  "import.adv.meanBits": "VBIT mean bits",
+  "import.adv.shard": "Shard max (GB)",
+  "import.adv.noShard": "no sharding",
+  "import.adv.skipMtp": "skip MTP head",
 };
@@ -200,6 +200,7 @@ export default {
  "pg.temp": "Temperatura",
  "pg.maxTokens": "Tokens máx.",
  "pg.send": "Enviar",
+  "pg.stream": "Streaming",
  "pg.sending": "Enviando…",
  "pg.answer": "Respuesta",
  "pg.empty": "La respuesta y la decisión de enrutamiento aparecerán aquí.",
@@ -253,4 +254,15 @@ export default {
  "toast.deleted": "Eliminado",
  "toast.probing": "Sondeando…",
  "toast.imported": "Configuración importada",
+  "dash.cache": "Caché semántico",
+  "dash.cache.hitRate": "Aciertos",
+  "dash.cache.hits": "Aciertos",
+  "dash.cache.saved": "Ahorrado",
+  "dash.cache.disabled": "El caché semántico está desactivado — actívalo en Ajustes.",
+  "settings.cache": "Caché semántico",
+  "settings.cache.enabled": "Activar caché semántico",
+  "settings.cache.threshold": "Umbral de similitud",
+  "settings.cache.ttl": "TTL de entrada",
+  "settings.cache.embed": "Modelo de embeddings",
+  "settings.cache.note": "Los ajustes del caché se aplican tras reiniciar.",
 };
@@ -200,6 +200,7 @@ export default {
  "pg.temp": "Température",
  "pg.maxTokens": "Tokens max",
  "pg.send": "Envoyer",
+  "pg.stream": "Streaming",
  "pg.sending": "Envoi…",
  "pg.answer": "Réponse",
  "pg.empty": "La réponse et la décision de routage s'afficheront ici.",
@@ -253,4 +254,15 @@ export default {
  "toast.deleted": "Supprimé",
  "toast.probing": "Sondage…",
  "toast.imported": "Config importée",
+  "dash.cache": "Cache sémantique",
+  "dash.cache.hitRate": "Taux de succès",
+  "dash.cache.hits": "Succès",
+  "dash.cache.saved": "Économisé",
+  "dash.cache.disabled": "Le cache sémantique est désactivé — activez-le dans les Paramètres.",
+  "settings.cache": "Cache sémantique",
+  "settings.cache.enabled": "Activer le cache sémantique",
+  "settings.cache.threshold": "Seuil de similarité",
+  "settings.cache.ttl": "TTL d'entrée",
+  "settings.cache.embed": "Modèle d'embeddings",
+  "settings.cache.note": "Les paramètres du cache prennent effet après un redémarrage.",
 };
@@ -27,12 +27,14 @@ export default {
  // nav
  "nav.dashboard": "Дашборд",
  "nav.models": "Модели",
+  "nav.import": "Импорт модели",
  "nav.routing": "Роутинг",
  "nav.protocols": "Протоколы",
  "nav.keys": "API-ключи",
  "nav.playground": "Playground",
  "nav.settings": "Настройки",
  "nav.foot": "Шлюз · v{v}",
+  "nav.billing": "Ключ роутера и оплата",

  // topbar / status
  "status.router": "Роутер",
@@ -40,6 +42,7 @@ export default {
  "status.online": "в сети",
  "status.offline": "недоступен",
  "status.checking": "проверка…",
+  "status.noKey": "нет ключа",
  "theme.toggle": "Тема",
  "logout": "Выйти",

@@ -88,6 +91,7 @@ export default {
  "dash.health.title": "Состояние",
  "dash.health.reachable": "доступен",
  "dash.health.unreachable": "недоступен",
+  "dash.health.getKey": "Получить ключ роутера",

  // models
  "models.title": "Модели",
@@ -200,6 +204,7 @@ export default {
  "pg.temp": "Температура",
  "pg.maxTokens": "Макс. токены",
  "pg.send": "Отправить",
+  "pg.stream": "Стриминг",
  "pg.sending": "Отправка…",
  "pg.answer": "Ответ",
  "pg.empty": "Ответ и решение роутера появятся здесь.",
@@ -223,7 +228,37 @@ export default {
  "settings.router.keyEnv": "Имя env-переменной с ключом роутера",
  "settings.router.timeout": "Таймаут (мс)",
  "settings.router.verifyTls": "Проверять TLS-сертификат",
+  "settings.router.enabled": "Использовать роутер (выкл = только локальные модели)",
+  "settings.cmf": "Локальные модели (CMF)",
+  "settings.cmf.localOnly": "Использовать только локальные модели",
+  "settings.cmf.manage": "Запускать локальный сервер моделей",
+  "settings.cmf.autoInstall": "Автоустановка cortiq (crates.io)",
+  "settings.cmf.autoUpdate": "Автообновление cortiq",
+  "settings.cmf.model": "Путь к локальной .cmf-модели",
+  "settings.cmf.port": "Порт локального сервера",
+  "settings.cmf.modelId": "ID модели в пуле",
+  "settings.cmf.note": "Обслуживает локальный .cmf через cortiq (github.com/infosave2007/cmf). Работает даже когда роутер недоступен.",
+  "settings.cmf.threads": "Рабочие потоки",
+  "settings.cmf.gpu": "Использовать GPU (Metal / wgpu)",
+  "settings.cmf.speedNote": "Потоки распараллеливают декод-матвеки (~×3; 0 = по умолчанию рантайма). GPU-оффлоад помогает в основном на prefill длинных промптов — декод упирается в пропускную способность памяти, поэтому на Apple Silicon почти нейтрален.",
  "settings.router.taxonomy": "ID таксономии",
+  "settings.router.getKey": "Получить или пополнить ключ роутера:",
+  "settings.router.secret": "Ключ роутера (значение)",
+  "settings.router.saveKey": "Сохранить ключ",
+  "settings.router.keySaved": "Ключ роутера сохранён",
+  "settings.router.keyEmpty": "Сначала вставьте ключ",
+  "settings.router.secretHelp": "Вставьте ключ из письма с api.allaigate.com и нажмите «Сохранить ключ». Это НЕ ключ из раздела «Ключи» — те выдаются клиентам для доступа к вашему шлюзу.",
+  "settings.router.keyStoredMask": "•••••••••••• — ключ сохранён (вставьте снова, чтобы заменить)",
+  "settings.router.test": "Проверить роутер",
+  "router.probe.ok": "Роутер работает ({ms} мс)",
+  "router.probe.noKey": "Ключ роутера не задан",
+  "router.probe.auth": "Ключ отклонён — недействителен или отозван",
+  "router.probe.payment": "Требуется оплата — тариф истёк или квота исчерпана",
+  "router.probe.quota": "Превышен лимит запросов",
+  "router.probe.timeout": "Роутер не ответил вовремя",
+  "router.probe.unreachable": "Роутер недоступен",
+  "router.probe.error": "Ошибка роутера",
+  "router.usage": "Использование",
  "settings.route": "Входные данные роутинга",
  "settings.route.strategy": "Стратегия текста",
  "settings.route.maxChars": "Макс. символов",
@@ -253,4 +288,57 @@ export default {
  "toast.deleted": "Удалено",
  "toast.probing": "Проверка…",
  "toast.imported": "Конфигурация импортирована",
+  "dash.cache": "Семантический кэш",
+  "dash.cache.hitRate": "Попаданий",
+  "dash.cache.hits": "Попадания",
+  "dash.cache.saved": "Сэкономлено",
+  "dash.cache.disabled": "Семантический кэш выключен — включите в Настройках.",
+  "settings.cache": "Семантический кэш",
+  "settings.cache.enabled": "Включить семантический кэш",
+  "settings.cache.threshold": "Порог сходства",
+  "settings.cache.ttl": "TTL записи",
+  "settings.cache.embed": "Модель эмбеддингов",
+  "settings.cache.note": "Настройки кэша применяются после перезапуска.",
+
+  // импорт (HuggingFace → CMF)
+  "import.subtitle": "Найдите модель на HuggingFace, сконвертируйте в локальный .cmf и подключите.",
+  "import.searchPlaceholder": "Поиск моделей на HuggingFace (напр. qwen3, llama)…",
+  "import.searching": "Ищу…",
+  "import.noResults": "Модели не найдены.",
+  "import.configure": "Настройка конвертации",
+  "import.outName": "Имя файла",
+  "import.outNameHint": "Имя .cmf (буквы, цифры, - и _).",
+  "import.quant": "Квантизация",
+  "import.convert": "Сконвертировать и загрузить",
+  "import.converting": "Конвертация",
+  "import.started": "Конвертация запущена",
+  "import.failed": "Конвертация не удалась — смотрите лог.",
+  "import.register": "Подключить как локальную модель",
+  "import.registered": "Подключено как '{id}'. Запустите cortiq-server для обслуживания.",
+  "import.jobs": "Конвертации",
+  "import.cancel": "Отменить",
+  "import.cancelling": "Отмена — удаляю недописанный файл…",
+  "import.delete": "Удалить",
+  "import.deleteConfirm": "Удалить эту модель и сконвертированный .cmf-файл с диска?",
+  "import.deleted": "Удалено вместе с файлами",
+  "import.state.running": "идёт",
+  "import.state.done": "готово",
+  "import.state.error": "ошибка",
+  "import.state.cancelled": "отменено",
+  "import.q.q8_2f": "8-бит двухполевой (𝒲×θ) — рекомендуется: лучшее качество/размер.",
+  "import.q.q8_row": "8-бит построчно — просто, надёжно.",
+  "import.q.q4": "4-бит блочно — компактнее, ниже качество.",
+  "import.q.vbit": "Переменная 3–8 бит — per-expert бюджеты.",
+  "import.q.f16": "16-бит — без квантования.",
+  "import.q.f32": "32-бит — полная точность (большой файл).",
+  "import.adv.title": "Расширенные настройки",
+  "import.adv.auto": "авто",
+  "import.adv.linearCore": "Линейное ядро",
+  "import.adv.linearHint": "Как переносятся/сворачиваются линейные слои внимания.",
+  "import.adv.nphase": "vmf_phase фаз/голову",
+  "import.adv.vbitShape": "Форма бюджета VBIT",
+  "import.adv.meanBits": "VBIT средних бит",
+  "import.adv.shard": "Шард макс (ГБ)",
+  "import.adv.noShard": "без шардинга",
+  "import.adv.skipMtp": "пропустить MTP-голову",
 };
@@ -200,6 +200,7 @@ export default {
  "pg.temp": "Sıcaklık",
  "pg.maxTokens": "Maks. token",
  "pg.send": "Gönder",
+  "pg.stream": "Akış",
  "pg.sending": "Gönderiliyor…",
  "pg.answer": "Yanıt",
  "pg.empty": "Yanıt ve yönlendirme kararı burada görünecek.",
@@ -253,4 +254,15 @@ export default {
  "toast.deleted": "Silindi",
  "toast.probing": "Sınanıyor…",
  "toast.imported": "Yapılandırma içe aktarıldı",
+  "dash.cache": "Anlamsal önbellek",
+  "dash.cache.hitRate": "İsabet oranı",
+  "dash.cache.hits": "İsabet",
+  "dash.cache.saved": "Tasarruf",
+  "dash.cache.disabled": "Anlamsal önbellek kapalı — Ayarlar'dan etkinleştirin.",
+  "settings.cache": "Anlamsal önbellek",
+  "settings.cache.enabled": "Anlamsal önbelleği etkinleştir",
+  "settings.cache.threshold": "Benzerlik eşiği",
+  "settings.cache.ttl": "Giriş TTL",
+  "settings.cache.embed": "Gömme modeli",
+  "settings.cache.note": "Anlamsal önbellek ayarları yeniden başlatmadan sonra geçerli olur.",
 };
@@ -200,6 +200,7 @@ export default {
  "pg.temp": "温度",
  "pg.maxTokens": "最大 Token 数",
  "pg.send": "发送",
+  "pg.stream": "流式",
  "pg.sending": "发送中…",
  "pg.answer": "回答",
  "pg.empty": "回答和路由决策将在此显示。",
@@ -253,4 +254,15 @@ export default {
  "toast.deleted": "已删除",
  "toast.probing": "探测中…",
  "toast.imported": "配置已导入",
+  "dash.cache": "语义缓存",
+  "dash.cache.hitRate": "命中率",
+  "dash.cache.hits": "命中",
+  "dash.cache.saved": "已节省",
+  "dash.cache.disabled": "语义缓存已禁用 — 在设置中启用。",
+  "settings.cache": "语义缓存",
+  "settings.cache.enabled": "启用语义缓存",
+  "settings.cache.threshold": "相似度阈值",
+  "settings.cache.ttl": "条目 TTL",
+  "settings.cache.embed": "嵌入模型",
+  "settings.cache.note": "缓存设置将在重启后生效。",
 };
@@ -108,6 +108,22 @@ export async function renderDashboard() {
        barList(breakdown, num)
      ),

+      // semantic cache
+      stats.cache && stats.cache.enabled
+        ? h(
+            "div",
+            { class: "card" },
+            h("div", { class: "card-head" }, h("h3", {}, t("dash.cache"))),
+            h(
+              "div",
+              { class: "kpis" },
+              kpi(pct(stats.cache.hit_rate), t("dash.cache.hitRate")),
+              kpi(num(stats.cache.hits), t("dash.cache.hits")),
+              kpi(money(stats.cache.saved_usd), t("dash.cache.saved"))
+            )
+          )
+        : null,
+
      // recent
      h(
        "div",
@@ -1,7 +1,7 @@
 // Playground — send a prompt through the live pipeline, inspect routing decision.
-import { h, mount, money, ms, num } from "../ui.js";
+import { h, mount, money, ms } from "../ui.js";
 import { t } from "../i18n.js";
-import { api } from "../api.js";
+import { api, testStream } from "../api.js";
 import { appState } from "../app.js";

 export async function renderPlayground() {
@@ -16,6 +16,7 @@ export async function renderPlayground() {
  const promptIn = h("textarea", { placeholder: t("pg.promptPlaceholder"), rows: 5 });
  const tempIn = h("input", { type: "number", step: "0.1", value: 0.7 });
  const maxTokIn = h("input", { type: "number", value: 512 });
+  const streamCb = h("input", { type: "checkbox", checked: true });
  const profileSel = h("select", {}, ...(meta.profiles || []).map((p) => h("option", { value: p }, p)));
  const modelSel = h("select", {}, ...ids.map((id) => h("option", { value: id }, id)));

@@ -62,19 +63,43 @@ export async function renderPlayground() {
    sendBtn.disabled = true;
    sendBtn.textContent = t("pg.sending");
    mount(out, h("div", { class: "flex" }, h("span", { class: "spinner" }), " ", t("pg.sending")));
+    const body = {
+      model,
+      messages,
+      temperature: parseFloat(tempIn.value),
+      max_tokens: parseInt(maxTokIn.value) || null,
+    };
    try {
-      const r = await api.test({
-        model,
-        messages,
-        temperature: parseFloat(tempIn.value),
-        max_tokens: parseInt(maxTokIn.value) || null,
-      });
-      if (r.ok) {
-        mount(out, r.answer || h("span", { class: "muted" }, "—"));
-        renderDecision(r);
+      if (streamCb.checked) {
+        const started = performance.now();
+        let acc = "";
+        out.textContent = "";
+        const info = await testStream(body, (d) => {
+          acc += d;
+          out.textContent = acc;
+        });
+        out.textContent = acc || "—";
+        renderDecision({
+          cortiq: {
+            task_label: info.task_label,
+            complexity: { score: info.score, tier: info.tier },
+            selected_model: info.selected_model,
+            route_source: info.route_source,
+            cost_usd: info.cost_usd,
+            failover: false,
+          },
+          usage: {},
+          latency_ms: Math.round(performance.now() - started),
+        });
      } else {
-        mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
-        mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
+        const r = await api.test(body);
+        if (r.ok) {
+          mount(out, r.answer || h("span", { class: "muted" }, "—"));
+          renderDecision(r);
+        } else {
+          mount(out, h("div", { class: "callout" }, h("b", {}, t("pg.error") + ": "), r.error || ""));
+          mount(decision, h("div", { class: "card-head" }, h("h3", {}, t("pg.decision"))), h("div", { class: "empty" }, "—"));
+        }
      }
    } catch (e) {
      mount(out, h("div", { class: "callout" }, String(e.message || e)));
@@ -126,7 +151,12 @@ export async function renderPlayground() {
          h("label", { class: "field" }, h("span", {}, t("pg.mode")), modeSeg),
          modeExtra,
          h("div", { class: "row" }, h("label", { class: "field" }, h("span", {}, t("pg.temp")), tempIn), h("label", { class: "field" }, h("span", {}, t("pg.maxTokens")), maxTokIn)),
-          h("div", { class: "flex", style: "justify-content:flex-end" }, sendBtn)
+          h(
+            "div",
+            { class: "flex", style: "justify-content:space-between" },
+            h("label", { class: "check", style: "margin:0" }, streamCb, t("pg.stream")),
+            sendBtn
+          )
        ),
        h("div", { class: "card" }, h("div", { class: "card-head" }, h("h3", {}, t("pg.answer"))), out)
      ),
@@ -2,7 +2,7 @@
 import { h, mount, toast } from "../ui.js";
 import { t } from "../i18n.js";
 import { api } from "../api.js";
-import { appState } from "../app.js";
+import { appState, SITE_URL } from "../app.js";

 function field(label, control, hint) {
  return h("label", { class: "field" }, h("span", {}, label), control, hint ? h("div", { class: "hint" }, hint) : null);
@@ -12,9 +12,45 @@ function check(label, checked) {
  return { node: h("label", { class: "check" }, cb, label), cb };
 }

+// Flatten router usage (nested account/usage objects) into readable, wrapping
+// key: value rows — a raw JSON.stringify dump overflows the card horizontally.
+function usageRows(obj, prefix) {
+  const rows = [];
+  for (const [k, v] of Object.entries(obj || {})) {
+    const key = prefix ? prefix + "." + k : k;
+    if (v && typeof v === "object" && !Array.isArray(v)) {
+      rows.push(...usageRows(v, key));
+    } else {
+      rows.push(
+        h("div", { class: "mono kv" },
+          h("span", { class: "kv-k" }, key + ": "),
+          h("span", { class: "kv-v" }, Array.isArray(v) ? JSON.stringify(v) : String(v)))
+      );
+    }
+  }
+  return rows;
+}
+
+// Probe status → i18n key suffix (see /admin/api/router/probe).
+function probeMsgKey(status) {
+  const map = {
+    ok: "ok",
+    no_key: "noKey",
+    auth: "auth",
+    payment: "payment",
+    quota: "quota",
+    timeout: "timeout",
+    unreachable: "unreachable",
+  };
+  return "router.probe." + (map[status] || "error");
+}
+
+// Statuses where the fix is on the site (get / pay for a key).
+const PROBE_SITE_STATUSES = ["no_key", "auth", "payment", "quota"];
+
 export async function renderSettings() {
  const meta = appState.meta || (await api.meta());
-  const s = await api.getSettings();
+  const [s, secretsData] = await Promise.all([api.getSettings(), api.listSecrets()]);
  const root = h("div");

  const listenIn = h("input", { value: s.listen || "" });
@@ -24,6 +60,101 @@ export async function renderSettings() {
  const rTimeout = h("input", { type: "number", value: s.router?.timeout_ms ?? 800 });
  const rTax = h("input", { value: s.router?.taxonomy_id || "" });
  const rVerify = check(t("settings.router.verifyTls"), s.router?.verify_tls);
+  const rEnabled = check(t("settings.router.enabled"), s.router?.enabled !== false);
+  // router key value (write-only; stored in the secret store, like model keys)
+  const rKeySource =
+    (secretsData.secrets || []).find((x) => x.name === (s.router?.api_key_env || "CORTIQ_ROUTER_KEY"))?.source ||
+    "missing";
+  const keyStored = rKeySource === "store" || rKeySource === "env";
+  // When a key is already saved, show a masked "saved" hint instead of an empty
+  // field (an empty field reads as "nothing is set").
+  const rSecret = h("input", {
+    type: "password",
+    placeholder: keyStored ? t("settings.router.keyStoredMask") : t("models.form.secretPlaceholder"),
+  });
+  const rKeyBadge = h(
+    "span",
+    { class: "badge " + (rKeySource === "store" ? "store" : rKeySource === "env" ? "env" : "bad") },
+    rKeySource === "store"
+      ? t("models.form.secretStored")
+      : rKeySource === "env"
+        ? t("models.form.secretEnv")
+        : t("models.form.secretMissing")
+  );
+  const rSecretHint = h(
+    "div",
+    {},
+    h(
+      "div",
+      {},
+      rKeyBadge,
+      " · ",
+      t("settings.router.getKey") + " ",
+      h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, "api.allaigate.com")
+    ),
+    h("div", { class: "muted", style: "margin-top:4px" }, t("settings.router.secretHelp"))
+  );
+  // The key lives in the secret store under this env name (config default: CORTIQ_ROUTER_KEY).
+  const keyName = () => rKeyEnv.value.trim() || "CORTIQ_ROUTER_KEY";
+  // Persist just the pasted router key (independent of the big Save at page top).
+  async function saveKey() {
+    const v = rSecret.value.trim();
+    if (!v) return false;
+    if (!rKeyEnv.value.trim()) rKeyEnv.value = "CORTIQ_ROUTER_KEY";
+    await api.setSecret(keyName(), v);
+    rSecret.value = "";
+    rSecret.placeholder = t("settings.router.keyStoredMask"); // show "saved", not empty
+    rKeyBadge.textContent = t("models.form.secretStored");
+    rKeyBadge.className = "badge store";
+    return true;
+  }
+  const saveKeyBtn = h("button", { class: "btn sm" }, "✓ " + t("settings.router.saveKey"));
+  saveKeyBtn.addEventListener("click", async () => {
+    saveKeyBtn.disabled = true;
+    try {
+      if (await saveKey()) toast(t("settings.router.keySaved"), "good");
+      else toast(t("settings.router.keyEmpty"), "warn");
+    } catch (e) {
+      toast(String(e.message || e), "bad");
+    }
+    saveKeyBtn.disabled = false;
+  });
+  // "Test router" — real /v1/route call: tells key problems apart from a down router
+  const probeBtn = h("button", { class: "btn" }, t("settings.router.test"));
+  const probeOut = h("div", { class: "hint", style: "margin-top:8px" });
+  probeBtn.addEventListener("click", async () => {
+    probeBtn.disabled = true;
+    mount(probeOut, h("span", { class: "spinner" }));
+    try {
+      // If a key was just pasted but not saved, persist it first so the test uses it.
+      if (rSecret.value.trim()) {
+        try {
+          await saveKey();
+        } catch (_) {
+          /* fall through — probe will report no_key/auth */
+        }
+      }
+      const r = await api.probeRouter();
+      const parts = [t(probeMsgKey(r.status), { ms: r.latency_ms ?? "—" })];
+      if (r.message) parts.push(" — " + r.message);
+      const line = h("span", {}, ...parts);
+      const out = [line];
+      if (PROBE_SITE_STATUSES.includes(r.status)) {
+        out.push(" · ", h("a", { href: SITE_URL, target: "_blank", rel: "noopener" }, t("dash.health.getKey") + " ↗"));
+      }
+      if (r.usage && typeof r.usage === "object") {
+        out.push(
+          h("div", { class: "usage-box", style: "margin-top:4px" },
+            h("b", {}, t("router.usage")), ...usageRows(r.usage))
+        );
+      }
+      mount(probeOut, h("div", {}, h("span", { class: "badge " + (r.ok ? "ok" : "bad") }, r.ok ? "OK" : "×"), " ", ...out));
+      toast(t(probeMsgKey(r.status), { ms: r.latency_ms ?? "—" }), r.ok ? "good" : "bad");
+    } catch (e) {
+      mount(probeOut, String(e.message || e));
+    }
+    probeBtn.disabled = false;
+  });
  // route
  const strategySel = h("select", {}, ...(meta.text_strategies || []).map((x) => h("option", { value: x, selected: x === s.route?.text_strategy }, x)));
  const maxChars = h("input", { type: "number", value: s.route?.max_chars ?? 4000 });
@@ -42,13 +173,38 @@ export async function renderSettings() {
  const statsEnabled = check(t("settings.stats.enabled"), s.stats?.enabled);
  const statsFile = h("input", { value: s.stats?.file || "" });
  const statsRet = h("input", { value: s.stats?.retention || "7d" });
+  // cache
+  const modelsData = await api.listModels();
+  const embedModels = (modelsData.models || []).filter((m) => m.kind === "embedding").map((m) => m.id);
+  const cacheEnabled = check(t("settings.cache.enabled"), s.cache?.enabled);
+  const cacheThresh = h("input", { type: "number", step: "0.01", value: s.cache?.threshold ?? 0.92 });
+  const cacheTtlIn = h("input", { value: s.cache?.ttl || "1h" });
+  const cacheEmbedSel = h(
+    "select",
+    {},
+    h("option", { value: "" }, "auto"),
+    ...embedModels.map((id) => h("option", { value: id, selected: id === s.cache?.embed_model }, id))
+  );
+  // local models (CMF) — run a local .cmf server and/or force local-only routing
+  const cmfLocalOnly = check(t("settings.cmf.localOnly"), s.cmf?.local_only);
+  const cmfManage = check(t("settings.cmf.manage"), s.cmf?.manage_server);
+  const cmfAutoInstall = check(t("settings.cmf.autoInstall"), s.cmf?.auto_install);
+  const cmfAutoUpdate = check(t("settings.cmf.autoUpdate"), s.cmf?.auto_update);
+  const cmfModel = h("input", { value: s.cmf?.local_model || "", placeholder: "models/model.cmf" });
+  const cmfPort = h("input", { type: "number", value: s.cmf?.local_port ?? 8081 });
+  const cmfModelId = h("input", { value: s.cmf?.model_id || "cmf-local" });
+  const cmfThreads = h("input", { type: "number", min: 0, value: s.cmf?.threads ?? 8 });
+  const cmfGpu = check(t("settings.cmf.gpu"), s.cmf?.gpu);

  const saveBtn = h("button", { class: "btn primary" }, t("common.save"));
  saveBtn.addEventListener("click", async () => {
    saveBtn.disabled = true;
+    // a pasted router key needs an env name to live under — default it
+    if (rSecret.value.trim() && !rKeyEnv.value.trim()) rKeyEnv.value = "CORTIQ_ROUTER_KEY";
    const patch = {
      listen: listenIn.value.trim(),
      router: {
+        enabled: rEnabled.cb.checked,
        url: rUrl.value.trim(),
        api_key_env: rKeyEnv.value.trim() || null,
        timeout_ms: parseInt(rTimeout.value) || 800,
@@ -71,13 +227,40 @@ export async function renderSettings() {
        retention: statsRet.value.trim() || "7d",
        ring_size: s.stats?.ring_size ?? 500,
      },
+      cache: {
+        enabled: cacheEnabled.cb.checked,
+        threshold: parseFloat(cacheThresh.value) || 0.92,
+        ttl: cacheTtlIn.value.trim() || "1h",
+        max_entries: s.cache?.max_entries ?? 1000,
+        embed_model: cacheEmbedSel.value || null,
+      },
+      cmf: {
+        ...(s.cmf || {}),
+        local_only: cmfLocalOnly.cb.checked,
+        manage_server: cmfManage.cb.checked,
+        auto_install: cmfAutoInstall.cb.checked,
+        auto_update: cmfAutoUpdate.cb.checked,
+        local_model: cmfModel.value.trim(),
+        local_port: parseInt(cmfPort.value) || 8081,
+        model_id: cmfModelId.value.trim() || "cmf-local",
+        threads: parseInt(cmfThreads.value) || 0,
+        gpu: cmfGpu.cb.checked,
+      },
    };
    // serde: drop null optionals
    if (!patch.router.api_key_env) delete patch.router.api_key_env;
    if (!patch.router.taxonomy_id) delete patch.router.taxonomy_id;
    if (!patch.telemetry.otlp_endpoint_env) delete patch.telemetry.otlp_endpoint_env;
+    if (!patch.cache.embed_model) delete patch.cache.embed_model;
    try {
      const r = await api.putSettings(patch);
+      const secretVal = rSecret.value.trim();
+      if (secretVal) {
+        await api.setSecret(rKeyEnv.value.trim(), secretVal);
+        rSecret.value = "";
+        rKeyBadge.textContent = t("models.form.secretStored");
+        rKeyBadge.className = "badge store";
+      }
      toast(r.needs_restart ? t("settings.needsRestart") : t("toast.saved"), r.needs_restart ? "warn" : "good");
    } catch (e) {
      toast(String(e.message || e), "bad");
@@ -130,8 +313,13 @@ export async function renderSettings() {
        h("div", { class: "card-head" }, h("h3", {}, t("settings.router"))),
        field(t("settings.router.url"), rUrl),
        field(t("settings.router.keyEnv"), rKeyEnv),
+        field(t("settings.router.secret"), h("div", { class: "key-row" }, rSecret, saveKeyBtn), rSecretHint),
        h("div", { class: "row" }, field(t("settings.router.timeout"), rTimeout), field(t("settings.router.taxonomy"), rTax)),
-        rVerify.node
+        rEnabled.node,
+        rVerify.node,
+        h("div", { class: "divider" }),
+        h("div", { class: "flex" }, probeBtn),
+        probeOut
      ),
      h(
        "div",
@@ -170,6 +358,28 @@ export async function renderSettings() {
        statsEnabled.node,
        field(t("settings.stats.file"), statsFile),
        field(t("settings.stats.retention"), statsRet)
+      ),
+      h(
+        "div",
+        { class: "card" },
+        h("div", { class: "card-head" }, h("h3", {}, t("settings.cache"))),
+        cacheEnabled.node,
+        h("div", { class: "row" }, field(t("settings.cache.threshold"), cacheThresh), field(t("settings.cache.ttl"), cacheTtlIn)),
+        field(t("settings.cache.embed"), cacheEmbedSel),
+        h("div", { class: "hint" }, t("settings.cache.note"))
+      ),
+      h(
+        "div",
+        { class: "card" },
+        h("div", { class: "card-head" }, h("h3", {}, t("settings.cmf"))),
+        cmfLocalOnly.node,
+        cmfManage.node,
+        h("div", { class: "row" }, cmfAutoInstall.node, cmfAutoUpdate.node),
+        field(t("settings.cmf.model"), cmfModel),
+        h("div", { class: "row" }, field(t("settings.cmf.port"), cmfPort), field(t("settings.cmf.modelId"), cmfModelId)),
+        h("div", { class: "row" }, field(t("settings.cmf.threads"), cmfThreads), cmfGpu.node),
+        h("div", { class: "hint" }, t("settings.cmf.note")),
+        h("div", { class: "hint" }, t("settings.cmf.speedNote"))
      )
    ),
    h(
Author	SHA1	Message	Date
infosave2007	0f89cdaa07	chore(release): v0.2.4 Docker / publish (push) Canceled after 0s Details Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details	2026-07-08 17:39:43 +03:00
infosave2007	d211eb6150	feat: fresh install starts with an empty model pool - config.rs: validate() no longer errors on zero models — an empty pool is allowed (warns instead), so the admin console comes up on a fresh install and the user adds models in /admin -> Models (or points [cmf] at a local .cmf). - gateway.example.toml: ships an EMPTY pool (no seed cmf-local/local-embed, empty routing), so `cp example -> gateway.toml && run` shows an empty list — what a new user expects.	2026-07-08 17:39:43 +03:00
infosave2007	04580196ad	fix: add promotion + router-status modules referenced by the CMF integration Docker / publish (push) Canceled after 0s Details Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details The v0.2.3 CMF-integration commit referenced crate::promotion (main.rs), router_client::RouterLastStatus (state.rs) and router_client::classify_status (admin), but those definitions were not part of the commit, so a clean build failed (E0583/E0432/E0425). Add the two source files; cargo build --release now compiles.	2026-07-08 16:59:58 +03:00
infosave2007	7a3d6a7bb4	chore(release): v0.2.3	2026-07-08 16:39:30 +03:00
infosave2007	07d011263d	config: example ships a local CMF model, drop cloud placeholders The example config pool now shows one local CMF model (served by the managed cortiq server) instead of local-qwen/gpt-4o-mini/claude-opus placeholders, with routing pointing at it, and a documented [cmf] section (manage_server, threads, gpu, ...). Add cloud/local providers by copying the [[models]] shape.	2026-07-08 16:39:30 +03:00
infosave2007	0040efc948	admin: surface CMF worker threads + GPU toggle in Settings Expose the two speed knobs in the /admin Settings panel instead of TOML-only: - [cmf].gpu (new) -> exports CMF_GPU=1 to the managed cortiq serve (Metal/wgpu). On Apple Silicon decode is memory-bandwidth bound so this is ~neutral for generation (helps prefill); off by default. - [cmf].threads (added earlier) now has a UI number input. - config.rs: CmfCfg.gpu bool (serde default false). - cmf_runtime.rs: spawn_serve sets CMF_GPU=1 when gpu is on. - admin/mod.rs: put_settings flags needs_restart on threads/gpu change (the server is spawned once at startup with those env vars). - settings.js: threads input + gpu checkbox + save payload + speed hint. - i18n en/ru: threads/gpu/speedNote keys (other locales fall back to en, as the whole cmf section already does there).	2026-07-08 16:18:31 +03:00
infosave2007	16a3e209b6	cmf: parallelise the managed local server (CMF_THREADS) The gateway spawned `cortiq serve` with no thread hint, so the runtime ran the decode loop single-threaded (~0.25 tok/s on a 15B CPU model). Export CMF_THREADS from a new [cmf].threads config field (default 8) in spawn_serve. Measured: 0.25 -> 0.81 tok/s (~3.2x) on qwopus-vbit7 (15B). Scaling is sublinear (GatedDeltaNet state recurrence is serial per token, Amdahl-bound); a real jump needs the GPU path.	2026-07-08 16:18:31 +03:00
infosave2007	c19edee7bb	feat(cmf): integrate CMF as a managed local model server Run models in the CMF (.cmf) format locally, via the native `cortiq` CLI from crates.io — no Python and no external model server. - cmf_runtime: install/update `cortiq-cli` from crates.io, spawn and health-check `cortiq serve` as a managed local backend. - import: model-import jobs shell out to the native `cortiq convert` (HF repo -> .cmf, quant mapped Q8_2F->q8_2f / Q8_ROW->q8 / Q4->q4); the Python converter is used only for advanced/exotic options. - pipeline: local-first routing — serve from the local CMF model when the router is disabled/unreachable, with a "local only" hard switch. - registry/state: register the local CMF server as an OpenAI provider and track local candidates. - config: data-dir resolution + CmfCfg (auto-install/update, managed server, host/port/model) and relaxed validation for the managed model. - admin + settings UI (en/ru): "Local models (CMF)" settings card.	2026-07-08 16:18:31 +03:00
infosave2007andClaude Opus 4.8	7af90a5eef	docs: add docker-compose quickstart (gateway + Ollama) and GHCR badge - docker-compose.yml + config/gateway.compose.toml (routes to a local Ollama service) - README (EN+RU): ghcr badge + Docker Compose section Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 13:40:24 +03:00
infosave2007andClaude Opus 4.8	8d7e5fe653	chore(release): v0.2.2 Docker / publish (push) Canceled after 0s Details Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details Benchmarks (latency + accuracy), router docs + LAUNCH1 promo, Docker image on ghcr. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 11:28:56 +03:00
infosave2007andClaude Opus 4.8	2605eccb64	feat: Docker image + ghcr publishing - multi-stage Dockerfile (non-root, embeds SPA, default config, ~160MB) + .dockerignore - .github/workflows/docker.yml: push to ghcr.io/infosave2007/cortiq-gateway (master -> :edge/:sha amd64; tags -> :X.Y.Z/:latest, linux/amd64+arm64) - README (EN+RU): docker run instructions; CHANGELOG - verified locally: image builds, container serves /healthz + /admin Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 11:23:09 +03:00
infosave2007andClaude Opus 4.8	1317b8024b	docs: document the allaigate router + launch promo - docs/ROUTER.md + docs/ROUTER.ru.md: what the semantic router does, hosted vs self-hosted, config, API contract, pricing, and the LAUNCH1 $1 Starter promo - README (EN+RU): "The router (allaigate)" section with the promo callout Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 10:35:29 +03:00
infosave2007andClaude Opus 4.8	9242806795	docs: add task-type accuracy benchmark (router vs keyword heuristic) - bench/tasks.jsonl (37 natural-language prompts, 7 task types) + bench/accuracy.py - measured on the live allaigate router: 100% (37/37) vs keyword heuristic 32% - BENCHMARKS.md accuracy section; README (EN+RU) perf+accuracy tables and test matrix - run.sh runs accuracy when CORTIQ_ROUTER_KEY is set Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 07:45:16 +03:00
infosave2007andClaude Opus 4.8	6714c4f556	docs: add test/verification matrix to README (EN+RU) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 18:31:13 +03:00
infosave2007andClaude Opus 4.8	a5966dd479	docs: add latency benchmark vs LiteLLM/Portkey - bench/ reproducible harness: instant mock backend + Apache Bench (ab -k -r) - measures gateway overhead in passthrough mode (apples-to-apples) - BENCHMARKS.md: methodology, caveats, results (cortiq ~57k rps p99 1ms vs Portkey ~5.8k p99 9ms vs LiteLLM 4-workers ~1.2k p99 59ms) - README (EN+RU): Performance section + highlight Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 18:11:26 +03:00
infosave2007andClaude Opus 4.8	e015555dff	feat: finish remaining inbound protocols (models, completions, native /route) Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details - GET /v1/models: pool + virtual cortiq-auto - POST /v1/completions (legacy): non-stream + streaming text_completion - POST /route: native passthrough returning the routing decision - all inbound protocol adapters now implemented; admin meta + README/CHANGELOG updated - bump to 0.2.1 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 17:42:45 +03:00
infosave2007andClaude Opus 4.8	f00bf8355a	chore(release): v0.2.0 Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details Streaming (SSE), Anthropic provider/inbound, embeddings, semantic cache, MCP server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 17:17:04 +03:00
infosave2007andClaude Opus 4.8	ddca3a8363	feat: MCP (Model Context Protocol) server - POST /mcp (JSON-RPC 2.0): initialize, ping, tools/list, tools/call; notifications -> 202 - tools: cortiq_route (classify -> routing decision) and cortiq_chat (route -> answer) - gated by protocols.mcp; admin meta marks it implemented - docs: README/README.ru protocol tables, highlights and status updated for streaming, Anthropic, embeddings, semantic cache and MCP Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 17:12:55 +03:00
infosave2007andClaude Opus 4.8	a67564bc24	feat: semantic (embedding-based) response cache - cache.rs: cosine-similarity store (threshold, TTL, ring buffer), partitioned by routing signature; tracks hit rate and model cost saved - pipeline: embed routing text, look up before the model call (route_source=cache on hit), insert successful responses; stats record cache hits - config [cache] + admin settings card; dashboard shows hit-rate and savings - embedding model resolved from the pool (or first kind="embedding") Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 17:04:53 +03:00
infosave2007andClaude Opus 4.8	7dbc39c6a6	feat: Anthropic provider/inbound + embeddings - Anthropic outbound provider: /v1/messages, system extraction, x-api-key + anthropic-version; streaming translates Anthropic SSE -> unified OpenAI wire format - Anthropic inbound adapter (POST /v1/messages): non-streaming JSON + streaming SSE (OpenAI -> Anthropic event translation) - Embeddings: POST /v1/embeddings inbound + Provider::embed(); embedding models resolvable from the pool (registry no longer skips them) - protocols mounted always, gated on live flags; admin meta marks them implemented Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 16:50:22 +03:00
infosave2007andClaude Opus 4.8	b6fdf23aa6	feat: streaming (SSE) end-to-end - Provider::chat_stream; OpenAI provider streams via stream_options.include_usage - pipeline.run_stream: shared route/select, failover before first byte - /v1/chat/completions forwards SSE verbatim with X-Cortiq-* headers; usage tapped for stats - admin /test/stream + live streaming in Playground (stream toggle, 7 langs) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 16:37:35 +03:00
				`@@ -0,0 +1 @@`
				`{"model":"bench","messages":[{"role":"user","content":"ping"}],"max_tokens":1}`