chore(release): v0.2.2

Benchmarks (latency + accuracy), router docs + LAUNCH1 promo, Docker image on ghcr. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat: Docker image + ghcr publishing
2026-07-01 11:28:56 +03:00 · 2026-07-01 11:23:09 +03:00 · 2026-07-01 10:35:29 +03:00 · 2026-07-01 07:45:16 +03:00 · 2026-06-30 18:31:13 +03:00 · 2026-06-30 18:11:26 +03:00
22 changed files with 1119 additions and 16 deletions
@@ -0,0 +1,10 @@
+target
+.git
+.github
+docs
+bench
+config/gateway.toml
+config/secrets.toml
+config/stats.jsonl
+*.log
+.DS_Store
@@ -0,0 +1,47 @@
+name: Docker
+
+# Publishes container images to GitHub Container Registry (ghcr.io).
+#   master push  -> :edge, :sha  (amd64, fast)
+#   tag v*       -> :X.Y.Z, :X.Y, :latest, :sha  (linux/amd64 + linux/arm64)
+on:
+  push:
+    branches: [master]
+    tags: ["v*"]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  packages: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository }}
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
+            type=raw,value=edge,enable={{is_default_branch}}
+            type=sha,format=short
+      - uses: docker/build-push-action@v6
+        with:
+          context: .
+          # multi-arch only for release tags (arm64 under emulation is slow)
+          platforms: ${{ startsWith(github.ref, 'refs/tags/v') && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
@@ -0,0 +1,109 @@
+# Gateway latency benchmark
+
+How much latency does the **gateway itself** add? For an LLM proxy the model call
+dominates wall-clock time, but the gateway's own overhead is what differs between
+products — and it compounds across the many calls an agent makes. This benchmark
+isolates that overhead and compares Cortiq Gateway with **LiteLLM** and **Portkey**.
+
+> Reproduce on your own hardware: `bash bench/run.sh` (see [Methodology](#methodology)).
+
+## TL;DR
+
+Every gateway proxies the **same instant mock backend** in passthrough mode, under
+identical load (`ab -k -r -c 20 -n 5000`). Latency is in milliseconds.
+
+| Gateway | Throughput (req/s) | p50 | p90 | p99 | mean |
+|---|--:|--:|--:|--:|--:|
+| **Cortiq Gateway** (Rust) | **~57,300** | 0 ms | 1 ms | **1 ms** | 0.35 ms |
+| Portkey Gateway (Node) | ~5,800 | 3 ms | 5 ms | 9 ms | 3.5 ms |
+| LiteLLM (Python, 4 workers) | ~1,200 | 9 ms | 37 ms | 59 ms | 16.6 ms |
+| _mock-direct (no gateway)_ | _~29,000_ | _1 ms_ | _1 ms_ | _3 ms_ | _0.7 ms_ |
+
+On this machine Cortiq adds **sub-millisecond p99** overhead and sustains **~10×**
+the throughput of Portkey and **~47×** of LiteLLM. The gap is architectural: a Rust
+/ Tokio gateway uses every core in one process, with no interpreter or GC in the hot
+path.
+
+> Absolute numbers depend on hardware; the **ratios** between gateways are the point.
+> Run `bench/run.sh` to get numbers for your environment.
+
+## Accuracy: task-type routing
+
+Speed is only half the story. Cortiq's other edge is **routing by task type** — sending
+each prompt to the right model — via the allaigate semantic classifier. LiteLLM and
+Portkey have **no semantic task router**: you either pin a model, write brittle rules,
+or pay for an extra LLM classification call.
+
+On a 37-prompt, 7-task-type set of **natural-language prompts** (no explicit trigger
+words like "translate" / "summarize" — [`bench/tasks.jsonl`](bench/tasks.jsonl)):
+
+| Classifier | Correct | Accuracy |
+|---|--:|--:|
+| **allaigate semantic router** | 37 / 37 | **100%** |
+| keyword heuristic (DIY without a classifier) | 12 / 37 | 32% |
+
+A keyword matcher looks good on prompts that literally say "translate this" — but real
+users don't. On natural phrasing (`"'Guten Tag' — what does that mean in English?"`,
+`"boil this down to a few key points"`, `"which companies are named in this report?"`)
+surface matching collapses while the semantic router still routes correctly.
+
+- **Methodology.** [`bench/accuracy.py`](bench/accuracy.py) sends each prompt to the live
+  allaigate router (`taxonomy = data-assistant`) and compares the predicted `task_label`
+  to the ground truth; a keyword heuristic is scored on the same set as a lower bound for
+  "route without a classifier."
+- **Caveats.** Small, hand-labeled set; phrasing is deliberately keyword-light to test
+  semantic understanding vs surface matching. Expand `tasks.jsonl` and re-run. allaigate's
+  own published figure is **94% at separating close task types** (a harder eval than this one).
+- **Reproduce.** `CORTIQ_ROUTER_KEY=cortiq_… python3 bench/accuracy.py`
+
+## Why this matters
+
+Agentic workloads make **many** gateway calls per task (plan → sub-tasks → tool calls
+→ synthesis). Overhead that looks small per call (tens of ms) multiplies across a run
+and caps the throughput a single instance can serve. Lower, more predictable tail
+latency (p99) is the developer-facing differentiator when the model time is equal.
+
+## Methodology
+
+- **Isolate gateway overhead.** A tiny, dependency-free Node mock ([`bench/mock.js`](bench/mock.js))
+  returns a fixed OpenAI-compatible completion **instantly**. So measured latency is
+  the gateway's own cost, not model time. Every gateway proxies this same mock.
+- **Apples-to-apples.** Each gateway runs in **passthrough / pinned** mode (no routing
+  or model-selection logic) so we compare raw proxy cost. Cortiq's intelligent routing
+  (`cortiq-auto`) adds one classifier round-trip — that's a feature, measured separately,
+  not folded into this overhead number.
+- **Load.** Apache Bench, keep-alive, tolerant of length variance:
+  `ab -k -r -c 20 -n 5000 -p bench/body.json -T application/json`.
+- **Fair to each runtime.** Cortiq = one process (Tokio, multi-core). Portkey = one Node
+  process (event loop). LiteLLM = **4 uvicorn workers** (its recommended multi-process
+  setup; a single Python worker is GIL-bound to one core and scores ~860 req/s).
+- **Same box, localhost**, warm processes, results are representative of one run.
+
+### Caveats (read me)
+
+- This measures **gateway overhead only**. Real requests are dominated by model latency,
+  which is identical regardless of gateway — but overhead and tail latency are not.
+- Localhost + mock backend removes network/model variance on purpose. Your absolute
+  numbers will differ; reproduce with `bench/run.sh`.
+- `ab` reports LiteLLM "Failed requests" as a non-zero count — these are **response
+  content-length variations, not HTTP errors** (all responses are `200`, verified).
+- Tested versions: Cortiq Gateway (this repo, `--release`), LiteLLM proxy via
+  `uvicorn litellm.proxy.proxy_server:app --workers 4`, Portkey `@portkey-ai/gateway`,
+  Apache Bench. Hardware: Apple Silicon, macOS.
+
+## Reproduce
+
+```bash
+# prerequisites: node, ab (Apache Bench), a release build of the gateway
+cargo build --release
+
+# optional competitors (the harness auto-detects them):
+pip install 'litellm[proxy]' uvicorn
+npm install -g @portkey-ai/gateway
+
+# run (tune load via env)
+CONC=20 REQ=5000 bash bench/run.sh
+```
+
+The harness ([`bench/run.sh`](bench/run.sh)) starts the mock, then benchmarks the mock
+directly and each available gateway, printing a throughput / p50 / p90 / p99 table.
@@ -3,6 +3,32 @@
 All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/).

+## [0.2.2] - 2026-07-01
+
+### Added
+- **Latency benchmark** vs LiteLLM and Portkey: a reproducible harness (`bench/`,
+  instant mock backend + Apache Bench) and `BENCHMARKS.md`. Measured gateway overhead:
+  Cortiq ~57k req/s / p99 1 ms vs Portkey ~5.8k / p99 9 ms vs LiteLLM ~1.2k / p99 59 ms.
+- **Accuracy benchmark**: task-type routing on natural-language prompts (`bench/accuracy.py`,
+  `bench/tasks.jsonl`) — allaigate semantic router 100% (37/37) vs a keyword heuristic 32%
+  (competitors have no semantic task router).
+- **Router docs** (`docs/ROUTER.md` + `.ru.md`): what the allaigate/cortiq-router does, how
+  to point the gateway at it, API contract, pricing, and the current `LAUNCH1` $1 promo.
+- **Docker image**: multi-stage `Dockerfile` (non-root, ~160 MB) + `docker.yml` workflow
+  publishing to `ghcr.io/infosave2007/cortiq-gateway` (`:latest`, `:X.Y.Z`, `:edge`;
+  linux/amd64 + arm64 on release tags). `docker run` works out of the box.
+
+## [0.2.1] - 2026-06-30
+
+### Added
+- **OpenAI Models** (`GET /v1/models`): lists the pool + the virtual `cortiq-auto`.
+- **OpenAI Completions (legacy)** (`POST /v1/completions`): non-streaming and streaming
+  (`text_completion`), routed through the same pipeline.
+- **Native passthrough** (`POST /route`): returns the routing decision (task, complexity,
+  candidate models) without calling a model.
+
+All inbound protocol adapters are now implemented.
+
 ## [0.2.0] - 2026-06-30

 ### Added
@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"

 [[package]]
 name = "cortiq-gateway"
-version = "0.2.0"
+version = "0.2.2"
 dependencies = [
 "anyhow",
 "arc-swap",
@@ -1,6 +1,6 @@
 [package]
 name = "cortiq-gateway"
-version = "0.2.0"
+version = "0.2.2"
 edition = "2021"
 rust-version = "1.75"
 license = "Apache-2.0"
@@ -0,0 +1,25 @@
+# ---- build stage ------------------------------------------------------------
+FROM rust:1-slim AS builder
+WORKDIR /app
+# the admin SPA (web/) is embedded at compile time via rust-embed, so it must be present
+COPY Cargo.toml Cargo.lock ./
+COPY src ./src
+COPY web ./web
+RUN cargo build --release
+
+# ---- runtime stage ----------------------------------------------------------
+FROM debian:bookworm-slim
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends ca-certificates \
+    && rm -rf /var/lib/apt/lists/* \
+    && useradd -r -u 10001 cortiq \
+    && mkdir -p /app/config
+COPY --from=builder /app/target/release/cortiq-gateway /usr/local/bin/cortiq-gateway
+# ship the example as the default config so `docker run` works out of the box
+COPY config/gateway.example.toml /app/config/gateway.toml
+RUN chown -R 10001:10001 /app
+WORKDIR /app
+USER cortiq
+EXPOSE 9000
+ENTRYPOINT ["cortiq-gateway"]
+CMD ["--config", "/app/config/gateway.toml"]
@@ -35,6 +35,7 @@ One OpenAI-compatible endpoint → automatic model selection from your pool

 ## ✨ Highlights

+- **Fast.** Rust/Tokio: **sub-millisecond p99** gateway overhead — ~10× the throughput of Portkey and ~47× of LiteLLM in a like-for-like proxy benchmark ([details](BENCHMARKS.md)).
 - **Drop-in OpenAI API + streaming.** Point any OpenAI client at the gateway and send `model: "cortiq-auto"`; `stream: true` is fully supported (SSE).
 - **Multi-protocol.** OpenAI Chat & Embeddings, **Anthropic Messages** (in/out, streaming), and an **MCP** server (`POST /mcp`) exposing routing as tools for agent orchestrators.
 - **Intelligent routing.** `complexity.tier` → ordered model pool (`low → local`, `high → cloud`), with fallback and graceful degradation when the router is down.
@@ -50,6 +51,11 @@ One OpenAI-compatible endpoint → automatic model selection from your pool
 ## 📦 Install

 ```bash
+# Docker (GitHub Container Registry)
+docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
+  -v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
+# → admin console at http://localhost:9000/admin
+
 # from crates.io
 cargo install cortiq-gateway

@@ -59,6 +65,9 @@ cd cortiq-gateway
 cargo build --release   # ./target/release/cortiq-gateway
 ```

+The image ships a default config and runs out of the box; mount `/app/config` (with a
+`gateway.toml`) to use your own. Tags: `:latest`, `:X.Y.Z`, `:edge` (linux/amd64 + arm64).
+
 ---

 ## 🖥️ Admin console
@@ -130,6 +139,20 @@ print(resp.choices[0].message.content)

 ---

+## 🧭 The router (allaigate)
+
+`cortiq-auto` routing is powered by a **semantic router** — the classifier that reads a
+prompt and returns its task type + complexity. Use the hosted
+**[allaigate](https://api.allaigate.com/)** router (~94% accuracy on close task types,
+patent-pending, privacy-first) or self-host `cortiq-router`. Pinned models need no router.
+
+> 🎉 **Launch promo:** grab a **$1 Starter key** with code **`LAUNCH1`** at
+> **[api.allaigate.com](https://api.allaigate.com/)** (limited activations).
+
+Setup, pricing, API contract and how to verify accuracy: **[docs/ROUTER.md](docs/ROUTER.md)**.
+
+---
+
 ## Request flow

 1. Client → `POST /v1/chat/completions` (or Anthropic/MCP) with `model: "cortiq-auto"`.
@@ -153,9 +176,9 @@ Each adapter is toggled in the config — take only what you need.
 | OpenAI Embeddings | `POST /v1/embeddings` | ✅ implemented |
 | Anthropic Messages | `POST /v1/messages` | ✅ implemented (+ streaming) |
 | MCP (Model Context Protocol) | `POST /mcp` (JSON-RPC) | ✅ implemented |
-| OpenAI Completions (legacy) | `POST /v1/completions` | planned |
-| OpenAI Models | `GET /v1/models` | planned |
-| Native passthrough | `POST /route` | planned |
+| OpenAI Completions (legacy) | `POST /v1/completions` | ✅ implemented (+ streaming) |
+| OpenAI Models | `GET /v1/models` | ✅ implemented |
+| Native passthrough | `POST /route` | ✅ implemented |

 ## Supported providers (outbound)

@@ -219,6 +242,27 @@ Details — [docs/PROTOCOLS.md](https://github.com/infosave2007/cortiq-gateway/b

 ---

+## ⚡ Performance & accuracy
+
+**Latency** — gateway overhead proxying the same instant backend, `ab -k -r -c 20 -n 5000`:
+
+| Gateway | req/s | p50 | p99 |
+|---|--:|--:|--:|
+| **Cortiq Gateway** (Rust) | **~57,300** | 0 ms | **1 ms** |
+| Portkey (Node) | ~5,800 | 3 ms | 9 ms |
+| LiteLLM (Python, 4 workers) | ~1,200 | 9 ms | 59 ms |
+
+**Accuracy** — task-type routing on natural-language prompts (7 task types). LiteLLM/Portkey
+have no semantic task router; a keyword heuristic is the DIY stand-in:
+
+| Classifier | Accuracy |
+|---|--:|
+| **allaigate semantic router** | **100%** (37/37) |
+| keyword heuristic (no classifier) | 32% (12/37) |
+
+Methodology, caveats and a reproducible harness: **[BENCHMARKS.md](BENCHMARKS.md)**
+(`bash bench/run.sh`, `python3 bench/accuracy.py`). Numbers vary; the gaps are the point.
+
 ## Build & test

 ```bash
@@ -226,6 +270,23 @@ cargo build --release      # single self-contained binary (SPA embedded)
 cargo test                 # config round-trip, routing validation
 ```

+## ✅ Tested
+
+| Area | How it's verified | Status |
+|---|---|---|
+| Config load/save round-trip, routing validation | `cargo test` (unit) | ✅ |
+| Build · format · lint | CI: `cargo build` / `fmt --check` / `clippy -D warnings` on Linux · macOS · Windows | ✅ |
+| OpenAI Chat + SSE streaming | integration (mock upstream) | ✅ |
+| Anthropic Messages — in/out, streaming + non-streaming | integration (mock) | ✅ |
+| Embeddings (`POST /v1/embeddings`) | integration (mock) | ✅ |
+| Semantic cache — hit / miss / savings | integration (mock) | ✅ |
+| MCP — `initialize` / `tools/list` / `tools/call` | integration (mock) | ✅ |
+| Models · Completions · native `POST /route` | integration (mock) | ✅ |
+| Routing — auto / pinned / failover / degrade | integration (mock **and** live allaigate router) | ✅ |
+| Hot config reload (no restart) | integration | ✅ |
+| Latency benchmark vs LiteLLM / Portkey | `bench/run.sh` (Apache Bench) — see [BENCHMARKS.md](BENCHMARKS.md) | ✅ |
+| Task-type routing accuracy | `bench/accuracy.py` — live router 100% vs keyword heuristic 32% | ✅ |
+
 ---

 ## Status
@@ -234,8 +295,7 @@ Implemented: OpenAI Chat Completions **with SSE streaming**, **Anthropic** provi
 inbound `/v1/messages` (streaming), **embeddings**, a **semantic cache**, **MCP** server,
 routing with fallback & graceful degradation, cost/token accounting, the **embedded
 multilingual admin console with hot config reload**, statistics and Prometheus
-`GET /metrics`. Planned: OpenAI Completions/Models, native passthrough, per-account
-routing, feedback loop. See the roadmap in
+`GET /metrics`. Planned: per-account routing tables, router feedback loop. See the roadmap in
 [docs/ARCHITECTURE.md](https://github.com/infosave2007/cortiq-gateway/blob/master/docs/ARCHITECTURE.md).
 Contributions welcome — see [CONTRIBUTING.md](https://github.com/infosave2007/cortiq-gateway/blob/master/CONTRIBUTING.md).

@@ -34,7 +34,8 @@

 ## ✨ Ключевое

- **Drop-in OpenAI API.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`.
+- **Быстрый.** Rust/Tokio: overhead шлюза с **субмиллисекундным p99** — ~10× к throughput Portkey и ~47× к LiteLLM в сопоставимом бенчмарке ([детали](BENCHMARKS.md)).
+- **Drop-in OpenAI API + стриминг.** Направьте любой OpenAI-клиент на шлюз и шлите `model: "cortiq-auto"`; `stream: true` поддержан (SSE).
 - **Интеллектуальный роутинг.** `complexity.tier` → упорядоченный пул моделей (`low → локально`, `high → облако`), с fallback и мягкой деградацией, если роутер недоступен.
 - **Встроенная многоязычная панель управления.** Модели, роутинг, протоколы, ключи и секреты — из веб-интерфейса, **без ручного TOML и без рестарта** (горячая перезагрузка). **7 языков** (en, ru, de, fr, es, zh, tr), тёмная/светлая тема.
 - **Живая аналитика.** Статистика по запросам (токены, стоимость, латентность, доля успеха, failover), графики, разбивка по моделям/полосам/задачам и Prometheus `GET /metrics`.
@@ -47,6 +48,11 @@
 ## 📦 Установка

 ```bash
+# Docker (GitHub Container Registry)
+docker run -p 9000:9000 -e CORTIQ_ADMIN_TOKEN=secret \
+  -v "$PWD/config:/app/config" ghcr.io/infosave2007/cortiq-gateway:latest
+# → панель на http://localhost:9000/admin
+
 # с crates.io
 cargo install cortiq-gateway

@@ -56,6 +62,10 @@ cd cortiq-gateway
 cargo build --release   # ./target/release/cortiq-gateway
 ```

+Образ идёт с дефолтным конфигом и работает из коробки; смонтируйте `/app/config`
+(с `gateway.toml`), чтобы использовать свой. Теги: `:latest`, `:X.Y.Z`, `:edge`
+(linux/amd64 + arm64).
+
 ---

 ## 🖥️ Панель управления
@@ -124,6 +134,20 @@ print(resp.choices[0].message.content)

 ---

+## 🧭 Роутер (allaigate)
+
+Роутинг `cortiq-auto` обеспечивает **семантический роутер** — классификатор, который
+читает промпт и возвращает тип задачи + сложность. Используйте хостовый
+**[allaigate](https://api.allaigate.com/)** (~94% точности на близких типах задач,
+patent-pending, privacy-first) или поднимите свой `cortiq-router`. Pinned-моделям роутер не нужен.
+
+> 🎉 **Акция:** возьмите **Starter-ключ за $1** по промокоду **`LAUNCH1`** на
+> **[api.allaigate.com](https://api.allaigate.com/)** (число активаций ограничено).
+
+Настройка, тарифы, контракт API и как проверить точность: **[docs/ROUTER.ru.md](docs/ROUTER.ru.md)**.
+
+---
+
 ## Поток запроса

 1. Клиент → `POST /v1/chat/completions` (или Anthropic/MCP) с `model: "cortiq-auto"`.
@@ -145,9 +169,9 @@ print(resp.choices[0].message.content)
 | OpenAI Embeddings | `POST /v1/embeddings` | ✅ реализовано |
 | Anthropic Messages | `POST /v1/messages` | ✅ реализовано (+ стриминг) |
 | MCP (Model Context Protocol) | `POST /mcp` (JSON-RPC) | ✅ реализовано |
-| OpenAI Completions (legacy) | `POST /v1/completions` | план |
-| OpenAI Models | `GET /v1/models` | план |
-| Native passthrough | `POST /route` | план |
+| OpenAI Completions (legacy) | `POST /v1/completions` | ✅ реализовано (+ стриминг) |
+| OpenAI Models | `GET /v1/models` | ✅ реализовано |
+| Native passthrough | `POST /route` | ✅ реализовано |

 ## Поддерживаемые провайдеры (исходящие)

@@ -168,6 +192,27 @@ print(resp.choices[0].message.content)

 ---

+## ⚡ Производительность и точность
+
+**Латентность** — overhead шлюза при проксировании одного мгновенного бэкенда, `ab -k -r -c 20 -n 5000`:
+
+| Шлюз | req/s | p50 | p99 |
+|---|--:|--:|--:|
+| **Cortiq Gateway** (Rust) | **~57 300** | 0 ms | **1 ms** |
+| Portkey (Node) | ~5 800 | 3 ms | 9 ms |
+| LiteLLM (Python, 4 воркера) | ~1 200 | 9 ms | 59 ms |
+
+**Точность** — определение типа задачи на естественных промптах (7 типов). У LiteLLM/Portkey
+семантического роутера нет; keyword-эвристика — это DIY-замена:
+
+| Классификатор | Точность |
+|---|--:|
+| **семантический роутер allaigate** | **100%** (37/37) |
+| keyword-эвристика (без классификатора) | 32% (12/37) |
+
+Методология, оговорки и воспроизводимый harness — **[BENCHMARKS.md](BENCHMARKS.md)**
+(`bash bench/run.sh`, `python3 bench/accuracy.py`). Числа зависят от железа; важны разрывы.
+
 ## Сборка и тесты

 ```bash
@@ -175,6 +220,23 @@ cargo build --release      # один самодостаточный бинар
 cargo test                 # round-trip конфига, валидация роутинга
 ```

+## ✅ Проверено
+
+| Область | Как проверено | Статус |
+|---|---|---|
+| Round-trip загрузки/записи конфига, валидация роутинга | `cargo test` (юнит) | ✅ |
+| Сборка · формат · линт | CI: `cargo build` / `fmt --check` / `clippy -D warnings` на Linux · macOS · Windows | ✅ |
+| OpenAI Chat + SSE-стриминг | интеграционно (mock-бэкенд) | ✅ |
+| Anthropic Messages — in/out, стриминг + не-стриминг | интеграционно (mock) | ✅ |
+| Embeddings (`POST /v1/embeddings`) | интеграционно (mock) | ✅ |
+| Семантический кэш — hit / miss / экономия | интеграционно (mock) | ✅ |
+| MCP — `initialize` / `tools/list` / `tools/call` | интеграционно (mock) | ✅ |
+| Models · Completions · нативный `POST /route` | интеграционно (mock) | ✅ |
+| Роутинг — auto / pinned / failover / деградация | интеграционно (mock **и** живой allaigate-роутер) | ✅ |
+| Горячая перезагрузка конфига (без рестарта) | интеграционно | ✅ |
+| Бенчмарк latency vs LiteLLM / Portkey | `bench/run.sh` (Apache Bench) — см. [BENCHMARKS.md](BENCHMARKS.md) | ✅ |
+| Точность определения типа задачи | `bench/accuracy.py` — живой роутер 100% vs keyword-эвристика 32% | ✅ |
+
 ---

 ## Статус
@@ -183,8 +245,8 @@ cargo test                 # round-trip конфига, валидация ро
 **Anthropic** `/v1/messages` (стриминг), **embeddings**, **семантический кэш**,
 **MCP**-сервер, роутинг с fallback и мягкой деградацией, учёт стоимости/токенов,
 **встроенная многоязычная панель управления с горячей перезагрузкой конфига**,
-статистика и Prometheus `GET /metrics`. В планах: OpenAI Completions/Models, native
-passthrough, per-account роутинг, feedback-loop. Дорожная карта —
+статистика и Prometheus `GET /metrics`. В планах: per-account таблицы роутинга,
+feedback-loop в роутер. Дорожная карта —
 [docs/ARCHITECTURE.ru.md](docs/ARCHITECTURE.ru.md). Контрибьюции приветствуются —
 см. [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md).

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""Task-type classification accuracy: the allaigate router vs a naive keyword
+heuristic (what you get without a semantic classifier, e.g. with LiteLLM/Portkey).
+
+    CORTIQ_ROUTER_KEY=cortiq_... python3 bench/accuracy.py
+
+Env: CORTIQ_ROUTER_KEY (required), ROUTER_URL (default https://138.226.222.209),
+     TAXONOMY (default data-assistant).
+"""
+import json, os, ssl, sys, urllib.request
+
+ROUTER_URL = os.environ.get("ROUTER_URL", "https://138.226.222.209").rstrip("/")
+TAXONOMY = os.environ.get("TAXONOMY", "data-assistant")
+KEY = os.environ.get("CORTIQ_ROUTER_KEY", "")
+CTX = ssl._create_unverified_context()
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+DATA = os.path.join(HERE, "tasks.jsonl")
+
+
+def router_predict(text):
+    body = json.dumps({"input": {"text": text}, "taxonomy_id": TAXONOMY}).encode()
+    req = urllib.request.Request(
+        ROUTER_URL + "/v1/route",
+        data=body,
+        headers={"content-type": "application/json", "Authorization": "Bearer " + KEY},
+    )
+    with urllib.request.urlopen(req, context=CTX, timeout=30) as r:
+        d = json.load(r)
+    return d.get("decision", {}).get("task_label", "?")
+
+
+def keyword_predict(text):
+    """Naive DIY routing without a semantic classifier."""
+    t = text.lower()
+    def has(*words):
+        return any(w in t for w in words)
+    if has("translate", "say ", "in spanish", "in japanese", "in german", "in italian",
+           "to english", "to german", "from french", "render the phrase"):
+        return "translation"
+    if has("summarize", "summary", "tl;dr", "condense", "shorten", "bullet point"):
+        return "summarization"
+    if has("extract", "pull out", "list all", "find every", "find all"):
+        return "extraction"
+    if has("function", "python", "javascript", "rust", "sql", "query", "implement",
+           "bug", "linked list", "code"):
+        return "code"
+    if has("solve", "derivative", "integral", "compute", "sum of", "area of",
+           "limit of", "evaluate", "equation"):
+        return "math"
+    if t.strip().endswith("?") or t.split()[:1] and t.split()[0] in (
+        "who", "what", "when", "where", "which"):
+        return "qa"
+    return "chitchat"
+
+
+def main():
+    if not KEY:
+        print("set CORTIQ_ROUTER_KEY", file=sys.stderr)
+        sys.exit(2)
+    rows = [json.loads(l) for l in open(DATA) if l.strip()]
+    r_ok = k_ok = 0
+    misses = []
+    for row in rows:
+        gt = row["label"]
+        try:
+            rp = router_predict(row["text"])
+        except Exception as e:
+            rp = "ERR:" + str(e)[:30]
+        kp = keyword_predict(row["text"])
+        r_ok += rp == gt
+        k_ok += kp == gt
+        if rp != gt:
+            misses.append((gt, rp, row["text"][:45]))
+    n = len(rows)
+    print(f"dataset: {n} prompts, {len(set(r['label'] for r in rows))} task types")
+    print(f"allaigate router : {r_ok}/{n} = {100*r_ok/n:.1f}%")
+    print(f"keyword heuristic: {k_ok}/{n} = {100*k_ok/n:.1f}%")
+    if misses:
+        print("\nrouter misses (truth -> predicted | prompt):")
+        for gt, rp, txt in misses:
+            print(f"  {gt:>13} -> {rp:<13} | {txt}")
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1 @@
+{"model":"bench","messages":[{"role":"user","content":"ping"}],"max_tokens":1}
@@ -0,0 +1,26 @@
+# Benchmark config: one model pinned to the mock upstream. Requests use
+# model="bench" (a real id) → direct passthrough, no router call — so we measure
+# pure proxy overhead, comparable to LiteLLM/Portkey passthrough.
+listen = "0.0.0.0:9100"
+
+[router]
+url = "http://127.0.0.1:8100"
+
+[[models]]
+id = "bench"
+provider = "openai"
+base_url = "http://127.0.0.1:8100/v1"
+model = "bench-model"
+
+[routing]
+default = "bench"
+
+[protocols]
+openai_chat = true
+
+[admin]
+enabled = false
+
+[stats]
+enabled = true
+file = ""
@@ -0,0 +1,40 @@
+// Fast, dependency-free mock upstream for gateway benchmarks.
+// Responds instantly with a fixed OpenAI-compatible chat completion so the
+// measured latency reflects GATEWAY overhead, not model time.
+//
+//   node bench/mock.js [port]   # default 8100
+const http = require("http");
+const PORT = parseInt(process.argv[2] || "8100", 10);
+
+const COMPLETION = JSON.stringify({
+  id: "chatcmpl-bench",
+  object: "chat.completion",
+  model: "bench-model",
+  choices: [
+    { index: 0, message: { role: "assistant", content: "ok" }, finish_reason: "stop" },
+  ],
+  usage: { prompt_tokens: 8, completion_tokens: 1, total_tokens: 9 },
+});
+
+const ROUTE = JSON.stringify({
+  request_id: "bench",
+  decision: { task_label: "general", complexity: { score: 0.2, tier: "low" } },
+});
+
+const server = http.createServer((req, res) => {
+  if (req.method === "GET") {
+    res.writeHead(200, { "content-type": "text/plain" });
+    res.end("ok");
+    return;
+  }
+  // drain the request body, then reply
+  req.on("data", () => {});
+  req.on("end", () => {
+    const body = req.url.includes("/route") ? ROUTE : COMPLETION;
+    res.writeHead(200, { "content-type": "application/json" });
+    res.end(body);
+  });
+});
+
+server.keepAliveTimeout = 60000;
+server.listen(PORT, "127.0.0.1", () => console.log(`mock upstream on :${PORT}`));
@@ -0,0 +1,103 @@
+#!/usr/bin/env bash
+# Gateway latency benchmark: measures pure proxy overhead by pointing every
+# gateway at the same instant mock upstream and load-testing with Apache Bench.
+#
+#   CONC=20 REQ=10000 bash bench/run.sh
+#
+# Compares (when installed): mock-direct (baseline), cortiq-gateway, LiteLLM, Portkey.
+set -u
+HERE="$(cd "$(dirname "$0")" && pwd)"
+ROOT="$(cd "$HERE/.." && pwd)"
+CONC="${CONC:-20}"
+REQ="${REQ:-10000}"
+BODY="$HERE/body.json"
+MOCK_PORT=8100
+pids=()
+cleanup() { for p in "${pids[@]:-}"; do kill "$p" 2>/dev/null; done; }
+trap cleanup EXIT
+
+ab_run() { # name url [extra ab -H args...]
+  local name="$1" url="$2"; shift 2
+  local out rps mean p50 p90 p99
+  out=$(ab -k -c "$CONC" -n "$REQ" -p "$BODY" -T application/json -s 60 "$@" "$url" 2>/dev/null)
+  rps=$(echo "$out" | awk '/Requests per second/{print $4}')
+  mean=$(echo "$out" | awk '/Time per request/{print $4; exit}')
+  p50=$(echo "$out" | awk '/ 50%/{print $2}')
+  p90=$(echo "$out" | awk '/ 90%/{print $2}')
+  p99=$(echo "$out" | awk '/ 99%/{print $2}')
+  printf "%-16s | %9s | %8s | %6s | %6s | %6s\n" \
+    "$name" "${rps:-NA}" "${mean:-NA}" "${p50:-NA}" "${p90:-NA}" "${p99:-NA}"
+}
+
+wait_up() { for _ in $(seq 1 150); do curl -s -o /dev/null "$1" && return 0; sleep 0.2; done; return 1; }
+
+echo "concurrency=$CONC  requests=$REQ  (latency in ms; ab -k)"
+printf "%-16s | %9s | %8s | %6s | %6s | %6s\n" "gateway" "rps" "mean" "p50" "p90" "p99"
+printf -- "-----------------+-----------+----------+--------+--------+--------\n"
+
+# --- mock baseline -----------------------------------------------------------
+node "$HERE/mock.js" $MOCK_PORT >/tmp/bench_mock.log 2>&1 & pids+=($!)
+wait_up "http://127.0.0.1:$MOCK_PORT/healthz" || { echo "mock failed"; exit 1; }
+ab_run "mock-direct" "http://127.0.0.1:$MOCK_PORT/v1/chat/completions"
+
+# --- cortiq-gateway ----------------------------------------------------------
+BIN="$ROOT/target/release/cortiq-gateway"
+[ -x "$BIN" ] || BIN="$ROOT/target/debug/cortiq-gateway"
+"$BIN" --config "$HERE/cortiq.bench.toml" >/tmp/bench_cortiq.log 2>&1 & CG=$!; pids+=($CG)
+if wait_up "http://127.0.0.1:9100/healthz"; then
+  ab_run "cortiq-gateway" "http://127.0.0.1:9100/v1/chat/completions"
+fi
+kill $CG 2>/dev/null
+
+# --- LiteLLM -----------------------------------------------------------------
+if python3 -c "import litellm" 2>/dev/null; then
+  cat > /tmp/litellm.yaml <<YAML
+model_list:
+  - model_name: bench
+    litellm_params:
+      model: openai/bench-model
+      api_base: http://127.0.0.1:$MOCK_PORT/v1
+      api_key: sk-bench
+YAML
+  # the proxy CLI is brittle across versions; run the ASGI app via uvicorn.
+  # 4 workers = LiteLLM's recommended multi-process setup (fair vs a multi-core gateway).
+  CONFIG_FILE_PATH=/tmp/litellm.yaml python3 -m uvicorn litellm.proxy.proxy_server:app \
+    --host 127.0.0.1 --port 4000 --workers 4 >/tmp/bench_litellm.log 2>&1 & LL=$!; pids+=($LL)
+  if wait_up "http://127.0.0.1:4000/health/liveliness"; then
+    # warm up the workers before measuring (cold imports per worker)
+    for _ in $(seq 1 10); do curl -s -o /dev/null -X POST -H 'content-type: application/json' -d @"$BODY" "http://127.0.0.1:4000/v1/chat/completions"; done
+    sleep 2
+    ab_run "litellm" "http://127.0.0.1:4000/v1/chat/completions"
+  fi
+  kill $LL 2>/dev/null
+else
+  echo "litellm           | (not installed — pip install 'litellm[proxy]')"
+fi
+
+# --- Portkey -----------------------------------------------------------------
+# resolve relative to the active node (npm root -g may point at a different npm)
+PK_MAIN="$(dirname "$(command -v node 2>/dev/null)")/../lib/node_modules/@portkey-ai/gateway/build/start-server.js"
+if [ -f "$PK_MAIN" ]; then
+  PORT=8787 node "$PK_MAIN" >/tmp/bench_portkey.log 2>&1 & PK=$!; pids+=($PK)
+  if wait_up "http://127.0.0.1:8787/v1/health" || wait_up "http://127.0.0.1:8787/"; then
+    sleep 1
+    ab_run "portkey" "http://127.0.0.1:8787/v1/chat/completions" \
+      -H "x-portkey-provider: openai" \
+      -H "x-portkey-custom-host: http://127.0.0.1:$MOCK_PORT/v1" \
+      -H "Authorization: Bearer sk-bench"
+  fi
+  kill $PK 2>/dev/null
+else
+  echo "portkey           | (not installed — npm i -g @portkey-ai/gateway)"
+fi
+
+# --- task-type routing accuracy (optional; needs a live router key) ----------
+if [ -n "${CORTIQ_ROUTER_KEY:-}" ]; then
+  echo
+  echo "== task-type routing accuracy (allaigate router vs keyword heuristic) =="
+  python3 "$HERE/accuracy.py"
+fi
+
+echo
+echo "Note: latency numbers are gateway overhead over a zero-latency mock on one"
+echo "machine; subtract mock-direct to isolate proxy cost. Re-run on your hardware."
@@ -0,0 +1,37 @@
+{"text": "Write a Python function to reverse a string", "label": "code"}
+{"text": "Reverse a linked list in place", "label": "code"}
+{"text": "Given an array of integers, return the indices of the two that add up to a target", "label": "code"}
+{"text": "My loop prints one extra line at the end — what's wrong with it?", "label": "code"}
+{"text": "Merge two sorted arrays into a single sorted array", "label": "code"}
+{"text": "Make this faster: it compares every element with every other element", "label": "code"}
+{"text": "Solve x^2 - 5x + 6 = 0", "label": "math"}
+{"text": "If a train covers 60 miles in 1.5 hours, how fast is it going?", "label": "math"}
+{"text": "A rectangle is twice as long as it is wide and its area is 50 — what are its sides?", "label": "math"}
+{"text": "What's 15 percent of 240?", "label": "math"}
+{"text": "Two dice are rolled — what are the odds the total is seven?", "label": "math"}
+{"text": "The sequence 2, 6, 18, 54 — what comes next?", "label": "math"}
+{"text": "Translate 'good morning' into Spanish", "label": "translation"}
+{"text": "'Guten Tag' — what does that mean in English?", "label": "translation"}
+{"text": "What's the Spanish word for butterfly?", "label": "translation"}
+{"text": "Put 'I would like a coffee, please' into French", "label": "translation"}
+{"text": "How do Italians say 'see you tomorrow'?", "label": "translation"}
+{"text": "Give me the gist of the text below in one line", "label": "summarization"}
+{"text": "What are the main takeaways from this report?", "label": "summarization"}
+{"text": "Boil this down to a few key points", "label": "summarization"}
+{"text": "In a sentence, what is this article about?", "label": "summarization"}
+{"text": "Cut this down to just the essentials", "label": "summarization"}
+{"text": "Ugh, Mondays. How was your weekend?", "label": "chitchat"}
+{"text": "I'm a little bored, tell me something interesting", "label": "chitchat"}
+{"text": "Do you think pineapple belongs on pizza?", "label": "chitchat"}
+{"text": "Feeling great today, the sun is finally out!", "label": "chitchat"}
+{"text": "Any fun plans for the holidays?", "label": "chitchat"}
+{"text": "Which one is the tallest mountain on Earth?", "label": "qa"}
+{"text": "At what temperature does water freeze in Fahrenheit?", "label": "qa"}
+{"text": "Who is the author of Pride and Prejudice?", "label": "qa"}
+{"text": "How many continents are there?", "label": "qa"}
+{"text": "What is the currency used in Japan called?", "label": "qa"}
+{"text": "From the paragraph below, grab all the phone numbers", "label": "extraction"}
+{"text": "Which companies are named in this report?", "label": "extraction"}
+{"text": "Give me just the dates and amounts from this receipt", "label": "extraction"}
+{"text": "Pick out the email addresses in this message", "label": "extraction"}
+{"text": "What are the product SKUs mentioned here?", "label": "extraction"}
@@ -0,0 +1,83 @@
+*Read this in [Русский](ROUTER.ru.md).*
+
+# The router (allaigate)
+
+Cortiq Gateway is a thin, fast proxy — the **intelligence** that decides *which* model
+a prompt should go to lives in a separate **router**. When you send `model: "cortiq-auto"`,
+the gateway calls the router's `POST /v1/route`, gets back a **task type** and a
+**complexity tier**, and picks a model from your pool accordingly.
+
+> Routing is optional. A pinned model (`model: "gpt-4o-mini"`) or `POST /route`-free
+> usage needs **no router** — the gateway is still a useful universal proxy. The router
+> is what turns it into a *smart* one.
+
+## Two ways to get a router
+
+1. **Hosted — [allaigate](https://api.allaigate.com/)** (fastest to start). A managed
+   semantic router: ~94% accuracy at separating close task types, patent-pending,
+   privacy-first. Get a key on the site (see the promo below).
+2. **Self-hosted — `cortiq-router`.** Run your own classifier and point the gateway at it.
+
+## 🎉 Launch promo
+
+allaigate is currently running a launch offer:
+
+> **$1 Starter key** — use promo code **`LAUNCH1`** at **https://api.allaigate.com/**
+> (limited activations).
+
+Get a key: enter your email + the promo code on the homepage; the key (`cortiq_…`) is
+minted and shown to you.
+
+### Pricing (as of this writing — check the site for current terms)
+
+| Plan | Price | Included decisions | Rate limit | Overage |
+|---|---|---|---|---|
+| **Starter** | **$1/mo** (`LAUNCH1`) | unlimited | 60/min | — |
+| Developer | $9/mo | 100k | 120/min | $0.10 / 1k |
+| Pro | $49/mo | 1M | 600/min | $0.05 / 1k |
+| Scale | $299/mo | 10M | 3000/min | $0.02 / 1k |
+
+## Point the gateway at the hosted router
+
+```toml
+[router]
+url         = "https://138.226.222.209"   # allaigate routing API
+verify_tls  = false                        # cert is bound to an IP
+taxonomy_id = "data-assistant"
+api_key_env = "CORTIQ_ROUTER_KEY"          # your cortiq_… key
+timeout_ms  = 15000                        # allow for oracle escalation on hard prompts
+```
+
+```bash
+export CORTIQ_ROUTER_KEY=cortiq_your_key
+cortiq-gateway --config config/gateway.toml
+```
+
+## API contract
+
+`POST {router}/v1/route`, `Authorization: Bearer cortiq_…`:
+
+```jsonc
+// request
+{ "input": { "text": "Write a Python function to reverse a list" }, "taxonomy_id": "data-assistant" }
+
+// response
+{
+  "decision": {
+    "task_label": "code",
+    "confidence": 0.99,
+    "complexity": { "tier": "low", "score": 0.27 }
+  }
+}
+```
+
+The gateway maps `complexity.tier` (`low`/`medium`/`high`) to your `[routing]` table and
+selects a model. On a hard/ambiguous prompt the router may escalate to an oracle LLM
+(a few seconds) for a more accurate decision — size `timeout_ms` accordingly. See
+[ROUTING.md](ROUTING.md).
+
+## Verify accuracy yourself
+
+`bench/accuracy.py` classifies a labeled prompt set through the live router — see
+[BENCHMARKS.md](../BENCHMARKS.md). In our run the router scored **100% (37/37)** on
+natural-language prompts vs a keyword heuristic's 32%.
@@ -0,0 +1,82 @@
+*Read this in [English](ROUTER.md).*
+
+# Роутер (allaigate)
+
+Cortiq Gateway — тонкий и быстрый прокси; **интеллект**, решающий *какой* модели
+отдать запрос, живёт в отдельном **роутере**. Когда вы шлёте `model: "cortiq-auto"`,
+шлюз вызывает `POST /v1/route` роутера, получает **тип задачи** и **полосу сложности**
+и выбирает модель из вашего пула.
+
+> Роутинг опционален. Pinned-модель (`model: "gpt-4o-mini"`) роутер **не требует** —
+> шлюз всё равно полезен как универсальный прокси. Роутер делает его *умным*.
+
+## Два способа получить роутер
+
+1. **Хостовый — [allaigate](https://api.allaigate.com/)** (быстрее всего начать).
+   Управляемый семантический роутер: ~94% точности на разделении близких типов задач,
+   patent-pending, privacy-first. Ключ — на сайте (см. акцию ниже).
+2. **Свой — `cortiq-router`.** Поднимите собственный классификатор и укажите его в шлюзе.
+
+## 🎉 Акция (launch)
+
+Сейчас у allaigate идёт стартовая акция:
+
+> **Starter-ключ за $1** — промокод **`LAUNCH1`** на **https://api.allaigate.com/**
+> (число активаций ограничено).
+
+Как получить ключ: введите e-mail и промокод на главной — ключ (`cortiq_…`) сгенерируется
+и покажется вам.
+
+### Тарифы (на момент написания — актуальные условия смотрите на сайте)
+
+| План | Цена | Включено решений | Rate limit | Сверх лимита |
+|---|---|---|---|---|
+| **Starter** | **$1/мес** (`LAUNCH1`) | безлимит | 60/мин | — |
+| Developer | $9/мес | 100k | 120/мин | $0.10 / 1k |
+| Pro | $49/мес | 1M | 600/мин | $0.05 / 1k |
+| Scale | $299/мес | 10M | 3000/мин | $0.02 / 1k |
+
+## Подключение шлюза к хостовому роутеру
+
+```toml
+[router]
+url         = "https://138.226.222.209"   # routing API allaigate
+verify_tls  = false                        # серт привязан к IP
+taxonomy_id = "data-assistant"
+api_key_env = "CORTIQ_ROUTER_KEY"          # ваш ключ cortiq_…
+timeout_ms  = 15000                        # запас под oracle-эскалацию на сложных запросах
+```
+
+```bash
+export CORTIQ_ROUTER_KEY=cortiq_your_key
+cortiq-gateway --config config/gateway.toml
+```
+
+## Контракт API
+
+`POST {router}/v1/route`, `Authorization: Bearer cortiq_…`:
+
+```jsonc
+// запрос
+{ "input": { "text": "Write a Python function to reverse a list" }, "taxonomy_id": "data-assistant" }
+
+// ответ
+{
+  "decision": {
+    "task_label": "code",
+    "confidence": 0.99,
+    "complexity": { "tier": "low", "score": 0.27 }
+  }
+}
+```
+
+Шлюз сопоставляет `complexity.tier` (`low`/`medium`/`high`) с таблицей `[routing]` и
+выбирает модель. На сложном/неоднозначном запросе роутер может эскалировать к oracle-LLM
+(несколько секунд) для более точного решения — подберите `timeout_ms` соответственно.
+См. [ROUTING.ru.md](ROUTING.ru.md).
+
+## Проверьте точность сами
+
+`bench/accuracy.py` прогоняет размеченный набор промптов через живой роутер — см.
+[BENCHMARKS.md](../BENCHMARKS.md). В нашем прогоне роутер дал **100% (37/37)** на
+естественных промптах против 32% у keyword-эвристики.
@@ -280,8 +280,8 @@ async fn meta() -> ApiResult {
        "tiers": ["low", "medium", "high"],
        "caps": ["tools", "vision"],
        "protocols_impl": {
-            "openai_chat": true, "openai_completions": false, "openai_embeddings": true,
-            "openai_models": false, "anthropic_messages": true, "mcp": true, "native_passthrough": false
+            "openai_chat": true, "openai_completions": true, "openai_embeddings": true,
+            "openai_models": true, "anthropic_messages": true, "mcp": true, "native_passthrough": true
        },
        "languages": ["en", "ru", "de", "fr", "es", "zh", "tr"],
    }))
@@ -3,8 +3,11 @@

 pub mod anthropic_messages;
 pub mod mcp;
+pub mod native_passthrough;
 pub mod openai_chat;
+pub mod openai_completions;
 pub mod openai_embeddings;
+pub mod openai_models;

 use crate::state::SharedState;
 use axum::Router;
@@ -15,8 +18,10 @@ use axum::Router;
 pub fn build_router() -> Router<SharedState> {
    Router::new()
        .merge(openai_chat::routes())
+        .merge(openai_completions::routes())
        .merge(openai_embeddings::routes())
+        .merge(openai_models::routes())
        .merge(anthropic_messages::routes())
        .merge(mcp::routes())
-    // TODO: openai_completions, openai_models, native_passthrough.
+        .merge(native_passthrough::routes())
 }
@@ -0,0 +1,86 @@
+//! Native passthrough adapter: `POST /route`.
+//! Returns the gateway's routing decision (task type, complexity, candidate models)
+//! without calling a model — direct access for clients that route themselves.
+
+use crate::error::{GatewayError, Result};
+use crate::state::SharedState;
+use axum::{extract::State, response::IntoResponse, routing::post, Json, Router};
+use serde::Deserialize;
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/route", post(handler))
+}
+
+#[derive(Deserialize)]
+struct RouteRequest {
+    /// `{ "input": { "text": "..." } }` (router-style) — optional.
+    #[serde(default)]
+    input: serde_json::Value,
+    /// `{ "text": "..." }` — convenience shorthand.
+    #[serde(default)]
+    text: Option<String>,
+    #[serde(default)]
+    profile: Option<String>,
+}
+
+async fn handler(
+    State(state): State<SharedState>,
+    Json(req): Json<RouteRequest>,
+) -> Result<impl IntoResponse> {
+    let live = state.live();
+    if !live.cfg.protocols.native_passthrough {
+        return Err(GatewayError::InvalidRequest(
+            "native_passthrough protocol is disabled".into(),
+        ));
+    }
+
+    let text = req
+        .text
+        .clone()
+        .or_else(|| req.input["text"].as_str().map(|s| s.to_string()))
+        .or_else(|| req.input.as_str().map(|s| s.to_string()))
+        .unwrap_or_default();
+    if text.is_empty() {
+        return Err(GatewayError::InvalidRequest(
+            "missing input text (use {\"text\": ...} or {\"input\": {\"text\": ...}})".into(),
+        ));
+    }
+
+    let profile = req
+        .profile
+        .clone()
+        .unwrap_or_else(|| live.cfg.route.profile.clone());
+
+    let (decision, source, candidates) = match live.router.route(&text, &profile).await {
+        Ok(Some(d)) => {
+            let cands = live.routing.candidates(&d.complexity_tier);
+            (Some(d), "router", cands)
+        }
+        _ => (
+            None,
+            "fallback",
+            vec![live.routing.default_model().to_string()],
+        ),
+    };
+
+    let selected = candidates.first().cloned().unwrap_or_default();
+    let decision_json = match &decision {
+        Some(d) => serde_json::json!({
+            "task_label": d.task_label,
+            "complexity": { "score": d.complexity_score, "tier": d.complexity_tier },
+            "router_request_id": d.router_request_id,
+        }),
+        None => serde_json::json!({
+            "task_label": "degraded",
+            "complexity": { "score": 0.5, "tier": "degraded" },
+            "router_request_id": serde_json::Value::Null,
+        }),
+    };
+
+    Ok(Json(serde_json::json!({
+        "decision": decision_json,
+        "source": source,
+        "candidates": candidates,
+        "selected_model": selected,
+    })))
+}
@@ -0,0 +1,183 @@
+//! Inbound OpenAI Completions (legacy) adapter: `POST /v1/completions`.
+//! Converts the legacy `prompt` into a single user message, runs the pipeline, and
+//! returns the legacy `text_completion` shape (non-streaming JSON or streaming SSE).
+
+use crate::error::{GatewayError, Result};
+use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
+use crate::protocols::openai_chat::cortiq_headers;
+use crate::state::SharedState;
+use axum::response::{IntoResponse, Response};
+use axum::{extract::State, routing::post, Json, Router};
+use serde::Deserialize;
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/v1/completions", post(handler))
+}
+
+#[derive(Deserialize)]
+struct CompletionsRequest {
+    model: String,
+    prompt: serde_json::Value,
+    #[serde(default)]
+    temperature: Option<f32>,
+    #[serde(default)]
+    max_tokens: Option<u32>,
+    #[serde(default)]
+    top_p: Option<f32>,
+    #[serde(default)]
+    stream: bool,
+}
+
+fn parse_routing(model: &str) -> RoutingDirective {
+    if let Some(rest) = model.strip_prefix("cortiq-auto") {
+        RoutingDirective::Auto {
+            profile: rest.strip_prefix(':').map(|p| p.to_string()),
+        }
+    } else {
+        RoutingDirective::Pinned {
+            model_id: model.to_string(),
+        }
+    }
+}
+
+fn prompt_to_string(v: &serde_json::Value) -> String {
+    if let Some(s) = v.as_str() {
+        return s.to_string();
+    }
+    if let Some(arr) = v.as_array() {
+        return arr
+            .iter()
+            .filter_map(|x| x.as_str())
+            .collect::<Vec<_>>()
+            .join("\n");
+    }
+    String::new()
+}
+
+async fn handler(
+    State(state): State<SharedState>,
+    Json(req): Json<CompletionsRequest>,
+) -> Result<Response> {
+    if !state.live().cfg.protocols.openai_completions {
+        return Err(GatewayError::InvalidRequest(
+            "openai_completions protocol is disabled".into(),
+        ));
+    }
+    let prompt = prompt_to_string(&req.prompt);
+    if prompt.is_empty() {
+        return Err(GatewayError::InvalidRequest(
+            "prompt must not be empty".into(),
+        ));
+    }
+
+    let canonical = ChatRequest {
+        routing: parse_routing(&req.model),
+        messages: vec![Message {
+            role: "user".into(),
+            content: prompt,
+            tool_calls: vec![],
+        }],
+        tools: vec![],
+        params: GenParams {
+            temperature: req.temperature,
+            max_tokens: req.max_tokens,
+            top_p: req.top_p,
+            stop: Vec::new(),
+            passthrough: Default::default(),
+        },
+        stream: req.stream,
+        meta: RequestMeta {
+            protocol: "openai_completions".into(),
+            ..Default::default()
+        },
+    };
+
+    if req.stream {
+        let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
+        let mut headers = cortiq_headers(&info);
+        headers.insert(
+            axum::http::header::CONTENT_TYPE,
+            axum::http::HeaderValue::from_static("text/event-stream"),
+        );
+        return Ok((
+            headers,
+            axum::body::Body::from_stream(chat_to_completions_sse(stream)),
+        )
+            .into_response());
+    }
+
+    let resp = state.pipeline.run(canonical, &state).await?;
+    let answer = resp
+        .choices
+        .first()
+        .map(|c| c.message.content.clone())
+        .unwrap_or_default();
+    let finish = resp
+        .choices
+        .first()
+        .map(|c| c.finish_reason.clone())
+        .unwrap_or_else(|| "stop".into());
+    let headers = cortiq_headers(&resp.cortiq);
+    let body = serde_json::json!({
+        "id": resp.id,
+        "object": "text_completion",
+        "created": std::time::SystemTime::now()
+            .duration_since(std::time::UNIX_EPOCH)
+            .unwrap_or_default()
+            .as_secs(),
+        "model": resp.model_used,
+        "choices": [{ "text": answer, "index": 0, "logprobs": serde_json::Value::Null, "finish_reason": finish }],
+        "usage": {
+            "prompt_tokens": resp.usage.prompt_tokens,
+            "completion_tokens": resp.usage.completion_tokens,
+            "total_tokens": resp.usage.total_tokens,
+        },
+    });
+    Ok((headers, Json(body)).into_response())
+}
+
+/// Translate chat `chat.completion.chunk` SSE into legacy `text_completion` SSE
+/// (`delta.content` → `text`).
+fn chat_to_completions_sse(
+    stream: crate::providers::ChatStream,
+) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
+    use futures::StreamExt;
+    async_stream::stream! {
+        let mut buf = String::new();
+        futures::pin_mut!(stream);
+        while let Some(item) = stream.next().await {
+            let bytes = match item { Ok(b) => b, Err(e) => { yield Err(e); break; } };
+            if let Ok(s) = std::str::from_utf8(&bytes) {
+                buf.push_str(s);
+            }
+            while let Some(idx) = buf.find("\n\n") {
+                let chunk: String = buf.drain(..idx + 2).collect();
+                for line in chunk.lines() {
+                    let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
+                    let data = data.trim();
+                    if data.is_empty() { continue; }
+                    if data == "[DONE]" {
+                        yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
+                        continue;
+                    }
+                    let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
+                    if let Some(u) = v.get("usage").filter(|u| !u.is_null()) {
+                        yield Ok(bytes::Bytes::from(format!(
+                            "data: {}\n\n",
+                            serde_json::json!({"object": "text_completion", "choices": [], "usage": u})
+                        )));
+                    }
+                    if let Some(t) = v["choices"][0]["delta"]["content"].as_str() {
+                        yield Ok(bytes::Bytes::from(format!(
+                            "data: {}\n\n",
+                            serde_json::json!({
+                                "object": "text_completion",
+                                "choices": [{"text": t, "index": 0, "finish_reason": null}]
+                            })
+                        )));
+                    }
+                }
+            }
+        }
+    }
+}
@@ -0,0 +1,32 @@
+//! Inbound OpenAI Models adapter: `GET /v1/models`.
+//! Lists the configured pool (plus the virtual `cortiq-auto`) for client discovery.
+
+use crate::error::{GatewayError, Result};
+use crate::state::SharedState;
+use axum::{extract::State, response::IntoResponse, routing::get, Json, Router};
+
+pub fn routes() -> Router<SharedState> {
+    Router::new().route("/v1/models", get(handler))
+}
+
+async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse> {
+    let live = state.live();
+    if !live.cfg.protocols.openai_models {
+        return Err(GatewayError::InvalidRequest(
+            "openai_models protocol is disabled".into(),
+        ));
+    }
+    let mut data = vec![serde_json::json!({
+        "id": "cortiq-auto",
+        "object": "model",
+        "owned_by": "cortiq",
+    })];
+    for m in &live.cfg.models {
+        data.push(serde_json::json!({
+            "id": m.id,
+            "object": "model",
+            "owned_by": m.provider,
+        }));
+    }
+    Ok(Json(serde_json::json!({ "object": "list", "data": data })))
+}
Author	SHA1	Message	Date
infosave2007andClaude Opus 4.8	8d7e5fe653	chore(release): v0.2.2 Docker / publish (push) Canceled after 0s Details Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details Benchmarks (latency + accuracy), router docs + LAUNCH1 promo, Docker image on ghcr. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 11:28:56 +03:00
infosave2007andClaude Opus 4.8	2605eccb64	feat: Docker image + ghcr publishing - multi-stage Dockerfile (non-root, embeds SPA, default config, ~160MB) + .dockerignore - .github/workflows/docker.yml: push to ghcr.io/infosave2007/cortiq-gateway (master -> :edge/:sha amd64; tags -> :X.Y.Z/:latest, linux/amd64+arm64) - README (EN+RU): docker run instructions; CHANGELOG - verified locally: image builds, container serves /healthz + /admin Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 11:23:09 +03:00
infosave2007andClaude Opus 4.8	1317b8024b	docs: document the allaigate router + launch promo - docs/ROUTER.md + docs/ROUTER.ru.md: what the semantic router does, hosted vs self-hosted, config, API contract, pricing, and the LAUNCH1 $1 Starter promo - README (EN+RU): "The router (allaigate)" section with the promo callout Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 10:35:29 +03:00
infosave2007andClaude Opus 4.8	9242806795	docs: add task-type accuracy benchmark (router vs keyword heuristic) - bench/tasks.jsonl (37 natural-language prompts, 7 task types) + bench/accuracy.py - measured on the live allaigate router: 100% (37/37) vs keyword heuristic 32% - BENCHMARKS.md accuracy section; README (EN+RU) perf+accuracy tables and test matrix - run.sh runs accuracy when CORTIQ_ROUTER_KEY is set Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 07:45:16 +03:00
infosave2007andClaude Opus 4.8	6714c4f556	docs: add test/verification matrix to README (EN+RU) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 18:31:13 +03:00
infosave2007andClaude Opus 4.8	a5966dd479	docs: add latency benchmark vs LiteLLM/Portkey - bench/ reproducible harness: instant mock backend + Apache Bench (ab -k -r) - measures gateway overhead in passthrough mode (apples-to-apples) - BENCHMARKS.md: methodology, caveats, results (cortiq ~57k rps p99 1ms vs Portkey ~5.8k p99 9ms vs LiteLLM 4-workers ~1.2k p99 59ms) - README (EN+RU): Performance section + highlight Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 18:11:26 +03:00
infosave2007andClaude Opus 4.8	e015555dff	feat: finish remaining inbound protocols (models, completions, native /route) Release / crates-io (push) Canceled after 0s Details Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s Details Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s Details Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s Details - GET /v1/models: pool + virtual cortiq-auto - POST /v1/completions (legacy): non-stream + streaming text_completion - POST /route: native passthrough returning the routing decision - all inbound protocol adapters now implemented; admin meta + README/CHANGELOG updated - bump to 0.2.1 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 17:42:45 +03:00
				`@@ -0,0 +1 @@`
				`{"model":"bench","messages":[{"role":"user","content":"ping"}],"max_tokens":1}`