fix(stream): no choice-less chunks unless the client asks for usage

Every stream used to carry two chunks with an empty choices array: usage,
which the upstream emits BEFORE the stop reason, and a cortiq_cost_usd chunk
the gateway added itself. OpenAI ships the usage chunk only when the request
sets stream_options: {"include_usage": true}, and chat UIs do not set it -
which is why the same tool, in the same Open WebUI, sees neither chunk from
Groq and two from us.

Honour the flag. With no stream_options a stream is now exactly
role -> content... -> tool_calls -> finish_reason -> [DONE], with no
choice-less chunks at all. When usage is requested it ships once, after the
stop reason, with cortiq_cost_usd folded into the same chunk rather than
trailing behind it.

Token accounting, cost and the X-Cortiq-* headers are untouched: those numbers
were never read back off the wire.

Verified: the default stream matches the reference shape byte for byte, the
opt-in stream matches OpenAI's, the official OpenAI SDK reassembles the
streamed tool call, and the full Open WebUI 0.11.0 tool loop still runs the
tool and answers from its result.

CHANGELOG.md

@@ -3,6 +3,143 @@
 All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/).
 
+## [0.2.49] - 2026-08-21
+
+### Fixed
+- **The SSE stream now has the same shape every hosted provider sends.** Two choice-less
+  chunks used to ride along on every stream: `usage` (emitted by the upstream *before* the
+  stop reason) and a `cortiq_cost_usd` chunk of the gateway's own. OpenAI ships the usage
+  chunk only when the client sets `stream_options: {"include_usage": true}` — and chat UIs
+  do not set it, which is why a Groq stream carries neither. The gateway now honours that
+  flag: with no `stream_options`, a stream is exactly
+  `role → content… → tool_calls → finish_reason → [DONE]` with no choice-less chunks at
+  all. When usage *is* requested it ships once, **after** the stop reason, with
+  `cortiq_cost_usd` folded into the same chunk instead of trailing behind it.
+
+  Token accounting, cost and the `X-Cortiq-*` headers are unchanged — the numbers were
+  never taken from those chunks.
+
+## [0.2.48] - 2026-08-21
+
+Verified end to end: Open WebUI 0.11.0 → gateway → managed CMF/Qwen, native
+function calling, tool executed and its result fed back into the reply.
+
+### Fixed
+- **A tool call is never delivered twice.** Qwen-family templates write the call into the
+  generated text as `<tool_call>{…}</tool_call>`; some runtimes parse it into structured
+  `tool_calls` *and* still stream the raw markup as `content`, so the client sees the call
+  once as prose it must not show and once as the real thing. The gateway now strips the
+  markup from `content` — including when the marker is split across SSE chunks — while
+  passing the structured call through untouched. Ordinary text containing `<` is not
+  affected, and an unterminated block is handed back as text rather than swallowed.
+- **`finish_reason` agrees with the reply.** A response carrying tool calls now always ends
+  with `finish_reason: "tool_calls"`, whatever the upstream called it — clients branch on
+  that field to decide whether to run the tool.
+- **Streamed `tool_calls` are brought to the OpenAI shape**: `index` on every entry, `id`
+  and `type` exactly once per call (continuation deltas that carry only an argument
+  fragment are left alone, as the spec requires), and `function.arguments` always a string
+  even when the runtime sends an object.
+
+### Added
+- **Unparsed `<tool_call>` markup is promoted into a real call.** When the upstream never
+  parsed it, the gateway turns it into a proper `tool_calls` entry with a generated
+  `call_…` id, so the call is executable instead of decorative. Markup that is not a JSON
+  object with a `name` is ignored — a stray marker must not become a bogus call.
+
+## [0.2.47] - 2026-08-21
+
+### Changed
+- **The hosted allaigate router is addressed by hostname:
+  `https://router.allaigate.com`** is now the default `[router].url`, and
+  `verify_tls` defaults to `true`. The endpoint used to be reachable only by bare IP,
+  whose certificate is issued for the hostname — so every hosted setup had to run with
+  certificate verification switched off. It no longer does.
+- **Existing configs are migrated on load.** A `[router].url` still pointing at the old
+  `138.226.222.209` (with or without a scheme or trailing slash) is rewritten to the
+  hostname, `verify_tls` is turned back on, and the file is saved — the endpoint is the
+  same machine, and this is the difference between a verified connection and a blind one.
+  A self-hosted router URL is never touched.
+- README, `docs/ROUTER.md` (both languages), `config/gateway.example.toml` and
+  `bench/accuracy.py` updated to the hostname.
+
+## [0.2.46] - 2026-08-21
+
+Verified end to end against a real Open WebUI container talking to the gateway.
+
+### Fixed
+- **Offered `tools` no longer make a local model unroutable.** Open WebUI attaches its ~26
+  built-in tools (knowledge bases, notes, calendar, …) to *every* message, and the gateway
+  treated that offer as a hard capability requirement — so a plain "hello" against a pool of
+  tools-less local models failed with «No healthy model available that satisfies required
+  capabilities». In the OpenAI contract `tools` is an offer: only `tool_choice: "required"`
+  (or a named function) makes the call impossible without them. The gateway now still
+  prefers a tool-capable model, but falls back to serving the request with the tools
+  stripped (along with `tool_choice`/`parallel_tool_calls`, which a provider rejects on
+  their own). `tool_choice: "required"` still fails loudly, as it must.
+- **A pinned model that merely lacks tool support is honoured** instead of being reported as
+  «not available — it is not in the pool (deleted?)». An unknown pin still fails loudly.
+
+### Added
+- `X-Cortiq-Tools-Dropped: true` (and `cortiq.tools_dropped` in the echo block) whenever the
+  gateway answers without the tools the client offered, so the downgrade is never silent.
+
+## [0.2.45] - 2026-08-21
+
+### Fixed
+- **Open WebUI (and every strict OpenAI client) can finally chat with the gateway.**
+  `POST /v1/chat/completions` rejected perfectly ordinary payloads with a bare HTTP 422 —
+  a plain-text body no OpenAI client can parse, which is why the chat window just spun with
+  no answer and no error. The request parser now accepts every shape real clients send:
+  `content` as a string, `null`, or an array of content parts (text parts concatenated,
+  images skipped), `tools: null`, `stream: null`, a string `reasoning_effort`
+  (`low`/`medium`/`high`) as well as a numeric budget, `max_completion_tokens` as an alias
+  for `max_tokens`, `messages[].name` and `messages[].tool_call_id` proxied through, and a
+  missing `Content-Type`. A body that genuinely cannot be read now returns an OpenAI error
+  envelope with HTTP 400.
+- **Strict OpenAI response JSON, no optional `null`s.** Absent tool calls no longer emit
+  `"tool_calls": null` (the key is omitted entirely); `finish_reason` is mapped to the
+  spec's closed set, so `cortiq serve`'s `max_tokens` reaches clients as `length`; the
+  response always carries a non-empty `id`.
+- **`model` echoes the id the client requested** (`cortiq-auto` stays `cortiq-auto`, not the
+  upstream's `cortiq`), matching OpenAI's own behaviour. The model that actually answered
+  stays in `cortiq.selected_model` and `X-Cortiq-Selected-Model`. Applies to
+  `/v1/chat/completions`, `/v1/completions` and `/v1/messages`.
+- **Streaming is re-normalised instead of forwarded verbatim.** Every chunk is reissued as a
+  strict `chat.completion.chunk`: the stream opens with a `role` delta, `message`-shaped
+  chunks are converted to `delta`, empty deltas are dropped, and it always closes with one
+  terminal `finish_reason` and `data: [DONE]` — even when the upstream stops early or dies
+  mid-flight, in which case the failure arrives as an SSE `error` event rather than a client
+  waiting forever. SSE responses also carry `X-Accel-Buffering: no` so nginx/Traefik cannot
+  buffer the stream into a hang.
+- **Client bookkeeping keys are no longer proxied upstream.** `chat_id`, `session_id`,
+  `metadata`, `features`, `background_tasks` and friends (plus a non-string `user`) are
+  stripped before the provider call, so strict backends stop rejecting the request.
+
+### Added
+- **`GET /v1/models` lists managed local CMF models.** Every `[[cmf.servers]]` entry now
+  appears alongside `cortiq-auto` and the `[[models]]` pool, with live `running`/`healthy`
+  state in a non-standard `cortiq` field, so Open WebUI and other clients can pin a local
+  `.cmf` directly. Entries also carry `created`.
+
+### Changed
+- Dropped leftover `[to_wire]` debug prints from the OpenAI provider — they wrote to stderr
+  on every single request.
+
+## [0.2.44] - 2026-08-08
+
+### Added
+- **Power controls in Settings**: «Stop everything & power off» stops every managed local
+  model server and exits the gateway (the panel shows a clear "gateway is off" screen);
+  «Restart everything» (relabeled) re-execs the gateway with its local servers. Both behind
+  confirm dialogs, translated in all seven languages.
+
+### Fixed
+- `cortiq-gateway` typed in any directory picks up your configuration. Resolution order:
+  `--config` → a readable `./config/gateway.toml` → `$CORTIQ_GATEWAY_HOME` →
+  `~/.cortiq-gateway/gateway.toml`. An unrelated project that merely contains `Cargo.toml`
+  and a `config/` folder no longer hijacks the config path ("cannot read config
+  config/gateway.toml").
+
 ## [0.2.43] - 2026-08-08
 
 ### Added

Cargo.lock

@@ -285,7 +285,7 @@ checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
 
 [[package]]
 name = "cortiq-gateway"
-version = "0.2.43"
+version = "0.2.49"
 dependencies = [
  "anyhow",
  "arc-swap",

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "cortiq-gateway"
-version = "0.2.43"
+version = "0.2.49"
 edition = "2021"
 rust-version = "1.75"
 license = "Apache-2.0"

README.md

@@ -175,9 +175,9 @@ print(resp.choices[0].message.content)
 `model: "cortiq-auto"` enables routing. Any **real** model name from the config
 (`"gpt-4o-mini"`, `"local-qwen"`) is a direct passthrough, without routing.
 
-> **Using the hosted allaigate router?** Set `url = "https://138.226.222.209"`,
-> `verify_tls = false`, `taxonomy_id = "data-assistant"`, and a `cortiq_…` key in
-> `CORTIQ_ROUTER_KEY`. On hard prompts the router escalates to an oracle (~10 s), so
+> **Using the hosted allaigate router?** It is the default: `url =
+> "https://router.allaigate.com"` with `verify_tls = true`. Set `taxonomy_id =
+> "data-assistant"` and a `cortiq_…` key in `CORTIQ_ROUTER_KEY`. On hard prompts the router escalates to an oracle (~10 s), so
 > set `timeout_ms = 12000+` — otherwise the gateway gracefully degrades to the default model.
 
 ---

README.ru.md

@@ -167,9 +167,9 @@ print(resp.choices[0].message.content)
 `model: "cortiq-auto"` включает роутинг. Любое **реальное** имя модели из конфига
 (`"gpt-4o-mini"`, `"local-qwen"`) — это passthrough напрямую, без роутинга.
 
-> **Используете хостовый allaigate-роутер?** Укажите `url = "https://138.226.222.209"`,
-> `verify_tls = false`, `taxonomy_id = "data-assistant"` и ключ `cortiq_…` в
-> `CORTIQ_ROUTER_KEY`. На сложных запросах роутер эскалирует к oracle (~10 с), поэтому
+> **Используете хостовый allaigate-роутер?** Он стоит по умолчанию: `url =
+> "https://router.allaigate.com"`, `verify_tls = true`. Задайте `taxonomy_id =
+> "data-assistant"` и ключ `cortiq_…` в `CORTIQ_ROUTER_KEY`. На сложных запросах роутер эскалирует к oracle (~10 с), поэтому
 > ставьте `timeout_ms = 12000+` — иначе шлюз мягко деградирует на default-модель.
 
 ---

bench/accuracy.py

@@ -4,12 +4,12 @@ heuristic (what you get without a semantic classifier, e.g. with LiteLLM/Portkey
 
     CORTIQ_ROUTER_KEY=cortiq_... python3 bench/accuracy.py
 
-Env: CORTIQ_ROUTER_KEY (required), ROUTER_URL (default https://138.226.222.209),
+Env: CORTIQ_ROUTER_KEY (required), ROUTER_URL (default https://router.allaigate.com),
      TAXONOMY (default data-assistant).
 """
 import json, os, ssl, sys, urllib.request
 
-ROUTER_URL = os.environ.get("ROUTER_URL", "https://138.226.222.209").rstrip("/")
+ROUTER_URL = os.environ.get("ROUTER_URL", "https://router.allaigate.com").rstrip("/")
 TAXONOMY = os.environ.get("TAXONOMY", "data-assistant")
 KEY = os.environ.get("CORTIQ_ROUTER_KEY", "")
 CTX = ssl._create_unverified_context()

config/gateway.example.toml

@@ -8,13 +8,13 @@
 listen = "0.0.0.0:9000"
 
 # --- connection to cortiq-router (decides task type / complexity) -----------
-# Hosted allaigate router: url = "https://138.226.222.209", verify_tls = false
+# Hosted allaigate router: url = "https://router.allaigate.com" (the default)
 # (cert is bound to an IP), taxonomy_id = "data-assistant", a cortiq_… key in
 # CORTIQ_ROUTER_KEY.
 # NOTE: on hard prompts the router escalates to an oracle LLM (~10 s) — for the
 # hosted router set timeout_ms = 12000+, otherwise the gateway degrades to default.
 [router]
-url         = "https://138.226.222.209"   # hosted allaigate router (or your own cortiq-router URL)
+url         = "https://router.allaigate.com"  # hosted allaigate router (or your own cortiq-router URL)
 api_key_env = "CORTIQ_ROUTER_KEY"   # name of the env var holding the router key
 timeout_ms  = 12000                  # hard prompts escalate to an oracle (~10 s)
 taxonomy_id = "data-assistant"       # the hosted router's task taxonomy

docs/PROTOCOLS.md

@@ -53,7 +53,24 @@ Routing rule common to all protocols:
 ```
 
 Standard OpenAI fields are supported; unknown fields are proxied to the provider unchanged
-(forward-compatible).
+(forward-compatible), minus the bookkeeping keys chat UIs attach to the payload
+(`chat_id`, `session_id`, `metadata`, `features`, …), which are dropped so strict upstreams
+do not reject the call.
+
+The parser is deliberately lenient about the shapes real clients send:
+`content` may be a string, `null`, or an array of content parts (text parts are
+concatenated, images and other non-text parts are skipped); `tools` may be `null`;
+`reasoning_effort` may be a string (`low` | `medium` | `high`) or a token budget;
+`max_completion_tokens` is accepted as an alias for `max_tokens`. A body that still
+cannot be read comes back as an OpenAI error envelope with HTTP 400 — never a bare 422,
+which OpenAI clients cannot parse (they show no answer at all).
+
+`tools` is treated as an **offer**, not a requirement. The gateway prefers a tool-capable
+model, but when none is available it answers without the tools rather than failing, and says
+so via `X-Cortiq-Tools-Dropped: true` (chat UIs attach their built-in tools to every message,
+so the strict reading made ordinary chat impossible on a tools-less local pool).
+`tool_choice: "required"` — or a named function — is a real requirement and still fails with
+`502 no_healthy_model` when no model can honour it.
 
 ### Response (non-streaming)
 
@@ -62,12 +79,12 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
   "id": "chatcmpl-9f...",
   "object": "chat.completion",
   "created": 1718800000,
-  "model": "local-qwen",                  // the ACTUALLY selected model
+  "model": "cortiq-auto",                 // the model the CLIENT asked for, echoed back
   "choices": [
     {
       "index": 0,
       "message": { "role": "assistant", "content": "x = 2 or x = 3" },
-      "finish_reason": "stop"
+      "finish_reason": "stop"               // always one of stop|length|tool_calls|content_filter
     }
   ],
   "usage": {
@@ -86,10 +103,44 @@ Standard OpenAI fields are supported; unknown fields are proxied to the provider
 }
 ```
 
+> `model` echoes the id the client requested, exactly as OpenAI does — clients match the
+> reply against the id they sent. The model that actually answered is in
+> `cortiq.selected_model` and in the `X-Cortiq-Selected-Model` header.
+
+> Optional fields are omitted rather than sent as `null`: absent tool calls mean no
+> `tool_calls` key at all. `"tool_calls": null` is not in the OpenAI schema and trips
+> strict clients.
+
 > The `cortiq` field is an optional extension. OpenAI SDKs ignore it. If the client does
 > not want extra fields, disable `cortiq.echo` — the metadata will remain in the response
 > headers.
 
+Streaming (`stream: true`) emits strict `chat.completion.chunk` events: the stream always
+opens with a `role` delta, carries exactly one terminal `finish_reason`, and ends with
+`data: [DONE]` — even when the upstream stops early or dies mid-flight, in which case the
+failure arrives as an SSE `error` event. A client can therefore never be left waiting on a
+half-formed stream.
+
+Streaming carries no choice-less chunks unless asked: `usage` is emitted only when the
+request sets `stream_options: {"include_usage": true}`, and then exactly once, after the
+`finish_reason` chunk, with `cortiq_cost_usd` folded in. Without it a stream is
+`role → content… → tool_calls → finish_reason → [DONE]` and nothing else — the shape the
+hosted providers send.
+
+### Tool calls
+
+`tools` and `tool_calls` are proxied through, with the wire kept clean:
+
+* raw `<tool_call>…</tool_call>` markup (how Qwen-family templates express a call in the
+  generated text) is stripped from `content`, so a client never receives the same call
+  twice — once as prose and once structured. Markers split across SSE chunks are handled;
+  ordinary text containing `<` is not touched;
+* when the upstream never parsed that markup, it is promoted into a real `tool_calls`
+  entry with a generated `call_…` id, so the call is executable rather than decorative;
+* streamed entries carry `index` always, `id`/`type` once per call, and
+  `function.arguments` as a string even if the runtime sent an object;
+* a reply carrying tool calls always ends with `finish_reason: "tool_calls"`.
+
 ### Metadata in response headers (always present)
 
 ```
@@ -164,20 +215,27 @@ single user message and follows the same pipeline. Response `object` is `"text_c
 ## 4. OpenAI Models
 
 `GET /v1/models` — auto-discovery for clients. Returns the gateway's virtual models
-(`cortiq-auto`, `cortiq-embed`) and all real models in the pool.
+(`cortiq-auto`, `cortiq-embed`), every model in the `[[models]]` pool, and every managed
+local `[[cmf.servers]]` model, so a client can pin a local `.cmf` directly.
 
 ```jsonc
 {
   "object": "list",
   "data": [
-    { "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
-    { "id": "local-qwen",  "object": "model", "owned_by": "cortiq-gateway" },
-    { "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
-    { "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
+    { "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
+    { "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
+    { "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
+    { "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
+      "cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
   ]
 }
 ```
 
+Managed CMF models are listed as soon as they are configured, with their live state in the
+non-standard `cortiq` field (clients ignore it). They are not filtered by health: a large
+`.cmf` takes minutes to load, and a model that disappears from the list while it warms up
+is worse than one marked "not ready yet".
+
 ---
 
 ## 5. Anthropic Messages
@@ -198,7 +256,7 @@ single user message and follows the same pipeline. Response `object` is `"text_c
   "id": "msg_...",
   "type": "message",
   "role": "assistant",
-  "model": "local-qwen",
+  "model": "cortiq-auto",
   "content": [ { "type": "text", "text": "x = 2 or x = 3" } ],
   "stop_reason": "end_turn",
   "usage": { "input_tokens": 24, "output_tokens": 11 }

docs/PROTOCOLS.ru.md

@@ -53,7 +53,24 @@
 ```
 
 Поддерживаются стандартные поля OpenAI; неизвестные поля проксируются к провайдеру
-без изменений (forward-compatible).
+без изменений (forward-compatible) — кроме служебных ключей, которые чат-интерфейсы
+добавляют в payload для собственных нужд (`chat_id`, `session_id`, `metadata`,
+`features`, …): они отбрасываются, чтобы строгий upstream не отклонил вызов.
+
+Парсер намеренно терпим к тем формам, которые реально шлют клиенты: `content` может быть
+строкой, `null` или массивом content-частей (текстовые части склеиваются, изображения и
+прочие нетекстовые пропускаются); `tools` может быть `null`; `reasoning_effort` — строкой
+(`low` | `medium` | `high`) или бюджетом токенов; `max_completion_tokens` принимается как
+синоним `max_tokens`. Если тело всё-таки не читается, возвращается OpenAI-конверт ошибки с
+HTTP 400 — но не «голый» 422, который OpenAI-клиенты не умеют разбирать (для пользователя
+это выглядит как отсутствие ответа).
+
+`tools` трактуется как **предложение**, а не требование. Шлюз предпочитает модель с
+поддержкой tools, но если такой нет — отвечает без них, а не падает, и сообщает об этом
+заголовком `X-Cortiq-Tools-Dropped: true` (чат-интерфейсы добавляют свои встроенные tools в
+каждое сообщение, поэтому строгое прочтение делало обычный чат на локальном пуле без tools
+невозможным). `tool_choice: "required"` — или именованная функция — это уже настоящее
+требование, и оно по-прежнему завершается `502 no_healthy_model`, если его некому выполнить.
 
 ### Ответ (без стриминга)
 
@@ -62,12 +79,12 @@
   "id": "chatcmpl-9f...",
   "object": "chat.completion",
   "created": 1718800000,
-  "model": "local-qwen",                  // РЕАЛЬНО выбранная модель
+  "model": "cortiq-auto",                 // модель, которую ЗАПРОСИЛ клиент — возвращается как есть
   "choices": [
     {
       "index": 0,
       "message": { "role": "assistant", "content": "x = 2 или x = 3" },
-      "finish_reason": "stop"
+      "finish_reason": "stop"               // всегда одно из stop|length|tool_calls|content_filter
     }
   ],
   "usage": {
@@ -86,9 +103,43 @@
 }
 ```
 
+> `model` возвращает тот id, который запросил клиент, — ровно как это делает OpenAI:
+> клиенты сопоставляют ответ с отправленным id. Модель, которая реально ответила, — в
+> `cortiq.selected_model` и в заголовке `X-Cortiq-Selected-Model`.
+
+> Необязательные поля не сериализуются вместо того, чтобы приходить как `null`: если
+> tool-вызовов нет, ключа `tool_calls` в ответе нет вообще. `"tool_calls": null` не входит
+> в схему OpenAI и ломает строгие клиенты.
+
 > Поле `cortiq` — необязательное расширение. OpenAI-SDK его игнорируют. Если клиент
 > не хочет «лишних» полей — выключите `cortiq.echo`, метаданные останутся в заголовках.
 
+Стриминг (`stream: true`) отдаёт строгие события `chat.completion.chunk`: поток всегда
+открывается дельтой с `role`, содержит ровно один финальный `finish_reason` и завершается
+`data: [DONE]` — даже если upstream оборвался или упал на середине (тогда ошибка приходит
+SSE-событием `error`). Клиент никогда не остаётся ждать недосказанный поток.
+
+В стриме нет чанков без `choices`, если их не просили: `usage` отдаётся, только когда
+запрос содержит `stream_options: {"include_usage": true}`, и тогда ровно один раз — после
+чанка с `finish_reason`, с вложенным `cortiq_cost_usd`. Без него поток выглядит как
+`role → content… → tool_calls → finish_reason → [DONE]` и ничего больше — ровно так же,
+как у хостовых провайдеров.
+
+### Tool calls
+
+`tools` и `tool_calls` проксируются, но провод держится чистым:
+
+* сырая разметка `<tool_call>…</tool_call>` (так Qwen-подобные шаблоны выражают вызов
+  прямо в тексте) вырезается из `content` — клиент никогда не получает один и тот же
+  вызов дважды, прозой и структурой. Маркер, разорванный между SSE-чанками, обрабатывается
+  корректно; обычный текст с символом `<` не трогается;
+* если апстрим эту разметку не распарсил, она повышается до настоящего элемента
+  `tool_calls` со сгенерированным `call_…` id — вызов становится исполнимым, а не
+  декоративным;
+* в стриме у каждого элемента всегда есть `index`, `id`/`type` — ровно один раз на вызов,
+  а `function.arguments` — всегда строка, даже если рантайм прислал объект;
+* ответ с tool-вызовами всегда завершается `finish_reason: "tool_calls"`.
+
 ### Метаданные в заголовках (всегда)
 
 ```
@@ -163,20 +214,27 @@ data: [DONE]
 ## 4. OpenAI Models
 
 `GET /v1/models` — автодискавери для клиентов. Возвращает виртуальные модели шлюза
-(`cortiq-auto`, `cortiq-embed`) и все реальные модели пула.
+(`cortiq-auto`, `cortiq-embed`), все модели пула `[[models]]` и все управляемые локальные
+модели `[[cmf.servers]]` — чтобы клиент мог напрямую закрепить локальный `.cmf`.
 
 ```jsonc
 {
   "object": "list",
   "data": [
-    { "id": "cortiq-auto", "object": "model", "owned_by": "cortiq-gateway" },
-    { "id": "local-qwen",  "object": "model", "owned_by": "cortiq-gateway" },
-    { "id": "gpt-4o-mini", "object": "model", "owned_by": "cortiq-gateway" },
-    { "id": "claude-opus", "object": "model", "owned_by": "cortiq-gateway" }
+    { "id": "cortiq-auto", "object": "model", "created": 1718800000, "owned_by": "cortiq" },
+    { "id": "gpt-4o-mini", "object": "model", "created": 1718800000, "owned_by": "openai" },
+    { "id": "claude-opus", "object": "model", "created": 1718800000, "owned_by": "anthropic" },
+    { "id": "cmf-qwen3-4b", "object": "model", "created": 1718800000, "owned_by": "cortiq-cmf",
+      "cortiq": { "kind": "cmf", "local": true, "running": true, "healthy": true } }
   ]
 }
 ```
 
+Управляемые CMF-модели попадают в список сразу, как только настроены; их живое состояние —
+в нестандартном поле `cortiq` (клиенты его игнорируют). По здоровью они не фильтруются:
+крупный `.cmf` грузится минутами, и модель, исчезающая из списка на время прогрева, хуже
+модели, помеченной «ещё не готова».
+
 ---
 
 ## 5. Anthropic Messages

docs/ROUTER.md

@@ -41,11 +41,11 @@ minted and shown to you.
 
 ```toml
 [router]
-url         = "https://138.226.222.209"   # allaigate routing API
-verify_tls  = false                        # cert is bound to an IP
+url         = "https://router.allaigate.com"  # allaigate routing API (the default)
+verify_tls  = true                            # valid certificate — keep verification on
 taxonomy_id = "data-assistant"
-api_key_env = "CORTIQ_ROUTER_KEY"          # your cortiq_… key
-timeout_ms  = 15000                        # allow for oracle escalation on hard prompts
+api_key_env = "CORTIQ_ROUTER_KEY"             # your cortiq_… key
+timeout_ms  = 15000                           # allow for oracle escalation on hard prompts
 ```
 
 ```bash

docs/ROUTER.ru.md

@@ -40,11 +40,11 @@ Cortiq Gateway — тонкий и быстрый прокси; **интелле
 
 ```toml
 [router]
-url         = "https://138.226.222.209"   # routing API allaigate
-verify_tls  = false                        # серт привязан к IP
+url         = "https://router.allaigate.com"  # routing API allaigate (значение по умолчанию)
+verify_tls  = true                            # валидный сертификат — проверку не выключаем
 taxonomy_id = "data-assistant"
-api_key_env = "CORTIQ_ROUTER_KEY"          # ваш ключ cortiq_…
-timeout_ms  = 15000                        # запас под oracle-эскалацию на сложных запросах
+api_key_env = "CORTIQ_ROUTER_KEY"             # ваш ключ cortiq_…
+timeout_ms  = 15000                           # запас под oracle-эскалацию на сложных запросах
 ```
 
 ```bash

src/admin/mod.rs

@@ -184,6 +184,7 @@ pub fn api_routes(admin_token: String) -> Router<SharedState> {
         .route("/admin/api/logs", get(get_logs).delete(clear_logs))
         .route("/admin/api/system", get(get_system))
         .route("/admin/api/restart", post(restart_gateway))
+        .route("/admin/api/shutdown", post(shutdown_gateway))
         .route("/admin/api/bench", get(bench_list).post(bench_start))
         .route("/admin/api/config/export", get(config_export))
         .route("/admin/api/config/import", post(config_import))
@@ -869,6 +870,7 @@ async fn probe_model(State(state): State<SharedState>, Path(id): Path<String>) -
             role: "user".into(),
             content: "ping".into(),
             tool_calls: vec![],
+            ..Default::default()
         }],
         tools: vec![],
         params: GenParams {
@@ -2272,6 +2274,20 @@ async fn get_system(State(state): State<SharedState>) -> ApiResult {
     ok(serde_json::to_value(snap).unwrap_or_else(|_| json!({})))
 }
 
+/// Stop every managed local model server and power the gateway off. The 200
+/// goes out first so the SPA can show its "gateway is off" screen; nothing is
+/// respawned — the operator starts the gateway again by hand (or the Docker
+/// restart policy does, in which case this behaves like a restart).
+async fn shutdown_gateway(State(state): State<SharedState>) -> ApiResult {
+    state.cmf.stop().await;
+    tokio::spawn(async move {
+        tokio::time::sleep(std::time::Duration::from_millis(300)).await;
+        tracing::info!("shutdown requested from the admin console");
+        std::process::exit(0);
+    });
+    ok(json!({ "ok": true }))
+}
+
 /// Restart the gateway in place: stop managed model servers, spawn a fresh
 /// copy of ourselves (it retries the bind while our port frees), then exit.
 /// In Docker we only exit — we are PID 1, a spawned child would die with the

src/config.rs

@@ -318,6 +318,34 @@ fn default_verify_tls() -> bool {
     true
 }
 
+/// The hosted allaigate routing API.
+pub const DEFAULT_ROUTER_URL: &str = "https://router.allaigate.com";
+
+/// The bare IP the hosted router used to be addressed by. It served a
+/// certificate for `router.allaigate.com`, so every config pointing at it had to
+/// turn TLS verification off. The hostname now resolves, so configs written
+/// before it existed are moved over on load — the endpoint is the same machine,
+/// and this is the difference between a verified connection and a blind one.
+const LEGACY_ROUTER_HOSTS: &[&str] = &[
+    "https://138.226.222.209",
+    "http://138.226.222.209",
+    "138.226.222.209",
+];
+
+impl RouterCfg {
+    /// Point a legacy IP-addressed router config at the hostname and restore TLS
+    /// verification. Returns true when something changed.
+    fn migrate_hosted_endpoint(&mut self) -> bool {
+        let url = self.url.trim().trim_end_matches('/');
+        if !LEGACY_ROUTER_HOSTS.contains(&url) {
+            return false;
+        }
+        self.url = DEFAULT_ROUTER_URL.to_string();
+        self.verify_tls = true;
+        true
+    }
+}
+
 #[derive(Clone, Debug, Serialize, Deserialize)]
 pub struct RouteCfg {
     #[serde(default = "default_text_strategy")]
@@ -882,9 +910,9 @@ impl Default for Config {
             // models and the console asks for one; nothing breaks.
             router: RouterCfg {
                 enabled: true,
-                url: "https://138.226.222.209".to_string(),
+                url: DEFAULT_ROUTER_URL.to_string(),
                 timeout_ms: 15_000, // hard prompts escalate to an oracle (~10 s)
-                verify_tls: false,  // the hosted endpoint is IP-bound
+                verify_tls: true,
                 api_key_env: Some("CORTIQ_ROUTER_KEY".to_string()),
                 taxonomy_id: Some("data-assistant".to_string()),
             },
@@ -924,14 +952,23 @@ impl Default for Config {
 }
 
 /// Where the config lives when `--config` is not given.
-/// Repo checkout (`config/gateway.toml` exists or `config/` dir exists) → keep it there;
-/// installed binary (no repo) → `$data_dir/gateway.toml`.
+/// A readable `./config/gateway.toml` (a real gateway checkout / install dir)
+/// wins; otherwise the per-user home (`$CORTIQ_GATEWAY_HOME` or
+/// `~/.cortiq-gateway`) — so `cortiq-gateway` typed in any directory picks up
+/// the same configuration it was set up with. A foreign project that merely
+/// has `Cargo.toml` and a `config/` folder must NOT capture the config
+/// (that used to shadow the user's real config with a broken path).
 pub fn default_config_path() -> String {
     let repo_path = "config/gateway.toml";
-    if std::path::Path::new(repo_path).exists() {
+    // the file itself, readable — not just an entry that exists()
+    if std::fs::read_to_string(repo_path).is_ok() {
         return repo_path.to_string();
     }
-    if std::path::Path::new("Cargo.toml").exists() && std::path::Path::new("config").exists() {
+    // our own repo checkout before first run: config/ exists, file not yet
+    let ours = std::fs::read_to_string("Cargo.toml")
+        .map(|t| t.contains("name = \"cortiq-gateway\""))
+        .unwrap_or(false);
+    if ours && std::path::Path::new("config").is_dir() {
         return repo_path.to_string();
     }
     data_dir()
@@ -945,7 +982,14 @@ impl Config {
     pub fn load(path: &str) -> anyhow::Result<Self> {
         let raw = std::fs::read_to_string(path)
             .map_err(|e| anyhow::anyhow!("cannot read config {path}: {e}"))?;
-        let cfg: Config = toml::from_str(&raw)?;
+        let mut cfg: Config = toml::from_str(&raw)?;
+        if cfg.router.migrate_hosted_endpoint() {
+            tracing::info!(
+                url = DEFAULT_ROUTER_URL,
+                "router endpoint moved from the bare IP to the hostname; TLS verification is on"
+            );
+            let _ = cfg.save(path);
+        }
         cfg.validate()?;
         Ok(cfg)
     }
@@ -1082,6 +1126,39 @@ impl Config {
 mod tests {
     use super::*;
 
+    #[test]
+    fn the_hosted_router_defaults_to_the_hostname_with_tls_on() {
+        let cfg = Config::default();
+        assert_eq!(cfg.router.url, "https://router.allaigate.com");
+        assert!(cfg.router.verify_tls);
+    }
+
+    #[test]
+    fn a_legacy_ip_router_config_moves_to_the_hostname() {
+        for legacy in [
+            "https://138.226.222.209",
+            "https://138.226.222.209/",
+            "138.226.222.209",
+        ] {
+            let mut r = Config::default().router;
+            r.url = legacy.to_string();
+            r.verify_tls = false;
+            assert!(r.migrate_hosted_endpoint(), "{legacy} must migrate");
+            assert_eq!(r.url, DEFAULT_ROUTER_URL);
+            assert!(r.verify_tls, "TLS verification must be restored");
+        }
+    }
+
+    #[test]
+    fn a_self_hosted_router_url_is_left_alone() {
+        let mut r = Config::default().router;
+        r.url = "http://127.0.0.1:8081".to_string();
+        r.verify_tls = false;
+        assert!(!r.migrate_hosted_endpoint());
+        assert_eq!(r.url, "http://127.0.0.1:8081");
+        assert!(!r.verify_tls);
+    }
+
     const SAMPLE: &str = r#"
 listen = "0.0.0.0:9000"
 

src/main.rs

@@ -29,6 +29,7 @@ mod secrets;
 mod state;
 mod stats;
 mod sysmon;
+mod toolcalls;
 
 use axum::{routing::get, Router};
 use clap::Parser;

src/model.rs

@@ -15,13 +15,53 @@ pub enum RoutingDirective {
     Pinned { model_id: String },
 }
 
-#[derive(Clone, Debug, Serialize, Deserialize)]
+#[derive(Clone, Debug, Default, Serialize, Deserialize)]
 pub struct Message {
     pub role: String, // system | user | assistant | tool
-    #[serde(default)]
+    /// Always a plain string internally. On the wire OpenAI clients legitimately
+    /// send `null` (assistant turns that only carry tool calls) or an array of
+    /// content parts (Open WebUI and every other multimodal UI), so
+    /// [`de_content`] flattens both shapes instead of rejecting the request.
+    #[serde(default, deserialize_with = "de_content")]
     pub content: String,
     #[serde(default, skip_serializing_if = "Vec::is_empty")]
     pub tool_calls: Vec<serde_json::Value>,
+    /// Speaker name (`messages[].name`) — proxied through when the client sends it.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub name: Option<String>,
+    /// Set on `role = "tool"` replies; dropping it breaks multi-turn tool calling.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub tool_call_id: Option<String>,
+}
+
+/// Deserialize `messages[].content` from any shape an OpenAI-compatible client
+/// may send: a string, `null`, or an array of content parts. Text parts are
+/// concatenated; non-text parts (images, audio, files) are skipped — this
+/// gateway routes text, and a picture must not turn into a 422.
+fn de_content<'de, D>(d: D) -> std::result::Result<String, D::Error>
+where
+    D: serde::Deserializer<'de>,
+{
+    use serde::Deserialize as _;
+    Ok(match serde_json::Value::deserialize(d)? {
+        serde_json::Value::Null => String::new(),
+        serde_json::Value::String(s) => s,
+        serde_json::Value::Array(parts) => {
+            let mut out = String::new();
+            for part in parts {
+                // {"type":"text","text":"..."} / {"type":"input_text","text":"..."}
+                let text = part
+                    .get("text")
+                    .and_then(|t| t.as_str())
+                    .or_else(|| part.as_str());
+                if let Some(t) = text {
+                    out.push_str(t);
+                }
+            }
+            out
+        }
+        other => other.to_string(),
+    })
 }
 
 #[derive(Clone, Debug, Default)]
@@ -73,6 +113,11 @@ pub struct RouteInfo {
     pub cost_usd: f64,
     #[serde(default)]
     pub failover: bool,
+    /// The client offered `tools` but the model that answered cannot use them,
+    /// so they were dropped from the upstream call (see `Pipeline::resolve`).
+    /// Surfaced as `X-Cortiq-Tools-Dropped` so a silent downgrade is debuggable.
+    #[serde(default)]
+    pub tools_dropped: bool,
 }
 
 #[derive(Clone, Debug, Serialize)]

src/pipeline.rs

@@ -186,7 +186,7 @@ impl Pipeline {
         // Tool support detected via the local servers' healthz handshake —
         // treated as equivalent to declared caps when filtering candidates.
         detected_tools: &std::collections::HashSet<String>,
-    ) -> Result<(RouteDecision, String, Vec<String>)> {
+    ) -> Result<(RouteDecision, String, Vec<String>, bool)> {
         // 1. Extract text for classification
         let text = self.extract_text(
             &req.messages,
@@ -277,6 +277,7 @@ impl Pipeline {
 
         // Filter candidates by capabilities (e.g. tools support)
         let needs_tools = !req.tools.is_empty();
+        let tools_mandatory = needs_tools && tool_choice_is_mandatory(&req.params.passthrough);
         let has_tools = |id: &str, prov: &std::sync::Arc<dyn crate::providers::Provider>| {
             prov.caps().tools || detected_tools.contains(id)
         };
@@ -294,6 +295,31 @@ impl Pipeline {
             }
         }
 
+        // `tools` is an OFFER in the OpenAI contract, not a demand: only
+        // `tool_choice: "required"` (or a named function) makes the call
+        // impossible without them. Chat UIs lean on that — Open WebUI attaches
+        // its ~26 built-in tools to every single message — so treating the offer
+        // as a hard requirement made an ordinary "hello" unanswerable on a pool
+        // of tools-less local models. Serve the request without the tools and
+        // flag the downgrade (`X-Cortiq-Tools-Dropped`) instead of failing.
+        let mut tools_dropped = false;
+
+        // A pin stays a pin: when the pinned model exists and merely lacks tool
+        // support, honour the pin and drop the offer rather than reporting the
+        // model as missing.
+        if final_candidates.is_empty() && needs_tools && !tools_mandatory {
+            if let RoutingDirective::Pinned { model_id } = &req.routing {
+                if live.registry.get(model_id).is_some() {
+                    final_candidates.push(model_id.clone());
+                    tools_dropped = true;
+                    tracing::info!(
+                        model = %model_id,
+                        "pinned model does not support the offered tools — serving without them"
+                    );
+                }
+            }
+        }
+
         // An explicit pin must never silently answer from a different model: if
         // the pinned id is unknown or was filtered out, fail loudly so the caller
         // knows their choice was not honoured (instead of getting a stray reply
@@ -326,7 +352,7 @@ impl Pipeline {
         // (an agentic request is worth a capable model), then the rest of the
         // pool. Coding agents (OpenCode, Cline, …) always send tools; without
         // this every low-tier verdict was a hard failure.
-        if final_candidates.is_empty() {
+        let widened_order = || -> Vec<String> {
             let mut widened: Vec<String> = Vec::new();
             for tier in ["medium", "high"] {
                 for id in live.routing.candidates(tier) {
@@ -340,7 +366,10 @@ impl Pipeline {
                     widened.push(id.clone());
                 }
             }
-            for id in widened {
+            widened
+        };
+        if final_candidates.is_empty() {
+            for id in widened_order() {
                 if let Some(prov) = live.registry.get(&id) {
                     if needs_tools && !has_tools(&id, &prov) {
                         continue;
@@ -355,20 +384,40 @@ impl Pipeline {
                 );
             }
         }
+
+        // Same relaxation for an AUTO route once every narrower pass came up empty.
+        if final_candidates.is_empty() && needs_tools && !tools_mandatory {
+            for id in widened_order() {
+                if live.registry.get(&id).is_some() {
+                    final_candidates.push(id);
+                }
+            }
+            if !final_candidates.is_empty() {
+                tools_dropped = true;
+                tracing::info!(
+                    chosen = %final_candidates[0],
+                    "no model in the pool supports the offered tools — serving without them"
+                );
+            }
+        }
+
         if final_candidates.is_empty() {
             return Err(GatewayError::UpstreamUnavailable(
                 "No healthy model available that satisfies required capabilities (e.g. tools)"
                     .to_string(),
             ));
         }
-        Ok((decision, route_source, final_candidates))
+        Ok((decision, route_source, final_candidates, tools_dropped))
     }
 
-    async fn run_inner(&self, req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
+    async fn run_inner(&self, mut req: ChatRequest, state: &SharedState) -> Result<ChatResponse> {
         let live = state.live();
         let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
-        let (decision, route_source, mut final_candidates) =
+        let (decision, route_source, mut final_candidates, tools_dropped) =
             self.resolve(&req, &live, &detected_tools).await?;
+        if tools_dropped {
+            strip_tools(&mut req);
+        }
 
         // Serving-when-promoted (gated): a promoted task-label is served by the
         // local model FIRST (failover to cloud on error). Per-request VETO uses
@@ -430,6 +479,7 @@ impl Pipeline {
                                 role: "system".into(),
                                 content: sys.clone(),
                                 tool_calls: Vec::new(),
+                                ..Default::default()
                             },
                         );
                     }
@@ -457,6 +507,7 @@ impl Pipeline {
                                 role: "system".into(),
                                 content: sys.clone(),
                                 tool_calls: Vec::new(),
+                                ..Default::default()
                             },
                         );
                     }
@@ -476,6 +527,7 @@ impl Pipeline {
                         router_request_id: decision.router_request_id.clone(),
                         cost_usd: cost,
                         failover: attempt > 0,
+                        tools_dropped,
                     };
                     return Ok(resp);
                 }
@@ -502,7 +554,7 @@ impl Pipeline {
     /// recorded when the stream completes (tokens taken from the final usage chunk).
     pub async fn run_stream(
         &self,
-        req: ChatRequest,
+        mut req: ChatRequest,
         state: &SharedState,
     ) -> Result<(RouteInfo, crate::providers::ChatStream)> {
         let started = Instant::now();
@@ -516,14 +568,17 @@ impl Pipeline {
 
         let live = state.live();
         let detected_tools = state.cmf.detected_tools.lock().unwrap().clone();
-        let (decision, route_source, final_candidates) =
+        let (decision, route_source, final_candidates, tools_dropped) =
             match self.resolve(&req, &live, &detected_tools).await {
-            Ok(t) => t,
-            Err(e) => {
-                record_error(state, &account, &protocol, &directive, started, &e);
-                return Err(e);
-            }
-        };
+                Ok(t) => t,
+                Err(e) => {
+                    record_error(state, &account, &protocol, &directive, started, &e);
+                    return Err(e);
+                }
+            };
+        if tools_dropped {
+            strip_tools(&mut req);
+        }
 
         let mut last_err = None;
         let mut attempt = 0usize;
@@ -564,6 +619,7 @@ impl Pipeline {
                                 role: "system".into(),
                                 content: sys.clone(),
                                 tool_calls: Vec::new(),
+                                ..Default::default()
                             },
                         );
                     }
@@ -591,6 +647,7 @@ impl Pipeline {
                                 role: "system".into(),
                                 content: sys.clone(),
                                 tool_calls: Vec::new(),
+                                ..Default::default()
                             },
                         );
                     }
@@ -608,6 +665,7 @@ impl Pipeline {
                         router_request_id: decision.router_request_id.clone(),
                         cost_usd: 0.0,
                         failover: attempt > 0,
+                        tools_dropped,
                     };
                     // for the client-facing usage estimate when the provider
                     // sends none: the prompt size is known here, the reply is
@@ -685,6 +743,26 @@ impl Pipeline {
     }
 }
 
+/// Remove the tool offer from a request the chosen model cannot honour. The
+/// companion fields must go too: an upstream given `tool_choice` with no
+/// `tools` answers 400.
+fn strip_tools(req: &mut ChatRequest) {
+    req.tools.clear();
+    req.params.passthrough.remove("tool_choice");
+    req.params.passthrough.remove("parallel_tool_calls");
+}
+
+/// Is the client's `tool_choice` a demand rather than a preference?
+/// `"required"` and `{"type":"function", …}` mean the caller cannot be served
+/// without tools; `"auto"`, `"none"` and an absent field are just an offer.
+fn tool_choice_is_mandatory(passthrough: &serde_json::Map<String, serde_json::Value>) -> bool {
+    match passthrough.get("tool_choice") {
+        Some(serde_json::Value::String(s)) => s.eq_ignore_ascii_case("required"),
+        Some(serde_json::Value::Object(_)) => true,
+        _ => false,
+    }
+}
+
 fn now_secs() -> u64 {
     SystemTime::now()
         .duration_since(UNIX_EPOCH)
@@ -734,6 +812,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
                 role: "assistant".into(),
                 content: hit.content.clone(),
                 tool_calls: vec![],
+                ..Default::default()
             },
             finish_reason: "stop".into(),
         }],
@@ -751,6 +830,7 @@ fn cached_response(hit: &crate::cache::CacheHit) -> ChatResponse {
             router_request_id: None,
             cost_usd: 0.0,
             failover: false,
+            tools_dropped: false,
         },
     }
 }
@@ -956,6 +1036,7 @@ fn shadow_req(model_id: &str, content: String, max_tokens: u32) -> ChatRequest {
             role: "user".to_string(),
             content,
             tool_calls: Vec::new(),
+            ..Default::default()
         }],
         tools: Vec::new(),
         params: GenParams {
@@ -1061,3 +1142,57 @@ async fn shadow_capture(
         judge_tokens,
     });
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn map(json: serde_json::Value) -> serde_json::Map<String, serde_json::Value> {
+        json.as_object().cloned().unwrap_or_default()
+    }
+
+    #[test]
+    fn an_offered_tool_list_is_not_a_demand() {
+        // Open WebUI attaches its built-in tools with no tool_choice at all.
+        assert!(!tool_choice_is_mandatory(&map(serde_json::json!({}))));
+        assert!(!tool_choice_is_mandatory(&map(
+            serde_json::json!({"tool_choice": "auto"})
+        )));
+        assert!(!tool_choice_is_mandatory(&map(
+            serde_json::json!({"tool_choice": "none"})
+        )));
+    }
+
+    #[test]
+    fn required_and_named_tool_choice_are_demands() {
+        assert!(tool_choice_is_mandatory(&map(
+            serde_json::json!({"tool_choice": "required"})
+        )));
+        assert!(tool_choice_is_mandatory(&map(
+            serde_json::json!({"tool_choice": {"type": "function", "function": {"name": "f"}}})
+        )));
+    }
+
+    #[test]
+    fn stripping_tools_also_drops_their_companion_fields() {
+        let mut req = ChatRequest {
+            routing: RoutingDirective::Auto { profile: None },
+            messages: vec![],
+            tools: vec![serde_json::json!({"type": "function"})],
+            params: GenParams {
+                passthrough: map(
+                    serde_json::json!({"tool_choice": "auto", "parallel_tool_calls": true, "top_k": 40}),
+                ),
+                ..Default::default()
+            },
+            stream: false,
+            meta: RequestMeta::default(),
+        };
+        strip_tools(&mut req);
+        assert!(req.tools.is_empty());
+        assert!(!req.params.passthrough.contains_key("tool_choice"));
+        assert!(!req.params.passthrough.contains_key("parallel_tool_calls"));
+        // unrelated passthrough must survive
+        assert!(req.params.passthrough.contains_key("top_k"));
+    }
+}

src/protocols/anthropic_messages.rs

@@ -5,7 +5,7 @@
 
 use crate::error::{GatewayError, Result};
 use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
-use crate::protocols::openai_chat::cortiq_headers;
+use crate::protocols::openai_chat::{cortiq_headers, sse_headers};
 use crate::state::SharedState;
 use axum::response::{IntoResponse, Response};
 use axum::{extract::State, routing::post, Json, Router};
@@ -84,6 +84,7 @@ async fn handler(
             role: "system".into(),
             content: sys,
             tool_calls: vec![],
+            ..Default::default()
         });
     }
     for m in &req.messages {
@@ -91,9 +92,12 @@ async fn handler(
             role: m["role"].as_str().unwrap_or("user").to_string(),
             content: content_to_string(&m["content"]),
             tool_calls: vec![],
+            ..Default::default()
         });
     }
 
+    // Echoed back as-is: clients match the reply against the id they asked for.
+    let requested_model = req.model.clone();
     let canonical = ChatRequest {
         routing: parse_routing(&req.model),
         messages,
@@ -116,13 +120,12 @@ async fn handler(
 
     if req.stream {
         let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
-        let mut headers = cortiq_headers(&info);
-        headers.insert(
-            axum::http::header::CONTENT_TYPE,
-            axum::http::HeaderValue::from_static("text/event-stream"),
-        );
-        let translated = openai_to_anthropic_sse(stream);
-        return Ok((headers, axum::body::Body::from_stream(translated)).into_response());
+        let translated = openai_to_anthropic_sse(stream, requested_model);
+        return Ok((
+            sse_headers(&info),
+            axum::body::Body::from_stream(translated),
+        )
+            .into_response());
     }
 
     let resp = state.pipeline.run(canonical, &state).await?;
@@ -136,7 +139,7 @@ async fn handler(
         "id": resp.id,
         "type": "message",
         "role": "assistant",
-        "model": resp.model_used,
+        "model": requested_model,
         "content": [{ "type": "text", "text": answer }],
         "stop_reason": "end_turn",
         "stop_sequence": serde_json::Value::Null,
@@ -152,6 +155,7 @@ async fn handler(
 /// events, so Anthropic-native clients can stream regardless of the upstream provider.
 fn openai_to_anthropic_sse(
     stream: crate::providers::ChatStream,
+    model: String,
 ) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
     use futures::StreamExt;
     async_stream::stream! {
@@ -162,7 +166,7 @@ fn openai_to_anthropic_sse(
         yield Ok(evt("message_start", serde_json::json!({
             "type": "message_start",
             "message": {"id": "msg_gw", "type": "message", "role": "assistant",
-                        "content": [], "model": "cortiq-auto", "stop_reason": null,
+                        "content": [], "model": model, "stop_reason": null,
                         "usage": {"input_tokens": 0, "output_tokens": 0}}
         })));
         yield Ok(evt("content_block_start", serde_json::json!({

src/protocols/mcp.rs

@@ -94,6 +94,7 @@ async fn handle_tool_call(
                     role: "user".into(),
                     content: prompt.to_string(),
                     tool_calls: vec![],
+                    ..Default::default()
                 }],
                 tools: vec![],
                 params: GenParams::default(),

src/protocols/openai_chat.rs

@@ -1,6 +1,17 @@
 //! Incoming adapter for OpenAI Chat Completions: `POST /v1/chat/completions`.
 //! Translates the OpenAI request body into a canonical [`ChatRequest`], passes it
 //! to the pipeline, then converts the canonical [`ChatResponse`] back to OpenAI format.
+//!
+//! Wire-compatibility rules this adapter enforces (see docs/PROTOCOLS.md §2):
+//!   * the request body is parsed leniently — every shape a real OpenAI client
+//!     sends (`content: null`, content-part arrays, `tools: null`, a string
+//!     `reasoning_effort`, UI-private extras) is accepted, and a body we truly
+//!     cannot read comes back as an OpenAI error envelope, never a bare 422;
+//!   * the response is strict OpenAI JSON — no optional `null` fields, a
+//!     spec-legal `finish_reason`, and the *requested* model id echoed back;
+//!   * the SSE stream is re-normalised chunk by chunk and always terminates
+//!     with `data: [DONE]`, so a client can never sit waiting on a half-formed
+//!     stream from an upstream that improvises.
 
 use crate::error::{GatewayError, Result};
 use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RouteInfo, RoutingDirective};
@@ -31,35 +42,119 @@ pub(crate) fn cortiq_headers(c: &RouteInfo) -> axum::http::HeaderMap {
     if let Some(id) = &c.router_request_id {
         put(&mut h, "X-Cortiq-Request-Id", id);
     }
+    if c.tools_dropped {
+        put(&mut h, "X-Cortiq-Tools-Dropped", "true");
+    }
     h
 }
 
+/// Headers every SSE response carries. `X-Accel-Buffering: no` matters when the
+/// gateway sits behind nginx/Traefik — without it the proxy buffers the whole
+/// stream and the client's chat window just spins.
+pub(crate) fn sse_headers(info: &RouteInfo) -> axum::http::HeaderMap {
+    use axum::http::HeaderValue;
+    let mut headers = cortiq_headers(info);
+    headers.insert(
+        axum::http::header::CONTENT_TYPE,
+        HeaderValue::from_static("text/event-stream; charset=utf-8"),
+    );
+    headers.insert(
+        axum::http::header::CACHE_CONTROL,
+        HeaderValue::from_static("no-cache"),
+    );
+    headers.insert("X-Accel-Buffering", HeaderValue::from_static("no"));
+    headers
+}
+
 pub fn routes() -> Router<SharedState> {
     Router::new().route("/v1/chat/completions", post(handler))
 }
 
+/// `reasoning_effort` is a string in the OpenAI spec (`low` | `medium` | `high`)
+/// but a token budget in several local runtimes. Accept both.
+#[derive(Deserialize)]
+#[serde(untagged)]
+enum Effort {
+    Budget(u32),
+    Level(String),
+}
+
+impl Effort {
+    fn to_budget(&self) -> Option<u32> {
+        match self {
+            Self::Budget(n) => Some(*n),
+            Self::Level(s) => match s.trim().to_ascii_lowercase().as_str() {
+                "none" | "off" | "minimal" => Some(0),
+                "low" => Some(512),
+                "medium" | "default" | "auto" => Some(2048),
+                "high" | "max" => Some(8192),
+                _ => None,
+            },
+        }
+    }
+}
+
 #[derive(Deserialize)]
 struct OpenAiChatRequest {
-    model: String,
+    model: Option<String>,
     messages: Vec<Message>,
     #[serde(default)]
     temperature: Option<f32>,
     #[serde(default)]
     max_tokens: Option<u32>,
+    /// The modern spelling of `max_tokens`; clients send one or the other.
+    #[serde(default)]
+    max_completion_tokens: Option<u32>,
     #[serde(default)]
     top_p: Option<f32>,
     #[serde(default)]
     think_budget: Option<u32>,
     #[serde(default)]
-    reasoning_effort: Option<u32>,
+    reasoning_effort: Option<Effort>,
+    /// `null` is a legal value here for several clients, hence `Option`.
     #[serde(default)]
-    stream: bool,
+    stream: Option<bool>,
     #[serde(default)]
-    tools: Vec<serde_json::Value>,
+    tools: Option<Vec<serde_json::Value>>,
     #[serde(flatten)]
     rest: serde_json::Map<String, serde_json::Value>,
 }
 
+/// Keys chat UIs attach to the payload for their own bookkeeping. They are not
+/// part of the OpenAI request schema, and forwarding them to a strict upstream
+/// (llama.cpp, vLLM, `cortiq serve`) is a needless way to earn a 400.
+const CLIENT_PRIVATE_KEYS: &[&str] = &[
+    "chat_id",
+    "session_id",
+    "id",
+    "metadata",
+    "background_tasks",
+    "features",
+    "variables",
+    "model_item",
+    "tool_ids",
+    "filter_ids",
+    "files",
+    "citations",
+    "params",
+    "direct",
+];
+
+/// Strip client-private bookkeeping from the passthrough map, and drop a
+/// non-string `user` (Open WebUI sends an object there for pipelines, which the
+/// OpenAI schema does not allow).
+fn sanitize_passthrough(
+    mut rest: serde_json::Map<String, serde_json::Value>,
+) -> serde_json::Map<String, serde_json::Value> {
+    for k in CLIENT_PRIVATE_KEYS {
+        rest.remove(*k);
+    }
+    if rest.get("user").map(|u| !u.is_string()).unwrap_or(false) {
+        rest.remove("user");
+    }
+    rest
+}
+
 /// Parse the `model` field value into a routing directive.
 /// `cortiq-auto[:profile]` → Auto; otherwise → Pinned(real id).
 fn parse_routing(model: &str) -> RoutingDirective {
@@ -73,10 +168,43 @@ fn parse_routing(model: &str) -> RoutingDirective {
     }
 }
 
+fn now_secs() -> u64 {
+    std::time::SystemTime::now()
+        .duration_since(std::time::UNIX_EPOCH)
+        .unwrap_or_default()
+        .as_secs()
+}
+
+/// Map whatever an upstream calls its stop reason onto the closed set the
+/// OpenAI schema defines. Clients that validate the field (LiteLLM, the
+/// official SDKs, several agent frameworks) reject anything else.
+pub(crate) fn normalize_finish_reason(raw: &str) -> &'static str {
+    match raw.trim().to_ascii_lowercase().as_str() {
+        "length" | "max_tokens" | "max_new_tokens" | "token_limit" => "length",
+        "tool_calls" | "tool_use" | "function_call" => "tool_calls",
+        "content_filter" | "safety" => "content_filter",
+        _ => "stop",
+    }
+}
+
+/// A response id every client can key on. Upstreams that omit `id` (or send an
+/// empty one) would otherwise leave the field blank, which some clients treat
+/// as a malformed completion.
+fn response_id(upstream: &str) -> String {
+    let trimmed = upstream.trim();
+    if !trimmed.is_empty() {
+        return trimmed.to_string();
+    }
+    format!("chatcmpl-{}", crate::admin::random_token(12))
+}
+
 async fn handler(
     State(state): State<SharedState>,
     acct: Option<axum::Extension<super::AccountTag>>,
-    Json(req): Json<OpenAiChatRequest>,
+    // Taken as raw bytes rather than `Json<T>`: axum's extractor rejection is a
+    // plain-text 422 that OpenAI clients cannot parse, and a client that cannot
+    // parse the failure has no answer to show — it just hangs.
+    body: axum::body::Bytes,
 ) -> Result<Response> {
     // hot protocol toggle: if the adapter is disabled in config — return 404
     if !state.live().cfg.protocols.openai_chat {
@@ -84,25 +212,53 @@ async fn handler(
             "openai_chat protocol is disabled".into(),
         ));
     }
+    let req: OpenAiChatRequest = serde_json::from_slice(&body)
+        .map_err(|e| GatewayError::InvalidRequest(format!("invalid request body: {e}")))?;
+
     if req.messages.is_empty() {
         return Err(GatewayError::InvalidRequest(
             "messages must not be empty".into(),
         ));
     }
 
+    // Echoed back verbatim: OpenAI returns the model the caller asked for, and
+    // clients (Open WebUI among them) match the reply against their own id. The
+    // model that actually answered travels in `cortiq.selected_model` and the
+    // `X-Cortiq-Selected-Model` header.
+    let requested_model = req
+        .model
+        .as_deref()
+        .map(str::trim)
+        .filter(|m| !m.is_empty())
+        .unwrap_or("cortiq-auto")
+        .to_string();
+    let stream = req.stream.unwrap_or(false);
+    // OpenAI only emits the trailing usage chunk when the client asks for it,
+    // and chat UIs do not ask. Mirroring that keeps our stream byte-shaped like
+    // every other provider's instead of carrying two extra choice-less chunks
+    // that each client has to know to ignore.
+    let include_usage = req
+        .rest
+        .get("stream_options")
+        .and_then(|o| o.get("include_usage"))
+        .and_then(|v| v.as_bool())
+        .unwrap_or(false);
+
     let canonical = ChatRequest {
-        routing: parse_routing(&req.model),
+        routing: parse_routing(&requested_model),
         messages: req.messages,
-        tools: req.tools,
+        tools: req.tools.unwrap_or_default(),
         params: GenParams {
             temperature: req.temperature,
-            max_tokens: req.max_tokens,
+            max_tokens: req.max_tokens.or(req.max_completion_tokens),
             top_p: req.top_p,
-            think_budget: req.think_budget.or(req.reasoning_effort),
+            think_budget: req
+                .think_budget
+                .or_else(|| req.reasoning_effort.as_ref().and_then(Effort::to_budget)),
             stop: Vec::new(),
-            passthrough: req.rest,
+            passthrough: sanitize_passthrough(req.rest),
         },
-        stream: req.stream,
+        stream,
         meta: RequestMeta {
             protocol: "openai_chat".into(),
             account: acct.map(|e| e.0 .0.clone()).unwrap_or_default(),
@@ -110,43 +266,51 @@ async fn handler(
         },
     };
 
-    // streaming: forward provider SSE chunks verbatim with X-Cortiq-* headers
-    if req.stream {
-        let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
-        let mut headers = cortiq_headers(&info);
-        headers.insert(
-            axum::http::header::CONTENT_TYPE,
-            axum::http::HeaderValue::from_static("text/event-stream"),
-        );
-        headers.insert(
-            axum::http::header::CACHE_CONTROL,
-            axum::http::HeaderValue::from_static("no-cache"),
-        );
-        return Ok((headers, axum::body::Body::from_stream(stream)).into_response());
+    // streaming: provider SSE is re-normalised into strict OpenAI chunks
+    if stream {
+        let (info, raw) = state.pipeline.run_stream(canonical, &state).await?;
+        let headers = sse_headers(&info);
+        let normalized = normalize_sse(raw, requested_model, include_usage);
+        return Ok((headers, axum::body::Body::from_stream(normalized)).into_response());
     }
 
     let resp = state.pipeline.run(canonical, &state).await?;
     let headers = cortiq_headers(&resp.cortiq);
 
-    let mut body = serde_json::json!({
-        "id": resp.id,
-        "object": "chat.completion",
-        "created": std::time::SystemTime::now()
-            .duration_since(std::time::UNIX_EPOCH)
-            .unwrap_or_default()
-            .as_secs(),
-        "model": resp.model_used,
-        "choices": resp.choices.iter().map(|c| {
+    let choices: Vec<serde_json::Value> = resp
+        .choices
+        .iter()
+        .map(|c| {
+            let mut message = serde_json::Map::new();
+            message.insert("role".into(), serde_json::json!(c.message.role));
+            message.insert("content".into(), serde_json::json!(c.message.content));
+            // Absent tool calls are omitted entirely — `"tool_calls": null` is
+            // not in the schema and trips strict clients.
+            if !c.message.tool_calls.is_empty() {
+                message.insert(
+                    "tool_calls".into(),
+                    serde_json::Value::Array(c.message.tool_calls.clone()),
+                );
+            }
+            let finish = if c.message.tool_calls.is_empty() {
+                normalize_finish_reason(&c.finish_reason)
+            } else {
+                "tool_calls"
+            };
             serde_json::json!({
                 "index": c.index,
-                "message": {
-                    "role": c.message.role,
-                    "content": c.message.content,
-                    "tool_calls": if c.message.tool_calls.is_empty() { serde_json::Value::Null } else { serde_json::Value::Array(c.message.tool_calls.clone()) },
-                },
-                "finish_reason": c.finish_reason,
+                "message": serde_json::Value::Object(message),
+                "finish_reason": finish,
             })
-        }).collect::<Vec<_>>(),
+        })
+        .collect();
+
+    let mut body = serde_json::json!({
+        "id": response_id(&resp.id),
+        "object": "chat.completion",
+        "created": now_secs(),
+        "model": requested_model,
+        "choices": choices,
         "usage": {
             "prompt_tokens": resp.usage.prompt_tokens,
             "completion_tokens": resp.usage.completion_tokens,
@@ -155,7 +319,7 @@ async fn handler(
     });
 
     if state.live().cfg.cortiq.echo {
-        body["cortiq"] = serde_json::json!({
+        let mut echo = serde_json::json!({
             "task_label": resp.cortiq.task_label,
             "complexity": {
                 "score": resp.cortiq.complexity_score,
@@ -163,10 +327,452 @@ async fn handler(
             },
             "selected_model": resp.cortiq.selected_model,
             "route_source": resp.cortiq.route_source,
-            "router_request_id": resp.cortiq.router_request_id,
             "cost_usd": resp.cortiq.cost_usd,
         });
+        if let Some(rid) = &resp.cortiq.router_request_id {
+            echo["router_request_id"] = serde_json::json!(rid);
+        }
+        if resp.cortiq.tools_dropped {
+            echo["tools_dropped"] = serde_json::json!(true);
+        }
+        body["cortiq"] = echo;
     }
 
     Ok((headers, Json(body)).into_response())
 }
+
+/// Rewrite an upstream SSE stream into strict OpenAI `chat.completion.chunk`
+/// events.
+///
+/// Every chunk is reissued with an id, `created`, and the *requested* model id;
+/// `message`-shaped chunks are converted to `delta`; stop reasons are mapped to
+/// the spec's closed set; empty deltas are dropped. The stream is guaranteed to
+/// open with a `role` delta, to carry exactly one terminal `finish_reason`, and
+/// to end with `data: [DONE]` — even when the upstream dies mid-flight, in
+/// which case the failure is delivered as an SSE `error` event so the client
+/// shows a message instead of waiting forever.
+///
+/// Tool calls are settled here too (see [`crate::toolcalls`]): raw
+/// `<tool_call>` markup never reaches `content`, structured deltas are brought
+/// to the OpenAI shape, markup is promoted to a real call when the upstream
+/// produced none, and a stream that emitted tool calls always finishes with
+/// `finish_reason: "tool_calls"`.
+fn normalize_sse(
+    stream: crate::providers::ChatStream,
+    model: String,
+    include_usage: bool,
+) -> impl futures::Stream<Item = Result<bytes::Bytes>> + Send + 'static {
+    use futures::StreamExt;
+    async_stream::stream! {
+        let fallback_id = response_id("");
+        let created = now_secs();
+        let mut buf = String::new();
+        let mut id: Option<String> = None;
+        // Which choice indices already carried a `role` delta (n > 1 gets one each).
+        let mut roled: std::collections::HashSet<u64> = std::collections::HashSet::new();
+        let mut sent_finish = false;
+        let mut failure: Option<String> = None;
+        // Tool-call hygiene: one markup filter per choice index, the set of
+        // tool-call indices already opened, and whether anything structured
+        // was emitted at all.
+        let mut markup: std::collections::HashMap<u64, crate::toolcalls::ToolMarkupFilter> =
+            std::collections::HashMap::new();
+        let mut tool_indices: std::collections::HashSet<u64> = std::collections::HashSet::new();
+        let mut emitted_tool_calls = false;
+        // `usage` / `cortiq_*` collected along the way, shipped after the finish.
+        let mut tail_extras: std::collections::BTreeMap<&str, serde_json::Value> =
+            std::collections::BTreeMap::new();
+
+        // Emit one normalised event; `None` means "nothing worth sending".
+        macro_rules! chunk {
+            ($choices:expr, $extra:expr) => {{
+                let mut evt = serde_json::json!({
+                    "id": id.clone().unwrap_or_else(|| fallback_id.clone()),
+                    "object": "chat.completion.chunk",
+                    "created": created,
+                    "model": model,
+                    "choices": $choices,
+                });
+                for (k, v) in $extra {
+                    evt[k] = v;
+                }
+                bytes::Bytes::from(format!("data: {evt}\n\n"))
+            }};
+        }
+
+        futures::pin_mut!(stream);
+        'outer: while let Some(item) = stream.next().await {
+            let bytes = match item {
+                Ok(b) => b,
+                Err(e) => {
+                    failure = Some(e.to_string());
+                    break 'outer;
+                }
+            };
+            buf.push_str(&String::from_utf8_lossy(&bytes));
+
+            while let Some(idx) = buf.find("\n\n") {
+                let event: String = buf.drain(..idx + 2).collect();
+                for line in event.lines() {
+                    let Some(data) = line.trim_start().strip_prefix("data:") else { continue };
+                    let data = data.trim();
+                    if data.is_empty() || data == "[DONE]" {
+                        continue;
+                    }
+                    let Ok(v) = serde_json::from_str::<serde_json::Value>(data) else { continue };
+
+                    // An upstream error delivered inside the stream: pass it on
+                    // and stop — the client needs to see it, not time out.
+                    if let Some(err) = v.get("error").filter(|e| !e.is_null()) {
+                        failure = Some(err.to_string());
+                        break 'outer;
+                    }
+
+                    if id.is_none() {
+                        id = v["id"].as_str().filter(|s| !s.is_empty()).map(str::to_string);
+                    }
+
+                    let mut choices = Vec::new();
+                    if let Some(arr) = v["choices"].as_array() {
+                        for (i, c) in arr.iter().enumerate() {
+                            // Some upstreams send a full `message` per chunk
+                            // instead of a `delta`; both mean the same thing here.
+                            let src = if c["delta"].is_object() { &c["delta"] } else { &c["message"] };
+                            let index = c["index"].as_u64().unwrap_or(i as u64);
+                            let mut delta = serde_json::Map::new();
+                            if roled.insert(index) {
+                                delta.insert("role".into(), serde_json::json!("assistant"));
+                            }
+                            if let Some(text) = src["content"].as_str() {
+                                // Strip any `<tool_call>` markup before it can
+                                // reach the client as prose — the structured
+                                // call is the only form a tool loop may see.
+                                let visible = markup
+                                    .entry(index)
+                                    .or_default()
+                                    .push(text);
+                                if !visible.is_empty() {
+                                    delta.insert("content".into(), serde_json::json!(visible));
+                                }
+                            }
+                            // Reasoning traces and citations are not in the base
+                            // schema but every client that understands them
+                            // ignores them safely, so they are worth keeping.
+                            for key in ["reasoning_content", "reasoning", "thinking", "refusal", "annotations"] {
+                                if let Some(val) = src.get(key).filter(|v| !v.is_null()) {
+                                    delta.insert(key.into(), val.clone());
+                                }
+                            }
+                            if let Some(tc) = src["tool_calls"].as_array().filter(|a| !a.is_empty()) {
+                                let normalized =
+                                    crate::toolcalls::normalize_delta_tool_calls(tc, &mut tool_indices);
+                                if !normalized.is_empty() {
+                                    emitted_tool_calls = true;
+                                    delta.insert(
+                                        "tool_calls".into(),
+                                        serde_json::Value::Array(normalized),
+                                    );
+                                }
+                            }
+
+                            let finish = c["finish_reason"].as_str().map(normalize_finish_reason);
+
+                            // Last chance to promote `<tool_call>` markup into a
+                            // real call: an upstream that never parsed it would
+                            // otherwise hand the client prose no tool loop runs.
+                            if finish.is_some() && !emitted_tool_calls {
+                                if let Some(filter) = markup.get_mut(&index) {
+                                    let leftover = filter.finish();
+                                    if !leftover.is_empty() {
+                                        let merged = match delta.get("content").and_then(|v| v.as_str()) {
+                                            Some(prev) => format!("{prev}{leftover}"),
+                                            None => leftover,
+                                        };
+                                        delta.insert("content".into(), serde_json::json!(merged));
+                                    }
+                                    let calls = crate::toolcalls::markup_to_tool_calls(filter.captured(), 0);
+                                    if !calls.is_empty() {
+                                        emitted_tool_calls = true;
+                                        let tc = serde_json::json!({
+                                            "index": index,
+                                            "delta": { "tool_calls": calls },
+                                        });
+                                        let extra: Vec<(&str, serde_json::Value)> = Vec::new();
+                                        yield Ok(chunk!(vec![tc], extra));
+                                    }
+                                }
+                            }
+
+                            if finish.is_none() && delta.is_empty() {
+                                continue;
+                            }
+                            let mut choice = serde_json::json!({
+                                "index": index,
+                                "delta": serde_json::Value::Object(delta),
+                            });
+                            if let Some(f) = finish {
+                                // A reply that carries tool calls stops for that
+                                // reason, whatever the upstream called it —
+                                // clients branch on this field to run the tool.
+                                let f = if emitted_tool_calls { "tool_calls" } else { f };
+                                choice["finish_reason"] = serde_json::json!(f);
+                                sent_finish = true;
+                            }
+                            choices.push(choice);
+                        }
+                    }
+
+                    // Terminal metadata is held back and shipped once, after the
+                    // stop reason — the order OpenAI uses. Emitting it mid-stream
+                    // (as several runtimes do) puts choice-less chunks between the
+                    // deltas and the finish, which is legal but unlike every
+                    // reference implementation.
+                    for key in ["usage", "cortiq_cost_usd", "cortiq_estimated"] {
+                        if let Some(val) = v.get(key).filter(|v| !v.is_null()) {
+                            tail_extras.insert(key, val.clone());
+                        }
+                    }
+
+                    if !choices.is_empty() {
+                        let none: Vec<(&str, serde_json::Value)> = Vec::new();
+                        yield Ok(chunk!(choices, none));
+                    }
+                }
+            }
+        }
+
+        if let Some(message) = failure {
+            let evt = serde_json::json!({
+                "error": { "message": message, "type": "upstream_unavailable", "code": "stream_failed" }
+            });
+            yield Ok(bytes::Bytes::from(format!("data: {evt}\n\n")));
+        }
+
+        // Close the stream properly whatever the upstream did: a client that
+        // never sees a stop reason and a [DONE] keeps its spinner turning.
+        if !sent_finish {
+            let mut delta = serde_json::Map::new();
+            if roled.insert(0) {
+                delta.insert("role".into(), serde_json::json!("assistant"));
+            }
+            // Held-back text and unparsed markup must not be lost with the stream.
+            for (index, filter) in markup.iter_mut() {
+                let leftover = filter.finish();
+                if !leftover.is_empty() && *index == 0 {
+                    delta.insert("content".into(), serde_json::json!(leftover));
+                }
+                if !emitted_tool_calls {
+                    let calls = crate::toolcalls::markup_to_tool_calls(filter.captured(), 0);
+                    if !calls.is_empty() {
+                        emitted_tool_calls = true;
+                        delta.insert("tool_calls".into(), serde_json::Value::Array(calls));
+                    }
+                }
+            }
+            let choices = vec![serde_json::json!({
+                "index": 0,
+                "delta": serde_json::Value::Object(delta),
+                "finish_reason": if emitted_tool_calls { "tool_calls" } else { "stop" },
+            })];
+            let extra: Vec<(&str, serde_json::Value)> = Vec::new();
+            yield Ok(chunk!(choices, extra));
+        }
+        // One trailing chunk, and only when the client asked for usage.
+        if include_usage {
+            if let Some(usage) = tail_extras.remove("usage") {
+                let mut extra: Vec<(&str, serde_json::Value)> = vec![("usage", usage)];
+                extra.extend(tail_extras.into_iter());
+                let no_choices: Vec<serde_json::Value> = Vec::new();
+                yield Ok(chunk!(no_choices, extra));
+            }
+        }
+        yield Ok(bytes::Bytes::from("data: [DONE]\n\n"));
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn content_parts_and_null_are_accepted() {
+        let body = serde_json::json!({
+            "model": "cortiq-auto",
+            "messages": [
+                {"role": "user", "content": [{"type": "text", "text": "hello "}, {"type": "image_url", "image_url": {"url": "x"}}, {"type": "text", "text": "world"}]},
+                {"role": "assistant", "content": null},
+                {"role": "tool", "content": "42", "tool_call_id": "call_1"}
+            ],
+            "tools": null,
+            "reasoning_effort": "high",
+            "chat_id": "owui-123"
+        });
+        let req: OpenAiChatRequest = serde_json::from_value(body).expect("must parse");
+        assert_eq!(req.messages[0].content, "hello world");
+        assert_eq!(req.messages[1].content, "");
+        assert_eq!(req.messages[2].tool_call_id.as_deref(), Some("call_1"));
+        assert!(req.tools.unwrap_or_default().is_empty());
+        assert_eq!(
+            req.reasoning_effort.as_ref().and_then(Effort::to_budget),
+            Some(8192)
+        );
+        assert!(!sanitize_passthrough(req.rest).contains_key("chat_id"));
+    }
+
+    #[test]
+    fn finish_reasons_map_to_the_openai_set() {
+        assert_eq!(normalize_finish_reason("max_tokens"), "length");
+        assert_eq!(normalize_finish_reason("eos"), "stop");
+        assert_eq!(normalize_finish_reason("tool_use"), "tool_calls");
+    }
+
+    /// Collect a normalised stream into one string for assertions.
+    async fn drain(chunks: Vec<&'static str>) -> String {
+        use futures::StreamExt;
+        let upstream = futures::stream::iter(
+            chunks
+                .into_iter()
+                .map(|c| Ok(bytes::Bytes::from(c)))
+                .collect::<Vec<_>>(),
+        );
+        normalize_sse(Box::pin(upstream), "cortiq-auto".into(), false)
+            .map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
+            .collect::<Vec<_>>()
+            .await
+            .join("")
+    }
+
+    #[tokio::test]
+    async fn tool_markup_is_not_duplicated_into_content() {
+        // The reported shape: the upstream streams the call as prose AND then
+        // sends it properly structured.
+        let out = drain(vec![
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Sure. <tool_call>\"}}]}\n\n",
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"{\\\"name\\\":\\\"get_weather\\\",\\\"arguments\\\":{\\\"city\\\":\\\"Paris\\\"}}\"}}]}\n\n",
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"</tool_call>\"}}]}\n\n",
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"}}]}}]}\n\n",
+            "data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
+            "data: [DONE]\n\n",
+        ])
+        .await;
+        assert!(
+            !out.contains("tool_call>"),
+            "markup leaked into content: {out}"
+        );
+        assert!(
+            out.contains("\"id\":\"call_1\""),
+            "structured call missing: {out}"
+        );
+        // exactly one structured call, and the stop reason agrees with it
+        assert_eq!(out.matches("\"tool_calls\":[").count(), 1);
+        assert!(out.contains("\"finish_reason\":\"tool_calls\""));
+        assert!(out.trim_end().ends_with("data: [DONE]"));
+    }
+
+    #[tokio::test]
+    async fn unparsed_markup_is_promoted_to_a_real_tool_call() {
+        // An upstream that never parses the markup: the call must still be
+        // executable by the client rather than arriving as prose.
+        let out = drain(vec![
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"<tool_call>{\\\"name\\\":\\\"get_weather\\\",\\\"arguments\\\":{\\\"city\\\":\\\"Paris\\\"}}</tool_call>\"}}]}\n\n",
+            "data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
+            "data: [DONE]\n\n",
+        ])
+        .await;
+        assert!(!out.contains("tool_call>"), "markup leaked: {out}");
+        assert!(
+            out.contains("\"name\":\"get_weather\""),
+            "call not promoted: {out}"
+        );
+        assert!(
+            out.contains("\"finish_reason\":\"tool_calls\""),
+            "wrong stop reason: {out}"
+        );
+    }
+
+    async fn drain_with_usage(chunks: Vec<&'static str>, include_usage: bool) -> String {
+        use futures::StreamExt;
+        let upstream = futures::stream::iter(
+            chunks
+                .into_iter()
+                .map(|c| Ok(bytes::Bytes::from(c)))
+                .collect::<Vec<_>>(),
+        );
+        normalize_sse(Box::pin(upstream), "cortiq-auto".into(), include_usage)
+            .map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
+            .collect::<Vec<_>>()
+            .await
+            .join("")
+    }
+
+    /// The upstream ships usage BEFORE the stop reason and the gateway adds a
+    /// cost chunk of its own. A client that did not ask for usage must see
+    /// neither — the same stream shape every hosted provider sends.
+    const USAGE_STREAM: &[&str] = &[
+        "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"}}]}\n\n",
+        "data: {\"choices\":[],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":1,\"total_tokens\":4}}\n\n",
+        "data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
+        "data: {\"choices\":[],\"cortiq_cost_usd\":0.0}\n\n",
+        "data: [DONE]\n\n",
+    ];
+
+    #[tokio::test]
+    async fn without_stream_options_no_choice_less_chunks_are_sent() {
+        let out = drain_with_usage(USAGE_STREAM.to_vec(), false).await;
+        assert!(!out.contains("\"usage\""), "unrequested usage chunk: {out}");
+        assert!(!out.contains("cortiq_cost_usd"), "stray cost chunk: {out}");
+        assert!(
+            !out.contains("\"choices\":[]"),
+            "choice-less chunk leaked: {out}"
+        );
+        assert!(out.contains("\"finish_reason\":\"stop\""));
+        assert!(out.trim_end().ends_with("data: [DONE]"));
+    }
+
+    #[tokio::test]
+    async fn requested_usage_is_one_chunk_after_the_finish_reason() {
+        let out = drain_with_usage(USAGE_STREAM.to_vec(), true).await;
+        assert_eq!(
+            out.matches("\"usage\"").count(),
+            1,
+            "usage must ship once: {out}"
+        );
+        let finish = out.find("\"finish_reason\"").expect("finish chunk");
+        let usage = out.find("\"usage\"").expect("usage chunk");
+        assert!(usage > finish, "usage must follow the stop reason: {out}");
+        assert!(
+            out.contains("cortiq_cost_usd"),
+            "cost folded into usage chunk: {out}"
+        );
+        assert!(out.trim_end().ends_with("data: [DONE]"));
+    }
+
+    #[tokio::test]
+    async fn ordinary_text_is_untouched_by_the_tool_filter() {
+        let out = drain(vec![
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a < b and 2<3\"}}]}\n\n",
+            "data: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
+            "data: [DONE]\n\n",
+        ])
+        .await;
+        assert!(out.contains("a < b and 2<3"), "text mangled: {out}");
+        assert!(out.contains("\"finish_reason\":\"stop\""));
+    }
+
+    #[tokio::test]
+    async fn stream_is_closed_even_when_the_upstream_stops_early() {
+        use futures::StreamExt;
+        let upstream = futures::stream::iter(vec![Ok(bytes::Bytes::from(
+            "data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"}}]}\n\n",
+        ))]);
+        let out: Vec<String> = normalize_sse(Box::pin(upstream), "cortiq-auto".into(), false)
+            .map(|b| String::from_utf8_lossy(&b.unwrap()).to_string())
+            .collect()
+            .await;
+        let joined = out.join("");
+        assert!(joined.contains("\"role\":\"assistant\""));
+        assert!(joined.contains("\"model\":\"cortiq-auto\""));
+        assert!(joined.contains("\"finish_reason\":\"stop\""));
+        assert!(joined.trim_end().ends_with("data: [DONE]"));
+    }
+}

src/protocols/openai_completions.rs

@@ -4,7 +4,7 @@
 
 use crate::error::{GatewayError, Result};
 use crate::model::{ChatRequest, GenParams, Message, RequestMeta, RoutingDirective};
-use crate::protocols::openai_chat::cortiq_headers;
+use crate::protocols::openai_chat::{cortiq_headers, normalize_finish_reason, sse_headers};
 use crate::state::SharedState;
 use axum::response::{IntoResponse, Response};
 use axum::{extract::State, routing::post, Json, Router};
@@ -71,12 +71,15 @@ async fn handler(
         ));
     }
 
+    // Echoed back as-is: clients match the reply against the id they asked for.
+    let requested_model = req.model.clone();
     let canonical = ChatRequest {
         routing: parse_routing(&req.model),
         messages: vec![Message {
             role: "user".into(),
             content: prompt,
             tool_calls: vec![],
+            ..Default::default()
         }],
         tools: vec![],
         params: GenParams {
@@ -97,13 +100,8 @@ async fn handler(
 
     if req.stream {
         let (info, stream) = state.pipeline.run_stream(canonical, &state).await?;
-        let mut headers = cortiq_headers(&info);
-        headers.insert(
-            axum::http::header::CONTENT_TYPE,
-            axum::http::HeaderValue::from_static("text/event-stream"),
-        );
         return Ok((
-            headers,
+            sse_headers(&info),
             axum::body::Body::from_stream(chat_to_completions_sse(stream)),
         )
             .into_response());
@@ -115,11 +113,12 @@ async fn handler(
         .first()
         .map(|c| c.message.content.clone())
         .unwrap_or_default();
-    let finish = resp
-        .choices
-        .first()
-        .map(|c| c.finish_reason.clone())
-        .unwrap_or_else(|| "stop".into());
+    let finish = normalize_finish_reason(
+        resp.choices
+            .first()
+            .map(|c| c.finish_reason.as_str())
+            .unwrap_or("stop"),
+    );
     let headers = cortiq_headers(&resp.cortiq);
     let body = serde_json::json!({
         "id": resp.id,
@@ -128,7 +127,7 @@ async fn handler(
             .duration_since(std::time::UNIX_EPOCH)
             .unwrap_or_default()
             .as_secs(),
-        "model": resp.model_used,
+        "model": requested_model,
         "choices": [{ "text": answer, "index": 0, "logprobs": serde_json::Value::Null, "finish_reason": finish }],
         "usage": {
             "prompt_tokens": resp.usage.prompt_tokens,

src/protocols/openai_models.rs

@@ -1,5 +1,10 @@
 //! Inbound OpenAI Models adapter: `GET /v1/models`.
-//! Lists the configured pool (plus the virtual `cortiq-auto`) for client discovery.
+//! Lists everything a client can address: the virtual `cortiq-auto`, the
+//! configured `[[models]]` pool, and the managed local `[[cmf.servers]]`.
+//!
+//! The managed CMF servers belong here because they are routable model ids like
+//! any other — omitting them left OpenAI clients (Open WebUI among them) unable
+//! to pin a local model even though the gateway would happily serve it.
 
 use crate::error::{GatewayError, Result};
 use crate::state::SharedState;
@@ -9,6 +14,13 @@ pub fn routes() -> Router<SharedState> {
     Router::new().route("/v1/models", get(handler))
 }
 
+fn now_secs() -> u64 {
+    std::time::SystemTime::now()
+        .duration_since(std::time::UNIX_EPOCH)
+        .unwrap_or_default()
+        .as_secs()
+}
+
 async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse> {
     let live = state.live();
     if !live.cfg.protocols.openai_models {
@@ -16,17 +28,51 @@ async fn handler(State(state): State<SharedState>) -> Result<impl IntoResponse>
             "openai_models protocol is disabled".into(),
         ));
     }
+    let created = now_secs();
     let mut data = vec![serde_json::json!({
         "id": "cortiq-auto",
         "object": "model",
+        "created": created,
         "owned_by": "cortiq",
     })];
+    let mut seen: std::collections::HashSet<String> =
+        std::collections::HashSet::from(["cortiq-auto".to_string()]);
+
     for m in &live.cfg.models {
+        if !seen.insert(m.id.clone()) {
+            continue;
+        }
         data.push(serde_json::json!({
             "id": m.id,
             "object": "model",
+            "created": created,
             "owned_by": m.provider,
         }));
     }
+
+    // Managed local CMF servers. Health is reported alongside (a non-standard
+    // field clients ignore) rather than used as a filter: a large `.cmf` takes
+    // minutes to load, and a model that vanishes from the list while it warms up
+    // is worse than one marked "not ready yet".
+    let cmf = state.cmf.status();
+    for s in live.cfg.cmf.effective_servers() {
+        if !seen.insert(s.id.clone()) {
+            continue; // a static [[models]] entry with the same id wins
+        }
+        let status = cmf.servers.iter().find(|st| st.id == s.id);
+        data.push(serde_json::json!({
+            "id": s.id,
+            "object": "model",
+            "created": created,
+            "owned_by": "cortiq-cmf",
+            "cortiq": {
+                "kind": "cmf",
+                "local": true,
+                "running": status.map(|st| st.running).unwrap_or(false),
+                "healthy": status.map(|st| st.healthy).unwrap_or(false),
+            },
+        }));
+    }
+
     Ok(Json(serde_json::json!({ "object": "list", "data": data })))
 }

src/providers/anthropic.rs

@@ -146,6 +146,7 @@ impl Provider for AnthropicProvider {
                     role: "assistant".into(),
                     content,
                     tool_calls: vec![],
+                    ..Default::default()
                 },
                 finish_reason: v["stop_reason"].as_str().unwrap_or("stop").to_string(),
             }],
@@ -163,6 +164,7 @@ impl Provider for AnthropicProvider {
                 router_request_id: None,
                 cost_usd: 0.0,
                 failover: false,
+                tools_dropped: false,
             },
         })
     }

src/providers/openai.rs

@@ -54,13 +54,11 @@ impl OpenAiProvider {
         if let Some(m) = req.params.max_tokens {
             body["max_tokens"] = m.into();
         }
-        eprintln!("[to_wire] think_budget={:?}", req.params.think_budget);
         if let Some(tb) = req.params.think_budget {
             body["think_budget"] = tb.into();
             body["reasoning_effort"] = tb.into();
             // cortiq serve uses enable_thinking (bool): 0 = off, >0 = on
             body["enable_thinking"] = (tb > 0).into();
-            eprintln!("[to_wire] SET enable_thinking={}", tb > 0);
         }
         if !req.tools.is_empty() {
             body["tools"] = serde_json::Value::Array(req.tools.clone());
@@ -69,10 +67,6 @@ impl OpenAiProvider {
         for (k, v) in &req.params.passthrough {
             body[k] = v.clone();
         }
-        eprintln!(
-            "[to_wire] enable_thinking={:?}",
-            body.get("enable_thinking")
-        );
         body
     }
 }
@@ -128,21 +122,36 @@ impl Provider for OpenAiProvider {
         for (i, c) in choices_arr.iter().enumerate() {
             let msg_val = &c["message"];
             let role = msg_val["role"].as_str().unwrap_or("assistant").to_string();
-            let content = msg_val["content"].as_str().unwrap_or("").to_string();
+            let raw_content = msg_val["content"].as_str().unwrap_or("");
+            // Qwen-style templates put the call in the text as `<tool_call>…`.
+            // Strip it: the structured array is the only form a client may act
+            // on, and prose that merely looks like a call is worse than none.
+            let (content, markup) = crate::toolcalls::split_tool_markup(raw_content);
 
-            let tool_calls = msg_val["tool_calls"]
+            let mut tool_calls = msg_val["tool_calls"]
                 .as_array()
                 .cloned()
                 .unwrap_or_default();
+            if tool_calls.is_empty() {
+                // Upstream never parsed the markup — promote it, so the call is
+                // executable instead of decorative.
+                tool_calls = crate::toolcalls::markup_to_tool_calls(&markup, 0);
+            }
 
+            let tool_calls_present = !tool_calls.is_empty();
             choices.push(crate::model::Choice {
                 index: c["index"].as_u64().unwrap_or(i as u64) as u32,
                 message: crate::model::Message {
                     role,
                     content,
                     tool_calls,
+                    ..Default::default()
+                },
+                finish_reason: if tool_calls_present {
+                    "tool_calls".to_string()
+                } else {
+                    c["finish_reason"].as_str().unwrap_or("stop").to_string()
                 },
-                finish_reason: c["finish_reason"].as_str().unwrap_or("stop").to_string(),
             });
         }
 
@@ -172,6 +181,7 @@ impl Provider for OpenAiProvider {
                 router_request_id: None,
                 cost_usd: 0.0,
                 failover: false,
+                tools_dropped: false,
             },
         })
     }

src/toolcalls.rs

@@ -0,0 +1,300 @@
+//! Tool-call hygiene for OpenAI-compatible responses.
+//!
+//! Qwen-family chat templates express a function call as literal markup in the
+//! generated text:
+//!
+//! ```text
+//! <tool_call>
+//! {"name": "get_weather", "arguments": {"city": "Paris"}}
+//! </tool_call>
+//! ```
+//!
+//! Runtimes handle that inconsistently. Some parse it into structured
+//! `tool_calls` **and** still stream the raw markup as `content` — the client
+//! then sees the call twice, once as prose it must not show and once as the real
+//! thing, and tool-loop implementations (Open WebUI's among them) get confused.
+//! Others never parse it at all, so a perfectly good call reaches the client as
+//! text no tool loop will ever execute.
+//!
+//! This module makes the gateway the place where that is settled: markup is
+//! stripped from the text, and when nothing structured came with it the markup
+//! is promoted into a real `tool_calls` array.
+
+const OPEN: &str = "<tool_call>";
+const CLOSE: &str = "</tool_call>";
+
+/// Longest suffix of `s` that is a proper prefix of `pat`. Used to hold back the
+/// tail of a streamed chunk that might be the beginning of a marker split across
+/// chunk boundaries (`"<tool"` + `"_call>"`).
+fn partial_marker_len(s: &str, pat: &str) -> usize {
+    let max = pat.len().saturating_sub(1).min(s.len());
+    (1..=max)
+        .rev()
+        .find(|&n| {
+            s.is_char_boundary(s.len() - n)
+                && pat.as_bytes().starts_with(&s.as_bytes()[s.len() - n..])
+        })
+        .unwrap_or(0)
+}
+
+/// Incremental `<tool_call>` filter for SSE deltas.
+///
+/// Feed it content fragments in order; it returns the text that is safe to
+/// forward and keeps the markup payloads for [`Self::captured`].
+#[derive(Default)]
+pub struct ToolMarkupFilter {
+    /// Text held back: either an unterminated marker prefix or the inside of an
+    /// open `<tool_call>` block.
+    pending: String,
+    inside: bool,
+    captured: Vec<String>,
+}
+
+impl ToolMarkupFilter {
+    pub fn new() -> Self {
+        Self::default()
+    }
+
+    /// Consume one content fragment; returns the part that may be forwarded now.
+    pub fn push(&mut self, chunk: &str) -> String {
+        self.pending.push_str(chunk);
+        let mut out = String::new();
+        loop {
+            if self.inside {
+                let Some(end) = self.pending.find(CLOSE) else {
+                    // still inside the block — nothing to forward
+                    return out;
+                };
+                let payload: String = self.pending[..end].to_string();
+                self.captured.push(payload);
+                self.pending = self.pending[end + CLOSE.len()..].to_string();
+                self.inside = false;
+                continue;
+            }
+            match self.pending.find(OPEN) {
+                Some(start) => {
+                    out.push_str(&self.pending[..start]);
+                    self.pending = self.pending[start + OPEN.len()..].to_string();
+                    self.inside = true;
+                }
+                None => {
+                    // Hold back a trailing partial marker so a split like
+                    // "…<tool" / "_call>…" is not forwarded as visible text.
+                    let hold = partial_marker_len(&self.pending, OPEN);
+                    let cut = self.pending.len() - hold;
+                    out.push_str(&self.pending[..cut]);
+                    self.pending = self.pending[cut..].to_string();
+                    return out;
+                }
+            }
+        }
+    }
+
+    /// End of stream: an unterminated block was never a tool call after all, so
+    /// give the text back rather than swallowing output.
+    pub fn finish(&mut self) -> String {
+        let mut out = String::new();
+        if self.inside {
+            out.push_str(OPEN);
+            self.inside = false;
+        }
+        out.push_str(&std::mem::take(&mut self.pending));
+        out
+    }
+
+    /// Payloads found between `<tool_call>` and `</tool_call>`.
+    pub fn captured(&self) -> &[String] {
+        &self.captured
+    }
+}
+
+/// Strip every complete `<tool_call>` block from a finished string.
+/// Returns the cleaned text and the payloads that were removed.
+pub fn split_tool_markup(text: &str) -> (String, Vec<String>) {
+    let mut f = ToolMarkupFilter::new();
+    let mut out = f.push(text);
+    out.push_str(&f.finish());
+    (out, f.captured)
+}
+
+/// Turn `<tool_call>` payloads into OpenAI `tool_calls` entries, numbered from
+/// `start_index`. Payloads that are not a JSON object with a `name` are skipped —
+/// a stray marker must not become a bogus call.
+pub fn markup_to_tool_calls(payloads: &[String], start_index: usize) -> Vec<serde_json::Value> {
+    let mut out = Vec::new();
+    for payload in payloads {
+        let Ok(v) = serde_json::from_str::<serde_json::Value>(payload.trim()) else {
+            continue;
+        };
+        let Some(name) = v.get("name").and_then(|n| n.as_str()) else {
+            continue;
+        };
+        // `arguments` is a JSON string on the wire; templates emit an object.
+        let arguments = match v.get("arguments").or_else(|| v.get("parameters")) {
+            Some(serde_json::Value::String(s)) => s.clone(),
+            Some(other) => other.to_string(),
+            None => "{}".to_string(),
+        };
+        out.push(serde_json::json!({
+            "index": start_index + out.len(),
+            "id": new_call_id(),
+            "type": "function",
+            "function": { "name": name, "arguments": arguments },
+        }));
+    }
+    out
+}
+
+pub fn new_call_id() -> String {
+    format!("call_{}", crate::admin::random_token(12))
+}
+
+/// Bring one streamed `tool_calls` array up to the OpenAI shape.
+///
+/// `seen` tracks which choice-local indices already had their opening delta, so
+/// the identity fields (`id`, `type`) are added once and continuation deltas —
+/// which legitimately carry nothing but an argument fragment — are left alone.
+pub fn normalize_delta_tool_calls(
+    raw: &[serde_json::Value],
+    seen: &mut std::collections::HashSet<u64>,
+) -> Vec<serde_json::Value> {
+    let mut out = Vec::new();
+    for (i, tc) in raw.iter().enumerate() {
+        let index = tc.get("index").and_then(|v| v.as_u64()).unwrap_or(i as u64);
+        let mut entry = serde_json::Map::new();
+        entry.insert("index".into(), serde_json::json!(index));
+
+        let first = seen.insert(index);
+        if let Some(id) = tc
+            .get("id")
+            .and_then(|v| v.as_str())
+            .filter(|s| !s.is_empty())
+        {
+            entry.insert("id".into(), serde_json::json!(id));
+        } else if first {
+            entry.insert("id".into(), serde_json::json!(new_call_id()));
+        }
+        if first {
+            let kind = tc
+                .get("type")
+                .and_then(|v| v.as_str())
+                .unwrap_or("function");
+            entry.insert("type".into(), serde_json::json!(kind));
+        }
+
+        if let Some(f) = tc.get("function") {
+            let mut func = serde_json::Map::new();
+            if let Some(name) = f.get("name").and_then(|v| v.as_str()) {
+                func.insert("name".into(), serde_json::json!(name));
+            } else if first {
+                func.insert("name".into(), serde_json::json!(""));
+            }
+            match f.get("arguments") {
+                // Arguments are a string on the wire; some runtimes send the object.
+                Some(serde_json::Value::String(s)) => {
+                    func.insert("arguments".into(), serde_json::json!(s));
+                }
+                Some(serde_json::Value::Null) | None => {
+                    if first {
+                        func.insert("arguments".into(), serde_json::json!(""));
+                    }
+                }
+                Some(other) => {
+                    func.insert("arguments".into(), serde_json::json!(other.to_string()));
+                }
+            }
+            entry.insert("function".into(), serde_json::Value::Object(func));
+        }
+        out.push(serde_json::Value::Object(entry));
+    }
+    out
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    const CALL: &str = r#"<tool_call>
+{"name": "get_weather", "arguments": {"city": "Paris"}}
+</tool_call>"#;
+
+    #[test]
+    fn markup_is_stripped_from_finished_text() {
+        let (clean, payloads) = split_tool_markup(&format!("Sure!{CALL}"));
+        assert_eq!(clean, "Sure!");
+        assert_eq!(payloads.len(), 1);
+    }
+
+    #[test]
+    fn markup_split_across_stream_chunks_never_leaks() {
+        let mut f = ToolMarkupFilter::new();
+        let mut seen = String::new();
+        // the marker itself is broken across three fragments
+        for chunk in [
+            "Let me check. <tool",
+            "_call>",
+            "{\"name\":\"get_weather\",",
+            "\"arguments\":{\"city\":\"Paris\"}}",
+            "</tool",
+            "_call>",
+            " done",
+        ] {
+            seen.push_str(&f.push(chunk));
+        }
+        seen.push_str(&f.finish());
+        assert_eq!(seen, "Let me check.  done");
+        assert!(!seen.contains("tool_call"));
+        assert_eq!(f.captured().len(), 1);
+    }
+
+    #[test]
+    fn captured_markup_becomes_a_real_tool_call() {
+        let (_, payloads) = split_tool_markup(CALL);
+        let calls = markup_to_tool_calls(&payloads, 0);
+        assert_eq!(calls.len(), 1);
+        assert_eq!(calls[0]["type"], "function");
+        assert_eq!(calls[0]["index"], 0);
+        assert_eq!(calls[0]["function"]["name"], "get_weather");
+        // arguments must be a STRING, not an object
+        assert_eq!(
+            calls[0]["function"]["arguments"].as_str().unwrap(),
+            r#"{"city":"Paris"}"#
+        );
+        assert!(calls[0]["id"].as_str().unwrap().starts_with("call_"));
+    }
+
+    #[test]
+    fn a_stray_marker_does_not_become_a_call() {
+        let calls = markup_to_tool_calls(&["not json at all".into(), "{\"no_name\":1}".into()], 0);
+        assert!(calls.is_empty());
+    }
+
+    #[test]
+    fn an_unterminated_block_is_given_back_as_text() {
+        let mut f = ToolMarkupFilter::new();
+        let mut out = f.push("thinking <tool_call>{\"name\":\"x\"");
+        out.push_str(&f.finish());
+        assert_eq!(out, "thinking <tool_call>{\"name\":\"x\"");
+        assert!(f.captured().is_empty());
+    }
+
+    #[test]
+    fn delta_tool_calls_get_identity_once_and_object_arguments_become_strings() {
+        let mut seen = std::collections::HashSet::new();
+        let opening = normalize_delta_tool_calls(
+            &[serde_json::json!({"index": 0, "function": {"name": "f", "arguments": {"a": 1}}})],
+            &mut seen,
+        );
+        assert_eq!(opening[0]["type"], "function");
+        assert!(opening[0]["id"].as_str().unwrap().starts_with("call_"));
+        assert_eq!(opening[0]["function"]["arguments"], r#"{"a":1}"#);
+
+        // a continuation delta must not be handed a second id
+        let cont = normalize_delta_tool_calls(
+            &[serde_json::json!({"index": 0, "function": {"arguments": "\"}"}})],
+            &mut seen,
+        );
+        assert!(cont[0].get("id").is_none());
+        assert!(cont[0].get("type").is_none());
+    }
+}

web/css/styles.css

@@ -706,3 +706,8 @@ tbody tr:last-child { animation: bm-in .25s ease; }
 @media (prefers-reduced-motion: reduce) {
   .sidebar, .mobile-nav-backdrop { transition: none; }
 }
+
+/* full-screen farewell after "stop everything & power off" */
+.shutdown-screen { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 100vh; gap: 12px; text-align: center; padding: 24px; }
+.shutdown-screen .shutdown-icon { font-size: 64px; color: var(--bad); }
+.shutdown-screen .hint { max-width: 480px; }

web/js/api.js

@@ -134,6 +134,7 @@ export const api = {
   logs: (limit = 300) => j("GET", "/logs?limit=" + limit),
   clearLogs: () => j("DELETE", "/logs"),
   restart: () => j("POST", "/restart"),
+  shutdown: () => j("POST", "/shutdown"),
   benchStart: (model) => j("POST", "/bench", { model }),
   benchList: () => j("GET", "/bench"),
 

web/js/i18n/de.js

@@ -536,9 +536,13 @@ export default {
   "keys.snip.withKey": "Die Beispiele unten enthalten bereits den gerade erstellten Schlüssel.",
   "keys.snip.openMode": "Keine API-Schlüssel konfiguriert — das Gateway akzeptiert jeden Wert.",
   "keys.snip.keyPh": "dein API-Schlüssel",
-  "settings.restart": "Gateway neu starten",
-  "settings.restartConfirm": "Gateway neu starten? Laufende Anfragen brechen ab; lokale Modellserver starten ebenfalls neu.",
+  "settings.restart": "Alles neu starten",
+  "settings.restartConfirm": "Gateway und alle lokalen Modellserver neu starten? Laufende Anfragen werden abgebrochen.",
   "settings.restarting": "Neustart — die Seite lädt neu, sobald das Gateway wieder da ist…",
+  "settings.shutdown": "Alles stoppen & ausschalten",
+  "settings.shutdownConfirm": "Alle lokalen Modellserver stoppen und das Gateway ausschalten? API und dieses Panel antworten erst wieder, wenn Sie das Gateway neu starten.",
+  "settings.shutdownDone": "Das Gateway ist ausgeschaltet.",
+  "settings.shutdownHint": "Alle lokalen Modellserver sind gestoppt, der Prozess wurde beendet. Starten Sie das Gateway auf dem Server neu, um das Panel zurückzubekommen.",
   "bench.title": "Geschwindigkeit lokaler Modelle",
   "bench.hint": "Führt `cortiq bench` auf dieser Maschine aus — echte tok/s, ein Modell nach dem anderen.",
   "bench.run": "Messen",

web/js/i18n/en.js

@@ -529,9 +529,13 @@ export default {
   "keys.snip.withKey": "Snippets below include the key you just created.",
   "keys.snip.openMode": "No API keys are configured — the gateway accepts any key value.",
   "keys.snip.keyPh": "your API key",
-  "settings.restart": "Restart gateway",
-  "settings.restartConfirm": "Restart the gateway? Requests in flight will be dropped; local model servers restart too.",
+  "settings.restart": "Restart everything",
+  "settings.restartConfirm": "Restart the gateway and all local model servers? Requests in flight will be dropped.",
   "settings.restarting": "Restarting — the page reloads when the gateway is back…",
+  "settings.shutdown": "Stop everything & power off",
+  "settings.shutdownConfirm": "Stop all local model servers and power the gateway off? The API and this panel stop answering until you start the gateway again.",
+  "settings.shutdownDone": "The gateway is off.",
+  "settings.shutdownHint": "All local model servers are stopped and the process has exited. Start the gateway again on the server to bring the panel back.",
   "bench.title": "Local model speed",
   "bench.hint": "Runs `cortiq bench` on this machine — real tok/s, one model at a time.",
   "bench.run": "Measure",

web/js/i18n/es.js

@@ -536,9 +536,13 @@ export default {
   "keys.snip.withKey": "Los ejemplos de abajo ya incluyen la clave que acabas de crear.",
   "keys.snip.openMode": "No hay claves API configuradas — la pasarela acepta cualquier valor.",
   "keys.snip.keyPh": "tu clave API",
-  "settings.restart": "Reiniciar la pasarela",
-  "settings.restartConfirm": "¿Reiniciar la pasarela? Las peticiones en curso se cortarán; los servidores de modelos locales también se reinician.",
+  "settings.restart": "Reiniciar todo",
+  "settings.restartConfirm": "¿Reiniciar la pasarela y todos los servidores de modelos locales? Las solicitudes en curso se interrumpirán.",
   "settings.restarting": "Reiniciando — la página se recargará cuando la pasarela vuelva…",
+  "settings.shutdown": "Detener todo y apagar",
+  "settings.shutdownConfirm": "¿Detener todos los servidores de modelos locales y apagar la pasarela? La API y este panel dejarán de responder hasta que vuelva a iniciar la pasarela.",
+  "settings.shutdownDone": "La pasarela está apagada.",
+  "settings.shutdownHint": "Todos los servidores de modelos locales están detenidos y el proceso ha terminado. Inicie de nuevo la pasarela en el servidor para recuperar el panel.",
   "bench.title": "Velocidad de modelos locales",
   "bench.hint": "Ejecuta `cortiq bench` en esta máquina — tok/s reales, un modelo a la vez.",
   "bench.run": "Medir",

web/js/i18n/fr.js

@@ -536,9 +536,13 @@ export default {
   "keys.snip.withKey": "Les exemples ci-dessous incluent la clé que vous venez de créer.",
   "keys.snip.openMode": "Aucune clé API configurée — la passerelle accepte n'importe quelle valeur.",
   "keys.snip.keyPh": "votre clé API",
-  "settings.restart": "Redémarrer la passerelle",
-  "settings.restartConfirm": "Redémarrer la passerelle ? Les requêtes en cours seront coupées ; les serveurs de modèles locaux redémarrent aussi.",
+  "settings.restart": "Tout redémarrer",
+  "settings.restartConfirm": "Redémarrer la passerelle et tous les serveurs de modèles locaux ? Les requêtes en cours seront interrompues.",
   "settings.restarting": "Redémarrage — la page se recharge dès que la passerelle revient…",
+  "settings.shutdown": "Tout arrêter et éteindre",
+  "settings.shutdownConfirm": "Arrêter tous les serveurs de modèles locaux et éteindre la passerelle ? L'API et ce panneau ne répondront plus tant que vous n'aurez pas relancé la passerelle.",
+  "settings.shutdownDone": "La passerelle est éteinte.",
+  "settings.shutdownHint": "Tous les serveurs de modèles locaux sont arrêtés et le processus est terminé. Relancez la passerelle sur le serveur pour retrouver le panneau.",
   "bench.title": "Vitesse des modèles locaux",
   "bench.hint": "Exécute `cortiq bench` sur cette machine — de vrais tok/s, un modèle à la fois.",
   "bench.run": "Mesurer",

web/js/i18n/ru.js

@@ -528,9 +528,13 @@ export default {
   "keys.snip.withKey": "В примерах ниже уже подставлен только что созданный ключ.",
   "keys.snip.openMode": "API-ключи не настроены — шлюз принимает любое значение ключа.",
   "keys.snip.keyPh": "ваш API-ключ",
-  "settings.restart": "Перезапустить шлюз",
-  "settings.restartConfirm": "Перезапустить шлюз? Текущие запросы оборвутся; серверы локальных моделей тоже перезапустятся.",
+  "settings.restart": "Перезапустить всё",
+  "settings.restartConfirm": "Перезапустить шлюз и все серверы локальных моделей? Текущие запросы оборвутся.",
   "settings.restarting": "Перезапуск — страница обновится, когда шлюз поднимется…",
+  "settings.shutdown": "Остановить всё и выключить",
+  "settings.shutdownConfirm": "Остановить все серверы локальных моделей и выключить шлюз? API и эта панель перестанут отвечать, пока вы не запустите шлюз заново.",
+  "settings.shutdownDone": "Шлюз выключен.",
+  "settings.shutdownHint": "Все серверы локальных моделей остановлены, процесс завершён. Запустите шлюз на сервере заново, чтобы вернуть панель.",
   "bench.title": "Скорость локальных моделей",
   "bench.hint": "Запускает `cortiq bench` на этой машине — реальные tok/s, по одной модели за раз.",
   "bench.run": "Замерить",

web/js/i18n/tr.js

@@ -536,9 +536,13 @@ export default {
   "keys.snip.withKey": "Aşağıdaki örnekler az önce oluşturduğun anahtarı içeriyor.",
   "keys.snip.openMode": "API anahtarı yapılandırılmadı — ağ geçidi her değeri kabul eder.",
   "keys.snip.keyPh": "API anahtarın",
-  "settings.restart": "Ağ geçidini yeniden başlat",
-  "settings.restartConfirm": "Ağ geçidi yeniden başlatılsın mı? Devam eden istekler kesilir; yerel model sunucuları da yeniden başlar.",
+  "settings.restart": "Tümünü yeniden başlat",
+  "settings.restartConfirm": "Ağ geçidi ve tüm yerel model sunucuları yeniden başlatılsın mı? Devam eden istekler kesilecek.",
   "settings.restarting": "Yeniden başlatılıyor — ağ geçidi dönünce sayfa yenilenecek…",
+  "settings.shutdown": "Tümünü durdur ve kapat",
+  "settings.shutdownConfirm": "Tüm yerel model sunucuları durdurulup ağ geçidi kapatılsın mı? Ağ geçidini yeniden başlatana kadar API ve bu panel yanıt vermez.",
+  "settings.shutdownDone": "Ağ geçidi kapalı.",
+  "settings.shutdownHint": "Tüm yerel model sunucuları durduruldu ve süreç sonlandı. Paneli geri getirmek için ağ geçidini sunucuda yeniden başlatın.",
   "bench.title": "Yerel model hızı",
   "bench.hint": "Bu makinede `cortiq bench` çalıştırır — gerçek tok/s, aynı anda tek model.",
   "bench.run": "Ölç",

web/js/i18n/zh.js

@@ -536,9 +536,13 @@ export default {
   "keys.snip.withKey": "下面的示例已包含你刚创建的密钥。",
   "keys.snip.openMode": "未配置 API 密钥 — 网关接受任意密钥值。",
   "keys.snip.keyPh": "你的 API 密钥",
-  "settings.restart": "重启网关",
-  "settings.restartConfirm": "重启网关？进行中的请求会中断；本地模型服务器也会重启。",
+  "settings.restart": "全部重启",
+  "settings.restartConfirm": "重启网关和所有本地模型服务器？正在处理的请求将被中断。",
   "settings.restarting": "正在重启 — 网关恢复后页面会自动刷新……",
+  "settings.shutdown": "全部停止并关机",
+  "settings.shutdownConfirm": "停止所有本地模型服务器并关闭网关？在你重新启动网关之前，API 和此面板都将无法响应。",
+  "settings.shutdownDone": "网关已关闭。",
+  "settings.shutdownHint": "所有本地模型服务器已停止，进程已退出。请在服务器上重新启动网关以恢复面板。",
   "bench.title": "本地模型速度",
   "bench.hint": "在本机运行 `cortiq bench` — 真实 tok/s，一次一个模型。",
   "bench.run": "测速",

web/js/views/settings.js

@@ -559,6 +559,31 @@ export async function renderSettings() {
     }
   });
 
+  const stopBtn = h("button", { class: "btn danger" }, "\u23fb " + t("settings.shutdown"));
+  stopBtn.addEventListener("click", async () => {
+    if (!(await confirmDialog(t("settings.shutdownConfirm")))) return;
+    stopBtn.disabled = true;
+    restartBtn.disabled = true;
+    try {
+      await api.shutdown();
+      // the server is gone in ~300ms — replace the app with a static farewell
+      document.body.innerHTML = "";
+      document.body.appendChild(
+        h(
+          "div",
+          { class: "shutdown-screen" },
+          h("div", { class: "shutdown-icon" }, "\u23fb"),
+          h("h2", {}, t("settings.shutdownDone")),
+          h("p", { class: "hint" }, t("settings.shutdownHint"))
+        )
+      );
+    } catch (e) {
+      toast(String(e.message || e), "bad");
+      stopBtn.disabled = false;
+      restartBtn.disabled = false;
+    }
+  });
+
   mount(
     root,
     h(
@@ -682,7 +707,8 @@ export async function renderSettings() {
         h("button", { class: "btn", onClick: () => fileIn.click() }, t("settings.import")),
         fileIn,
         h("div", { class: "grow" }),
-        restartBtn
+        restartBtn,
+        stopBtn
       )
     )
   );