fix(openai): strict OpenAI wire compatibility — Open WebUI can chat again
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

/v1/chat/completions answered ordinary client payloads with a bare HTTP 422
(a plain-text body no OpenAI client can parse), so Open WebUI showed neither
an answer nor an error — it just hung. Parse the body leniently instead:
content as string/null/content-part array, tools: null, stream: null, a
string reasoning_effort, max_completion_tokens, messages[].name and
tool_call_id, and a missing Content-Type. Unreadable bodies now come back as
an OpenAI error envelope with 400.

Responses are strict OpenAI JSON: no "tool_calls": null, finish_reason mapped
to the spec's closed set (cortiq serve's max_tokens -> length), never an empty
id, and `model` echoes the id the client requested — the model that actually
answered stays in cortiq.selected_model and X-Cortiq-Selected-Model.

Streaming is re-normalised rather than forwarded verbatim: a leading role
delta, message-shaped chunks converted to delta, one terminal finish_reason,
always a trailing data: [DONE], and an SSE error event when the upstream dies
mid-flight. Adds X-Accel-Buffering: no so a fronting proxy cannot buffer the
stream into a hang.

GET /v1/models now lists managed [[cmf.servers]] models with their live
running/healthy state, so clients can pin a local .cmf directly.

Also strips client bookkeeping keys (chat_id, session_id, metadata, features,
...) from the upstream call and drops the leftover [to_wire] debug prints.
This commit is contained in:
infosave2007
2026-08-21 09:10:31 +03:00
parent 34093330f6
commit b339c2b622
15 changed files with 660 additions and 93 deletions
+42
View File
@@ -3,6 +3,48 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.45] - 2026-08-21
### Fixed
- **Open WebUI (and every strict OpenAI client) can finally chat with the gateway.**
`POST /v1/chat/completions` rejected perfectly ordinary payloads with a bare HTTP 422 —
a plain-text body no OpenAI client can parse, which is why the chat window just spun with
no answer and no error. The request parser now accepts every shape real clients send:
`content` as a string, `null`, or an array of content parts (text parts concatenated,
images skipped), `tools: null`, `stream: null`, a string `reasoning_effort`
(`low`/`medium`/`high`) as well as a numeric budget, `max_completion_tokens` as an alias
for `max_tokens`, `messages[].name` and `messages[].tool_call_id` proxied through, and a
missing `Content-Type`. A body that genuinely cannot be read now returns an OpenAI error
envelope with HTTP 400.
- **Strict OpenAI response JSON, no optional `null`s.** Absent tool calls no longer emit
`"tool_calls": null` (the key is omitted entirely); `finish_reason` is mapped to the
spec's closed set, so `cortiq serve`'s `max_tokens` reaches clients as `length`; the
response always carries a non-empty `id`.
- **`model` echoes the id the client requested** (`cortiq-auto` stays `cortiq-auto`, not the
upstream's `cortiq`), matching OpenAI's own behaviour. The model that actually answered
stays in `cortiq.selected_model` and `X-Cortiq-Selected-Model`. Applies to
`/v1/chat/completions`, `/v1/completions` and `/v1/messages`.
- **Streaming is re-normalised instead of forwarded verbatim.** Every chunk is reissued as a
strict `chat.completion.chunk`: the stream opens with a `role` delta, `message`-shaped
chunks are converted to `delta`, empty deltas are dropped, and it always closes with one
terminal `finish_reason` and `data: [DONE]` — even when the upstream stops early or dies
mid-flight, in which case the failure arrives as an SSE `error` event rather than a client
waiting forever. SSE responses also carry `X-Accel-Buffering: no` so nginx/Traefik cannot
buffer the stream into a hang.
- **Client bookkeeping keys are no longer proxied upstream.** `chat_id`, `session_id`,
`metadata`, `features`, `background_tasks` and friends (plus a non-string `user`) are
stripped before the provider call, so strict backends stop rejecting the request.
### Added
- **`GET /v1/models` lists managed local CMF models.** Every `[[cmf.servers]]` entry now
appears alongside `cortiq-auto` and the `[[models]]` pool, with live `running`/`healthy`
state in a non-standard `cortiq` field, so Open WebUI and other clients can pin a local
`.cmf` directly. Entries also carry `created`.
### Changed
- Dropped leftover `[to_wire]` debug prints from the OpenAI provider — they wrote to stderr
on every single request.
## [0.2.44] - 2026-08-08
### Added