fix(stream): no choice-less chunks unless the client asks for usage

Every stream used to carry two chunks with an empty choices array: usage, which the upstream emits BEFORE the stop reason, and a cortiq_cost_usd chunk the gateway added itself. OpenAI ships the usage chunk only when the request sets stream_options: {"include_usage": true}, and chat UIs do not set it - which is why the same tool, in the same Open WebUI, sees neither chunk from Groq and two from us. Honour the flag. With no stream_options a stream is now exactly role -> content... -> tool_calls -> finish_reason -> [DONE], with no choice-less chunks at all. When usage is requested it ships once, after the stop reason, with cortiq_cost_usd folded into the same chunk rather than trailing behind it. Token accounting, cost and the X-Cortiq-* headers are untouched: those numbers were never read back off the wire. Verified: the default stream matches the reference shape byte for byte, the opt-in stream matches OpenAI's, the official OpenAI SDK reassembles the streamed tool call, and the full Open WebUI 0.11.0 tool loop still runs the tool and answers from its result.
2026-08-21 21:24:11 +03:00
parent 9fd4bdaa85
commit 0b071279ac
6 changed files with 123 additions and 15 deletions
@@ -3,6 +3,22 @@
 All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/).

+## [0.2.49] - 2026-08-21
+
+### Fixed
+- **The SSE stream now has the same shape every hosted provider sends.** Two choice-less
+  chunks used to ride along on every stream: `usage` (emitted by the upstream *before* the
+  stop reason) and a `cortiq_cost_usd` chunk of the gateway's own. OpenAI ships the usage
+  chunk only when the client sets `stream_options: {"include_usage": true}` — and chat UIs
+  do not set it, which is why a Groq stream carries neither. The gateway now honours that
+  flag: with no `stream_options`, a stream is exactly
+  `role → content… → tool_calls → finish_reason → [DONE]` with no choice-less chunks at
+  all. When usage *is* requested it ships once, **after** the stop reason, with
+  `cortiq_cost_usd` folded into the same chunk instead of trailing behind it.
+
+  Token accounting, cost and the `X-Cortiq-*` headers are unchanged — the numbers were
+  never taken from those chunks.
+
 ## [0.2.48] - 2026-08-21

 Verified end to end: Open WebUI 0.11.0 → gateway → managed CMF/Qwen, native