5 Commits

Include renames

Author	SHA1	Message	Date
infosave2007	0b071279ac	fix(stream): no choice-less chunks unless the client asks for usage ... Every stream used to carry two chunks with an empty choices array: usage, which the upstream emits BEFORE the stop reason, and a cortiq_cost_usd chunk the gateway added itself. OpenAI ships the usage chunk only when the request sets stream_options: {"include_usage": true}, and chat UIs do not set it - which is why the same tool, in the same Open WebUI, sees neither chunk from Groq and two from us. Honour the flag. With no stream_options a stream is now exactly role -> content... -> tool_calls -> finish_reason -> [DONE], with no choice-less chunks at all. When usage is requested it ships once, after the stop reason, with cortiq_cost_usd folded into the same chunk rather than trailing behind it. Token accounting, cost and the X-Cortiq-* headers are untouched: those numbers were never read back off the wire. Verified: the default stream matches the reference shape byte for byte, the opt-in stream matches OpenAI's, the official OpenAI SDK reassembles the streamed tool call, and the full Open WebUI 0.11.0 tool loop still runs the tool and answers from its result.	2026-08-21 21:24:11 +03:00
infosave2007	9fd4bdaa85	fix(tools): never deliver a tool call twice, and make an unparsed one executable ... Qwen-family templates write a function call into the generated text as <tool_call>{...}</tool_call>. Runtimes disagree on what to do with it: some parse it into structured tool_calls AND still stream the raw markup as content - the client then gets the same call twice, once as prose it must not show and once as the real thing - while others never parse it at all, so a perfectly good call arrives as text no tool loop will run. Settle it in the gateway. A streaming-aware filter strips the markup from content (handling a marker split across SSE chunks, leaving ordinary text containing "<" alone, and handing an unterminated block back as text), and when nothing structured came with it the markup is promoted into a real tool_calls entry with a generated call_... id. Markup that is not a JSON object with a name is ignored so a stray marker cannot become a bogus call. Streamed tool_calls are also brought to the OpenAI shape: index on every entry, id and type exactly once per call so continuation deltas carrying only an argument fragment stay legal, and function.arguments always a string. A reply that carries tool calls now always ends with finish_reason "tool_calls", whatever the upstream called it. The same cleanup runs on the non-streaming path, so every protocol adapter benefits. Verified end to end: Open WebUI 0.11.0 -> gateway -> managed CMF/Qwen with native function calling. The model returns get_weather({"city":"Paris"}), Open WebUI executes the tool, feeds the result back, and the assistant answers from it - no markup in content, finish_reason tool_calls, stream closed with [DONE].	2026-08-21 18:49:19 +03:00
infosave2007	f65f131c41	fix(routing): offered tools must not make a local model unroutable ... Verified against a real Open WebUI container: it attaches its ~26 built-in tools (knowledge bases, notes, calendar, ...) to every single message, and resolve() treated that offer as a hard capability requirement. On a pool of tools-less local models a plain "hello" therefore failed with "No healthy model available that satisfies required capabilities", and a pinned local model was reported as "not available - not in the pool (deleted?)". In the OpenAI contract `tools` is an offer; only tool_choice: "required" (or a named function) makes the call impossible without them. Keep preferring a tool-capable model, but fall back to serving the request with the tools stripped - together with tool_choice/parallel_tool_calls, which a provider rejects on their own. A pin whose only shortcoming is tool support is now honoured; an unknown pin and tool_choice: "required" still fail loudly. The downgrade is never silent: X-Cortiq-Tools-Dropped: true, plus cortiq.tools_dropped in the echo block.	2026-08-21 09:59:28 +03:00
infosave2007	b339c2b622	fix(openai): strict OpenAI wire compatibility — Open WebUI can chat again ... /v1/chat/completions answered ordinary client payloads with a bare HTTP 422 (a plain-text body no OpenAI client can parse), so Open WebUI showed neither an answer nor an error — it just hung. Parse the body leniently instead: content as string/null/content-part array, tools: null, stream: null, a string reasoning_effort, max_completion_tokens, messages[].name and tool_call_id, and a missing Content-Type. Unreadable bodies now come back as an OpenAI error envelope with 400. Responses are strict OpenAI JSON: no "tool_calls": null, finish_reason mapped to the spec's closed set (cortiq serve's max_tokens -> length), never an empty id, and `model` echoes the id the client requested — the model that actually answered stays in cortiq.selected_model and X-Cortiq-Selected-Model. Streaming is re-normalised rather than forwarded verbatim: a leading role delta, message-shaped chunks converted to delta, one terminal finish_reason, always a trailing data: [DONE], and an SSE error event when the upstream dies mid-flight. Adds X-Accel-Buffering: no so a fronting proxy cannot buffer the stream into a hang. GET /v1/models now lists managed [[cmf.servers]] models with their live running/healthy state, so clients can pin a local .cmf directly. Also strips client bookkeeping keys (chat_id, session_id, metadata, features, ...) from the upstream call and drops the leftover [to_wire] debug prints.	2026-08-21 09:10:31 +03:00
infosave2007	cbc385f6c7	feat: universal LLM gateway with embedded multilingual admin console ... - OpenAI-compatible gateway with intelligent routing via cortiq-router (allaigate) - Embedded build-free admin SPA at /admin: 7 languages (en/ru/de/fr/es/zh/tr), light/dark themes, hot config reload (no restart) - Admin API, per-request statistics + Prometheus /metrics, secret store, playground - English documentation (primary) with Russian translations (*.ru.md)	2026-06-30 10:51:37 +03:00