fix(tools): never deliver a tool call twice, and make an unparsed one executable
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

Qwen-family templates write a function call into the generated text as
<tool_call>{...}</tool_call>. Runtimes disagree on what to do with it: some
parse it into structured tool_calls AND still stream the raw markup as
content - the client then gets the same call twice, once as prose it must not
show and once as the real thing - while others never parse it at all, so a
perfectly good call arrives as text no tool loop will run.

Settle it in the gateway. A streaming-aware filter strips the markup from
content (handling a marker split across SSE chunks, leaving ordinary text
containing "<" alone, and handing an unterminated block back as text), and
when nothing structured came with it the markup is promoted into a real
tool_calls entry with a generated call_... id. Markup that is not a JSON
object with a name is ignored so a stray marker cannot become a bogus call.

Streamed tool_calls are also brought to the OpenAI shape: index on every
entry, id and type exactly once per call so continuation deltas carrying only
an argument fragment stay legal, and function.arguments always a string. A
reply that carries tool calls now always ends with finish_reason "tool_calls",
whatever the upstream called it. The same cleanup runs on the non-streaming
path, so every protocol adapter benefits.

Verified end to end: Open WebUI 0.11.0 -> gateway -> managed CMF/Qwen with
native function calling. The model returns get_weather({"city":"Paris"}), Open
WebUI executes the tool, feeds the result back, and the assistant answers from
it - no markup in content, finish_reason tool_calls, stream closed with [DONE].
This commit is contained in:
infosave2007
2026-08-21 18:49:19 +03:00
parent 9c15109f26
commit 9fd4bdaa85
9 changed files with 529 additions and 9 deletions
+27
View File
@@ -3,6 +3,33 @@
All notable changes to this project are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/).
## [0.2.48] - 2026-08-21
Verified end to end: Open WebUI 0.11.0 → gateway → managed CMF/Qwen, native
function calling, tool executed and its result fed back into the reply.
### Fixed
- **A tool call is never delivered twice.** Qwen-family templates write the call into the
generated text as `<tool_call>{…}</tool_call>`; some runtimes parse it into structured
`tool_calls` *and* still stream the raw markup as `content`, so the client sees the call
once as prose it must not show and once as the real thing. The gateway now strips the
markup from `content` — including when the marker is split across SSE chunks — while
passing the structured call through untouched. Ordinary text containing `<` is not
affected, and an unterminated block is handed back as text rather than swallowed.
- **`finish_reason` agrees with the reply.** A response carrying tool calls now always ends
with `finish_reason: "tool_calls"`, whatever the upstream called it — clients branch on
that field to decide whether to run the tool.
- **Streamed `tool_calls` are brought to the OpenAI shape**: `index` on every entry, `id`
and `type` exactly once per call (continuation deltas that carry only an argument
fragment are left alone, as the spec requires), and `function.arguments` always a string
even when the runtime sends an object.
### Added
- **Unparsed `<tool_call>` markup is promoted into a real call.** When the upstream never
parsed it, the gateway turns it into a proper `tool_calls` entry with a generated
`call_…` id, so the call is executable instead of decorative. Markup that is not a JSON
object with a `name` is ignored — a stray marker must not become a bogus call.
## [0.2.47] - 2026-08-21
### Changed