fix(tools): never deliver a tool call twice, and make an unparsed one executable
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Qwen-family templates write a function call into the generated text as
<tool_call>{...}</tool_call>. Runtimes disagree on what to do with it: some
parse it into structured tool_calls AND still stream the raw markup as
content - the client then gets the same call twice, once as prose it must not
show and once as the real thing - while others never parse it at all, so a
perfectly good call arrives as text no tool loop will run.
Settle it in the gateway. A streaming-aware filter strips the markup from
content (handling a marker split across SSE chunks, leaving ordinary text
containing "<" alone, and handing an unterminated block back as text), and
when nothing structured came with it the markup is promoted into a real
tool_calls entry with a generated call_... id. Markup that is not a JSON
object with a name is ignored so a stray marker cannot become a bogus call.
Streamed tool_calls are also brought to the OpenAI shape: index on every
entry, id and type exactly once per call so continuation deltas carrying only
an argument fragment stay legal, and function.arguments always a string. A
reply that carries tool calls now always ends with finish_reason "tool_calls",
whatever the upstream called it. The same cleanup runs on the non-streaming
path, so every protocol adapter benefits.
Verified end to end: Open WebUI 0.11.0 -> gateway -> managed CMF/Qwen with
native function calling. The model returns get_weather({"city":"Paris"}), Open
WebUI executes the tool, feeds the result back, and the assistant answers from
it - no markup in content, finish_reason tool_calls, stream closed with [DONE].
This commit is contained in:
@@ -121,6 +121,20 @@ opens with a `role` delta, carries exactly one terminal `finish_reason`, and end
|
||||
failure arrives as an SSE `error` event. A client can therefore never be left waiting on a
|
||||
half-formed stream.
|
||||
|
||||
### Tool calls
|
||||
|
||||
`tools` and `tool_calls` are proxied through, with the wire kept clean:
|
||||
|
||||
* raw `<tool_call>…</tool_call>` markup (how Qwen-family templates express a call in the
|
||||
generated text) is stripped from `content`, so a client never receives the same call
|
||||
twice — once as prose and once structured. Markers split across SSE chunks are handled;
|
||||
ordinary text containing `<` is not touched;
|
||||
* when the upstream never parsed that markup, it is promoted into a real `tool_calls`
|
||||
entry with a generated `call_…` id, so the call is executable rather than decorative;
|
||||
* streamed entries carry `index` always, `id`/`type` once per call, and
|
||||
`function.arguments` as a string even if the runtime sent an object;
|
||||
* a reply carrying tool calls always ends with `finish_reason: "tool_calls"`.
|
||||
|
||||
### Metadata in response headers (always present)
|
||||
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user