fix(tools): never deliver a tool call twice, and make an unparsed one executable
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Qwen-family templates write a function call into the generated text as
<tool_call>{...}</tool_call>. Runtimes disagree on what to do with it: some
parse it into structured tool_calls AND still stream the raw markup as
content - the client then gets the same call twice, once as prose it must not
show and once as the real thing - while others never parse it at all, so a
perfectly good call arrives as text no tool loop will run.
Settle it in the gateway. A streaming-aware filter strips the markup from
content (handling a marker split across SSE chunks, leaving ordinary text
containing "<" alone, and handing an unterminated block back as text), and
when nothing structured came with it the markup is promoted into a real
tool_calls entry with a generated call_... id. Markup that is not a JSON
object with a name is ignored so a stray marker cannot become a bogus call.
Streamed tool_calls are also brought to the OpenAI shape: index on every
entry, id and type exactly once per call so continuation deltas carrying only
an argument fragment stay legal, and function.arguments always a string. A
reply that carries tool calls now always ends with finish_reason "tool_calls",
whatever the upstream called it. The same cleanup runs on the non-streaming
path, so every protocol adapter benefits.
Verified end to end: Open WebUI 0.11.0 -> gateway -> managed CMF/Qwen with
native function calling. The model returns get_weather({"city":"Paris"}), Open
WebUI executes the tool, feeds the result back, and the assistant answers from
it - no markup in content, finish_reason tool_calls, stream closed with [DONE].
This commit is contained in:
@@ -121,6 +121,20 @@ opens with a `role` delta, carries exactly one terminal `finish_reason`, and end
|
||||
failure arrives as an SSE `error` event. A client can therefore never be left waiting on a
|
||||
half-formed stream.
|
||||
|
||||
### Tool calls
|
||||
|
||||
`tools` and `tool_calls` are proxied through, with the wire kept clean:
|
||||
|
||||
* raw `<tool_call>…</tool_call>` markup (how Qwen-family templates express a call in the
|
||||
generated text) is stripped from `content`, so a client never receives the same call
|
||||
twice — once as prose and once structured. Markers split across SSE chunks are handled;
|
||||
ordinary text containing `<` is not touched;
|
||||
* when the upstream never parsed that markup, it is promoted into a real `tool_calls`
|
||||
entry with a generated `call_…` id, so the call is executable rather than decorative;
|
||||
* streamed entries carry `index` always, `id`/`type` once per call, and
|
||||
`function.arguments` as a string even if the runtime sent an object;
|
||||
* a reply carrying tool calls always ends with `finish_reason: "tool_calls"`.
|
||||
|
||||
### Metadata in response headers (always present)
|
||||
|
||||
```
|
||||
|
||||
@@ -119,6 +119,21 @@ HTTP 400 — но не «голый» 422, который OpenAI-клиенты
|
||||
`data: [DONE]` — даже если upstream оборвался или упал на середине (тогда ошибка приходит
|
||||
SSE-событием `error`). Клиент никогда не остаётся ждать недосказанный поток.
|
||||
|
||||
### Tool calls
|
||||
|
||||
`tools` и `tool_calls` проксируются, но провод держится чистым:
|
||||
|
||||
* сырая разметка `<tool_call>…</tool_call>` (так Qwen-подобные шаблоны выражают вызов
|
||||
прямо в тексте) вырезается из `content` — клиент никогда не получает один и тот же
|
||||
вызов дважды, прозой и структурой. Маркер, разорванный между SSE-чанками, обрабатывается
|
||||
корректно; обычный текст с символом `<` не трогается;
|
||||
* если апстрим эту разметку не распарсил, она повышается до настоящего элемента
|
||||
`tool_calls` со сгенерированным `call_…` id — вызов становится исполнимым, а не
|
||||
декоративным;
|
||||
* в стриме у каждого элемента всегда есть `index`, `id`/`type` — ровно один раз на вызов,
|
||||
а `function.arguments` — всегда строка, даже если рантайм прислал объект;
|
||||
* ответ с tool-вызовами всегда завершается `finish_reason: "tool_calls"`.
|
||||
|
||||
### Метаданные в заголовках (всегда)
|
||||
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user