fix(routing): offered tools must not make a local model unroutable
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

Verified against a real Open WebUI container: it attaches its ~26 built-in
tools (knowledge bases, notes, calendar, ...) to every single message, and
resolve() treated that offer as a hard capability requirement. On a pool of
tools-less local models a plain "hello" therefore failed with "No healthy
model available that satisfies required capabilities", and a pinned local
model was reported as "not available - not in the pool (deleted?)".

In the OpenAI contract `tools` is an offer; only tool_choice: "required" (or
a named function) makes the call impossible without them. Keep preferring a
tool-capable model, but fall back to serving the request with the tools
stripped - together with tool_choice/parallel_tool_calls, which a provider
rejects on their own. A pin whose only shortcoming is tool support is now
honoured; an unknown pin and tool_choice: "required" still fail loudly.

The downgrade is never silent: X-Cortiq-Tools-Dropped: true, plus
cortiq.tools_dropped in the echo block.
This commit is contained in:
infosave2007
2026-08-21 09:59:28 +03:00
parent b339c2b622
commit f65f131c41
10 changed files with 187 additions and 10 deletions
+5
View File
@@ -113,6 +113,11 @@ pub struct RouteInfo {
pub cost_usd: f64,
#[serde(default)]
pub failover: bool,
/// The client offered `tools` but the model that answered cannot use them,
/// so they were dropped from the upstream call (see `Pipeline::resolve`).
/// Surfaced as `X-Cortiq-Tools-Dropped` so a silent downgrade is debuggable.
#[serde(default)]
pub tools_dropped: bool,
}
#[derive(Clone, Debug, Serialize)]