fix(pipeline): capability filtering widens the chain instead of killing the request
OpenCode (like every coding agent) sends tool definitions with each request. The router rated the prompt low → the low-tier chain contained only the local model → the tools filter emptied the chain → hard failure: "No healthy model available that satisfies required capabilities", with the agent stuck in a retry loop. On an AUTO route whose filtered chain comes up empty, the pipeline now widens to any pool model satisfying the required capabilities — preferring the routing table's own medium→high order (an agentic request deserves a capable model), then the rest of the pool — and logs the widening. Pinned requests still fail loudly as before. Verified with OpenCode's exact request shape: cortiq-auto + tools at tier low now answers from deepseek in 1.2s instead of erroring.
This commit is contained in:
@@ -313,6 +313,42 @@ impl Pipeline {
|
||||
}
|
||||
}
|
||||
}
|
||||
// Capability widening: a tier chain built of tools-less local models
|
||||
// must not kill an agent request. When the filtered chain is empty on
|
||||
// an AUTO route, widen to any pool model that satisfies the required
|
||||
// capabilities — preferring the routing table's own medium→high order
|
||||
// (an agentic request is worth a capable model), then the rest of the
|
||||
// pool. Coding agents (OpenCode, Cline, …) always send tools; without
|
||||
// this every low-tier verdict was a hard failure.
|
||||
if final_candidates.is_empty() {
|
||||
let mut widened: Vec<String> = Vec::new();
|
||||
for tier in ["medium", "high"] {
|
||||
for id in live.routing.candidates(tier) {
|
||||
if !widened.contains(&id) {
|
||||
widened.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in live.registry.ids() {
|
||||
if !widened.contains(id) {
|
||||
widened.push(id.clone());
|
||||
}
|
||||
}
|
||||
for id in widened {
|
||||
if let Some(prov) = live.registry.get(&id) {
|
||||
if needs_tools && !prov.caps().tools {
|
||||
continue;
|
||||
}
|
||||
final_candidates.push(id);
|
||||
}
|
||||
}
|
||||
if !final_candidates.is_empty() {
|
||||
tracing::info!(
|
||||
chosen = %final_candidates[0],
|
||||
"capability widening: tier chain had no model with the required capabilities"
|
||||
);
|
||||
}
|
||||
}
|
||||
if final_candidates.is_empty() {
|
||||
return Err(GatewayError::UpstreamUnavailable(
|
||||
"No healthy model available that satisfies required capabilities (e.g. tools)"
|
||||
|
||||
Reference in New Issue
Block a user