feat(cmf): auto-detect tool support of managed local servers
cortiq serve >=0.5.62 advertises the loaded model's capabilities in
/healthz ({"capabilities":{"tools":true}} when the chat template has a
tools branch). The gateway now reads that handshake when the managed
server becomes healthy and treats detected tool support as equivalent
to declared caps in the candidate filter — so a low-tier chain made of
a local model no longer widens agent (tools) traffic to cloud models
once the local server can actually drive tools.
Also adds an optional manual "caps" override to [[cmf.servers]] for
older serve builds that do not advertise capabilities.
This commit is contained in:
@@ -130,6 +130,10 @@ pub struct CmfServer {
|
||||
/// Disable Multi-Token Prediction speculative decoding (CMF_MTP=0).
|
||||
#[serde(default)]
|
||||
pub skip_mtp: bool,
|
||||
/// Manual capability override (e.g. ["tools"]). Usually unnecessary: the
|
||||
/// gateway auto-detects tool support from the server's healthz handshake.
|
||||
#[serde(default)]
|
||||
pub caps: Vec<String>,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -229,6 +233,7 @@ impl CmfCfg {
|
||||
o1_window: None,
|
||||
o1_sink: None,
|
||||
skip_mtp: false,
|
||||
caps: Vec::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
|
||||
Reference in New Issue
Block a user