feat(cmf): per-model reasoning switch — 'thoughts: off' for thinking models
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Each Local-models row gains a thoughts selector (auto/off). Off injects enable_thinking=false (cortiq >= 0.2.1 renders the chat template with an empty <think> block) into requests that don't set it themselves — the model answers directly, faster and shorter. A client-supplied enable_thinking always wins.
This commit is contained in:
@@ -97,6 +97,12 @@ pub struct CmfServer {
|
||||
pub top_p: Option<f32>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
/// Reasoning switch for thinking models (Qwen3/3.5 …): "off" injects
|
||||
/// enable_thinking=false into requests that don't set it themselves —
|
||||
/// the model answers directly, skipping the <think> phase. Needs
|
||||
/// cortiq ≥ 0.2.1. Empty = model/template default.
|
||||
#[serde(default)]
|
||||
pub thinking: String,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -190,6 +196,7 @@ impl CmfCfg {
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
thinking: String::new(),
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
|
||||
Reference in New Issue
Block a user