feat(cmf): per-model generation defaults; safer delete-file default
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
- Each Local-models row gains default generation params — temperature, top-p, max tokens — applied by the gateway when the incoming request doesn't set them (an explicit request value always wins). Works for both the plain and streaming paths; top_p is now forwarded to OpenAI-compatible providers. - The "also delete the model file from disk" checkbox in the delete dialog now defaults to UNCHECKED — deleting a pool entry no longer silently costs a multi-GB re-download/conversion.
This commit is contained in:
@@ -89,6 +89,14 @@ pub struct CmfServer {
|
||||
/// = force-enable on those layers.
|
||||
#[serde(default)]
|
||||
pub o1: String,
|
||||
/// Default generation params for this model — applied by the gateway when
|
||||
/// the incoming request does not set them (an explicit request value wins).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_tokens: Option<u32>,
|
||||
}
|
||||
|
||||
/// CMF-format model factory: import a HuggingFace model, convert it to a
|
||||
@@ -179,6 +187,9 @@ impl CmfCfg {
|
||||
threads: self.threads,
|
||||
gpu: self.gpu,
|
||||
o1: String::new(),
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
}];
|
||||
}
|
||||
Vec::new()
|
||||
|
||||
Reference in New Issue
Block a user