feat(cmf): per-model generation defaults; safer delete-file default
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

- Each Local-models row gains default generation params — temperature, top-p,
  max tokens — applied by the gateway when the incoming request doesn't set
  them (an explicit request value always wins). Works for both the plain and
  streaming paths; top_p is now forwarded to OpenAI-compatible providers.
- The "also delete the model file from disk" checkbox in the delete dialog now
  defaults to UNCHECKED — deleting a pool entry no longer silently costs a
  multi-GB re-download/conversion.
This commit is contained in:
infosave2007
2026-07-14 15:22:26 +03:00
parent 46c11b9b88
commit 3c4be5e966
9 changed files with 66 additions and 9 deletions
+11
View File
@@ -89,6 +89,14 @@ pub struct CmfServer {
/// = force-enable on those layers.
#[serde(default)]
pub o1: String,
/// Default generation params for this model — applied by the gateway when
/// the incoming request does not set them (an explicit request value wins).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub temperature: Option<f32>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub top_p: Option<f32>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_tokens: Option<u32>,
}
/// CMF-format model factory: import a HuggingFace model, convert it to a
@@ -179,6 +187,9 @@ impl CmfCfg {
threads: self.threads,
gpu: self.gpu,
o1: String::new(),
temperature: None,
top_p: None,
max_tokens: None,
}];
}
Vec::new()