chore(release): v0.2.8 — multiple managed local models
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s

- The gateway can now run several local .cmf models at once: a [[cmf.servers]]
  list, one supervised `cortiq serve` per entry on its own port, each registered
  in the pool and independently routable/pinnable. The single legacy
  local_model/local_port still works (back-compat via effective_servers()).
- Settings → Local models is now a rows editor (id · path · port + check · threads
  · GPU) with add/remove and per-server serving status.
- Deleting a managed model stops just its server and removes its file — unless
  another model still points at the same file (shared-file guard).
This commit is contained in:
infosave2007
2026-07-09 12:00:10 +03:00
parent 9d2181e82b
commit 357441e0da
10 changed files with 405 additions and 190 deletions
+9 -9
View File
@@ -28,17 +28,17 @@ impl Registry {
by_id.insert(m.id.clone(), provider);
}
// Managed local CMF server: register it as an OpenAI-compatible provider
// so it can be routed to like any other backend — including when the
// router is unavailable or the gateway is in local-only mode.
if cfg.cmf.manage_server
&& !cfg.cmf.local_model.trim().is_empty()
&& !by_id.contains_key(&cfg.cmf.model_id)
{
// Managed local CMF servers: register each as an OpenAI-compatible
// provider so it can be routed to like any other backend — including when
// the router is unavailable or the gateway is in local-only mode.
for s in cfg.cmf.effective_servers() {
if by_id.contains_key(&s.id) {
continue; // a static [[models]] entry with the same id wins
}
let m = ModelCfg {
id: cfg.cmf.model_id.clone(),
id: s.id.clone(),
provider: "openai".into(),
base_url: format!("http://{}:{}/v1", cfg.cmf.local_host, cfg.cmf.local_port),
base_url: format!("http://{}:{}/v1", cfg.cmf.local_host, s.port),
model: "cortiq".into(),
cost_tier: "local".into(),
price_in: 0.0,