chore(release): v0.2.8 — multiple managed local models
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
Docker / publish (push) Canceled after 0s
Release / crates-io (push) Canceled after 0s
Release / release (macos-14, aarch64-apple-darwin) (push) Canceled after 0s
Release / release (ubuntu-latest, x86_64-unknown-linux-gnu) (push) Canceled after 0s
Release / release (windows-latest, x86_64-pc-windows-msvc) (push) Canceled after 0s
- The gateway can now run several local .cmf models at once: a [[cmf.servers]] list, one supervised `cortiq serve` per entry on its own port, each registered in the pool and independently routable/pinnable. The single legacy local_model/local_port still works (back-compat via effective_servers()). - Settings → Local models is now a rows editor (id · path · port + check · threads · GPU) with add/remove and per-server serving status. - Deleting a managed model stops just its server and removes its file — unless another model still points at the same file (shared-file guard).
This commit is contained in:
Vendored
+4
@@ -244,6 +244,10 @@ export default {
|
||||
"settings.cmf.threads": "Worker threads",
|
||||
"settings.cmf.gpu": "Use GPU (Metal / wgpu)",
|
||||
"settings.cmf.speedNote": "Threads parallelise the decode matvecs (~3x faster; 0 = runtime default). GPU offload mainly helps prefill of long prompts — decode is memory-bandwidth-bound, so on Apple Silicon it is roughly neutral.",
|
||||
"settings.cmf.models": "Local models",
|
||||
"settings.cmf.modelsHelp": "Each row is one model the gateway runs (its own cortiq serve on its own port) and registers in the pool. Add as many as you have .cmf files and free ports.",
|
||||
"settings.cmf.rowHint": "id · path to .cmf · port · threads · GPU. Changes apply on restart; deleting a row stops that server.",
|
||||
"settings.cmf.addServer": "+ Add local model",
|
||||
"settings.cmf.checkPort": "Check port",
|
||||
"settings.cmf.portFree": "port is free",
|
||||
"settings.cmf.portBusy": "port is in use",
|
||||
|
||||
Reference in New Issue
Block a user